Design, Evaluation, and Analysis of Questionnaires for Survey Research, 2nd Edition

PART III

ESTIMATION AND PREDICTION OF THE QUALITY OF QUESTIONS

Until now, we have been discussing the different choices that have to be made in order to design survey items, a questionnaire, and a data collection instrument. The design of survey questionnaires can be a scientific activity if we know the quality of different possible forms of questions. A lot of research has been done to get this information.

In this part, we discuss the following steps in this process in sequence:

Chapter 9 discusses the criteria for the quality of survey questions: reliability, validity, and method effects.

In Chapter 10, we present the more classical design of MTMM experiments to estimate the reliability, validity, and method effects of survey questions.

Chapter 11 discusses the MTMM design that has been used in the European Social Survey (ESS) for estimation of the quality criteria.

In Chapter 12, we will describe the experiments that have been incorporated in the later analysis. Because the experiments of the period 1979–1997 have been described in Saris and Gallhofer (2007a), we will concentrate in this chapter on the experiments that have been done in the context of the ESS. We will pay attention to planned differences between questions and unplanned differences. We will also present some results with respect to the quality of the questions over all experiments carried out.

The quality of all possible questions cannot be studied by MTMM experiments. Therefore, the questions involved in the ...