Design, Evaluation, and Analysis of Questionnaires for Survey Research, 2nd Edition

11

SPLIT-BALLOT MULTITRAIT–MULTIMETHOD DESIGNS¹

Although the classical MTMM design is effective, it has one major problem, namely, that one has to ask the respondents three times nearly the same questions. As a consequence, people can become bored and answer with less seriousness as questions are repeated. It is also possible that they remember what they have said before, which means that the observed responses are not independent of each other.

In order to cope with this problem, there are two possible strategies: (1) to increase the time between the observations so that the previous answers cannot be remembered anymore or (2) to reduce the number of repeated observations. The first approach has been tried in the past. It was discussed that one can after 25 minutes repeat the same questions if similar questions are asked in between. This will be a solution for the second measures but not for the third ones. So, we have mentioned that Scherpenzeel (1995) have used in many experiments a panel design where at each point in time, only two observations of the same questions are made. However, this approach requires a panel that is commonly not available. The second strategy is to ask each respondent fewer questions while compensating for the “missing data by design” by collecting data from different subsamples of the population. In doing so, the designs look very similar to the frequently used split-ballot experiments and hence are called the “split-ballot MTMM design” or SB-MTMM design. ...