Effects of design in web surveys: Comparing trained and fresh respondents

In this paper, we investigate whether there are differences in the effect of instrument design between trained and fresh respondents. In three experiments, we varied the number of items on a screen, the choice of response categories, and the layout of a five-point rating scale. In general, effects of design carry over between trained and fresh respondents. We found little evidence that survey experience influences the question-answering process. Trained respondents seem to be more sensitive to satisficing. The shorter completion time, higher interitem correlations for multiple-item-per-screen formats, and the fact that they select the first response options more often indicate that trained respondents tend to take shortcuts in the response process and study the questions less carefully.