Overview
In recent studies, Lightspeed Research has observed an inverse relationship between questionnaire length and completion rates. Initial observations show decreasing completion rates as questionnaire length increases, and an increased number of respondent dropouts for lengthier surveys.
How can this affect your research? Lower completion rates and higher dropout rates may lead to lower quality data and can impact business decisions. Longer questionnaires also may cause respondent fatigue and poorer quality responses. In some cases, respondents may not give as much consideration to questions at the end of a long questionnaire as to questions at the beginning. When fatigued respondents take short-cuts in their thinking, researchers are left with response bias.
Key Research Questions
In June of 2008 Lightspeed Research fielded research-on-research to answer the following questions:
How long is too long?
What is the relationship between questionnaire length and dropout rates?
Are certain types of questions more likely to cause dropouts?
As questionnaires get longer, how does length impact the representivity of the sample?
How are key measures impacted as questionnaire length increases?
Are panelists less satisfied with the survey experience with longer questionnaires?
Does suspicious behavior increase as questionnaire length increases?
How many and what types of questions can be asked before exposure to a stimulus without impacting key measure scores?
Research Design
Six versions of a concept questionnaire were fielded for a new snack food idea. The length, as well as the order of the questions varied. The four main versions based on median completion time were:
8-minute questionnaire (ideal)
17-minute questionnaire
20-minute questionnaire
24-minute questionnaire
Two additional questionnaire versions asked extra screening questions prior to concept exposure. One version added approximately one minute and the other two minutes of interview time to the 8-minute questionnaire prior to concept exposure.
Response & Dropout Rates
Response rates are fairly consistent across the questionnaire lengths. However, dropout rates increase as questionnaire length increases.