When I’m talking on the phone to a client, I don’t need to see their body language to sense disappointment when I tell them that their questionnaire is too long.
This is very understandable, because through the discovery and intake process, the client is encouraged to think more about the kinds of information and answers that they need to make business and marketing decisions.
As part of the process, the market researcher incorporates the client’s input into the questionnaire design. Through no fault of the client's, and regardless of whether the market researcher designs the questionnaire or the client provides their own questionnaire, invariably the first draft is too long. We then need to consider whether to trim selected questions or response sets. It is only a small exaggeration to say that clients absolutely loathe having to think about this: “We wants it. We needs it. Must have the precious!”
In all fairness, market researchers don’t like to cut off potential sources of data any more than clients do, but there are sound reasons to make judicious editing part of the questionnaire design process. Here are seven suggestions to avoid hearing those five dreaded words, “your questionnaire is too long”.
- Make sure that you’ve defined the business and the research objectives succinctly, and stick to them. Well-designed questionnaires have a “flow” and too many unrelated questions can confuse respondents and make them lose focus.
- Think about your respondent’s attention span and patience. For an online questionnaire, it’s best to keep the survey length at 15 minutes or less for a consumer survey, and 20 minutes or less for a B2B survey. Depending on the topic, the skill of the interviewer, and the incentive, you can add about five minutes to the upper limits of these estimates when conducting in-depth interviews.
- Consider your respondent’s ability to understand your questions. Long questions are hard to follow, easily misinterpreted, and frustrate respondents.
- Be mindful of what your question is asking. Can the respondent actually answer it in a way that will provide meaningful, unambiguous data? Compound sentences with multiple topics are generally considered undesirable. For example, what subject is the respondent actually rating when faced with a question such as, “On a scale of one to seven, how would you rate your satisfaction with our company’s customer service, knowledge and responsiveness?" (Author's note: scale descriptors have been intentionally omitted for this discussion.)
- Limit your closed-end response sets to a manageable level. This of course will depend on how you plan to analyze the data, but a good target for quantitative surveys might be about seven or fewer short responses, and four or fewer longer responses per question. Respondents will have a hard time retaining and processing any more responses. One tip is to eliminate some of the expected lower incidence responses and add an open-ended “Other” as a response. For in-depth interviews, you probably will be better served asking an open-ended question instead.
- Ask fewer open-ended questions. In a quantitative survey, this type of question usually slows down the respondent. Even worse, it can result in the respondent skipping the question entirely, or giving a low-quality response. In an ideal world, you would conduct some qualitative research prior to conducting a quantitative study, and avoid this problem.
- Stay on top of your organization’s research needs, and budget accordingly. Unless you have an extremely small universe of customers and potential customers for which you need to take into account research fatigue, it is usually better to conduct shorter surveys on a more frequent basis. In other words, don't try to answer everyone’s research questions in one long survey conducted once every year or two. Shorter, more frequent surveys help ensure that you stay on top of changing consumer attitudes and behaviors, and it will help your respondents give you more thoughtful, meaningful answers.
In conclusion, there’s really no reason to fear hearing the words “your questionnaire is too long”. We hope we have convinced you that decreasing the length of the questionnaire can help improve the quality of your data.