I work with a lot of clients from all walks of life, most of whom don’t have a formal background in market research, and I see a lot of common recurring errors in the surveys they write that can negatively impact their results and data quality.
So if you’re interested in writing your own survey (but why would you, when you could take advantage of my great survey editing services?), here are 3 common design errors to look out for (and recommended solutions) when creating your own:
Error 1: Not considering that people might not actually like your idea at all
Let’s say you’re a restaurant owner, and you want to find out what people might like you to include on your menu. So a logical choice might be to ask a question like this:
Seems like a reasonable enough question, right?
The only problem is that the respondent might in fact not like any of these dishes.
Yet – and especially if you’ve made these questions required – the respondent is forced into a troubling dilemma. How to proceed? Likely he or she will simply mark anything to proceed to the next question, and you as the researcher will be left unknowingly with an inaccurate response.
This is known as introducing survey bias.
The solution?
Make sure you always think through the question, “what if my respondent wouldn’t like any of these options?”
In this case, you need to ensure you have a “release valve” for those answers, and one of the best options you can use is, “none of the above.”
In this way, the respondent can more accurately respond with his or her interest, and you as the researcher will be left with more accurate responses.
Error 2: Including too many answers in multiple choice questions
I recently ran a poll where the individual asked a question about a name preference, and then proceeded to list out 35 different potential answers.
Clearly, the researcher was trying to whittle down a large bevy of options into a more manageable amount, but the problem is that the results could never really be trusted.
That’s because providing too many potential answers introduces another form of survey bias. After all, when there are so many responses, the survey taker is likely to get overwhelmed, and just select anything versus considering them all, which will reduce the reliability & accuracy of the results.
The solution?
Simple – don’t provide too many answer options. Edit down ruthlessly.
Is there a hard and fast rule of how many multiple choice answers to ask? No, but like with all things, try to take a critical look at your question through the eyes of your audience. Would the amount of answers in the question overwhelm you? If so, be ruthless in your editing, and see where you can cut out other options.
In the case of the naming poll, I would have preferred him to try to select his 4-6 favorite items, and then proceed with further iteration in future surveys.
Error 3: Trying to make your survey do too much
As a corollary to the including too many answers, often I just get people asking too many questions with massively long surveys!
Again, the reasoning makes sense – the researcher wants to get as much insight as possible, so they’re going to ask as many questions as possible about a whole range of items on their mind.
However, again as before, your survey takers are human, and when you have a survey that just goes on and on, it’s likely that the respondent may start to lose focus or interest. Again, introducing survey bias.
Solutions?
Your survey isn’t Superman. It doesn’t have to do everything.
Whenever you first sit down to create a survey, you shouldn’t actually start writing questions. What you should do is answer the question, “What is the purpose of my running this survey at all?”
Your purpose should be focused. If you have a ton of things you’re hoping to get out of the survey, then consider breaking your survey into a few smaller, more focused surveys.
Additionally, always try to be as ruthless as possible in editing down and simplifying your survey creation. More is not necessarily better, and can often be worse. In general, the more efficient you can be in getting more information while asking less questions, the better.
Sometimes despite your best efforts you will need to field a longer survey, but in that case there are devices – like including progress bars or breadcrumbs – that can give the survey taker some context which will help with their motivation and focus.
In Conclusion (tl;dr)
Good survey design isn’t rocket science. It just comes down to being aware of a few simple general rules of thumb:
- Try to put yourself in your respondent’s shoes.
- Always assume your survey taker might not know, like, or care what you’re doing – and provide a release valve answer like “don’t know,” “not sure,” or “none of the above”
- Less tends to be more in survey design – try to be as ruthless as possible when simplifying and editing down your questions & answers
Follow these 3 simple rules and you’ll likely be miles ahead of other survey researchers and more importantly, receive more accurate & reliable data in your future surveys.
Leave a Reply