Surveys are great for quickly & inexpensively collecting customer feedback to help you rapidly iterate towards a minimum viable product. But like any research tool, there are some minefields you have to watch out for to make sure your results are actually useful.
The unfortunate fact of the matter is there’s no way to tell FOR SURE that the person taking your survey knows what you’re asking, or is just spacing out, checking things randomly and thinking about Doritos, or whatever.
These folks – whom I call, appropriately I think, “Dorito Munchers” – are people who whisk through your survey just checking boxes. This annoying, yet easy to weed out, creature we battle with in market research circles can easily torpedo the best planned out research study.
So aware of this threat, you can lay a trap in the form of an attention check – or what I call an “anti-Dorito muncher question” – to weed them out and keep your survey safe.
Anatomy of the Anti-Dorito Muncher question
This question is a tricky little bugger designed solely to outfox the easily outfoxable.
The way it works is that you just need to lure them in with a simple statement, then create a mini “wall of text”paragraph and then at the end, ask a simple question, the answer of which is in fact wrong. If they were in fact paying any attention at all, they would know that the likely answer is wrong. And if they answer the likely (wrong) answer, then they show they’re not paying attention, and they get disqualified, and your survey remains untainted.
You don’t have to follow my example verbatim – in fact, I just made it up on the spot. But basically a well-crafted attention check question will have these 3 elements:
- The unassuming intro. This is designed to lull your reader into a false sense of security. The Dorito Muncher in particular will see the first sentence, and the large paragraph of text, which will be too much work for them to in fact read, so they’ll skip down to the final sentence. That’s actually what we want. We use their lazy psychology against them.
- Level with your survey taker. Here’s where you tell the survey taker what is up. You just tell them that a lot of people often don’t pay attention, which is why you want them to answer the question in a way they normally wouldn’t
- This is where you ask the trick question. In this case, “what color is the sky likely to be?” Obviously if you’re not not reading the question, the answer is obvious – blue, not orange! But in fact when they answer blue incorrectly, you would then disqualify them, and off they would go skulking back into Dorito land.
Advanced anti-dorito muncher question techniques
Now could argue that in fact the above question wouldn’t screen out every Dorito muncher, since as is said the clock is often right twice a day, and they could be so spaced out (and lucky) that they’d answerthat the sky is more likely to be orange correctly. So if you want to add some additional protection, then what you could do is structure the question so that instead of answering wrong, they shouldn’t answer at all. In other words, tell them to skip the question. This would minimize further anyone getting through the cracks.
Then, once you have your surveys collected and it comes time to analyze it, you could simply filter out those that answered the question in your survey results.
Advanced? Not really… but I like the title advanced anti-dorito muncher, so please forgive.
Anyway there you go…an amazing and nearly sure-fire way of protecting your survey from the unaware Dorito Munchers of the world. So until next time…go forth into the world of market research protected from Dorito Munchers…and be sure to practice safe surveying!
