The perils of online respondent recruitment and how to avoid them
Quarter of a century ago, The New Yorker published a cartoon by Peter Steiner which showed a dog sitting on a chair in front of a computer, speaking to his canine companion on the floor. The caption reads “On the Internet, nobody knows you’re a dog”.*
The cartoon makes a general point about Internet anonymity, but its message is highly relevant to the topic of online recruitment for market research surveys. The Internet provides a rapid, inexpensive and convenient way to obtain responses from a vast pool of people, but how can you ensure that you get responses from your target audience, and not from other types of people (or dogs)?
The first thing you should consider is whether online is really the right methodology for your requirements. Is your audience actually reachable online? For B2C projects the answer is generally yes, but it requires respondents to have a certain level of IT and English literacy, and the time and inclination to take part in online surveys, so think carefully if you are trying to reach groups such as the elderly, recent immigrants, or high net-worth individuals. Some B2B audiences, such as small business owners and healthcare professionals, may be addressable through online panels, but other types of B2B audience will require a more targeted approach, e.g. through social networking or industry-specific bulletin boards.
The next point to bear in mind is screener design. If you are screening respondents online, you want to avoid the possibility that respondents outside your target group keep trying different answers until they find the combination that gets them screened in. Simple steps you can take include having no ‘Back’ button in the screener section of your questionnaire, and setting up the online platform so that it won’t accept multiple submissions from the same IP address in a short space of time.
When it comes to the survey itself, try to ask a couple of questions that respondents in your target group should be able to answer easily, but those outside will find hard to complete. This is easier said than done now that so much information is available at the click of a mouse but there are still ways of achieving this. For example, if you want to ask about brand awareness, try including a couple of non-existent brands in the list, or if you want to survey people who have recently bought a product, ask them to submit a photo of it or enter some information from the packaging.
Lastly, if you want a real sense of the quality of your data, consider specifying telephone follow-ups, at least with a sample of respondents. These will typically take much less time (and thus be cheaper) than conducting the whole survey by phone, but still give you reassurance that your respondents are actually who they claim to be. The GDPR means the collection of telephone numbers in market research needs to be thought through more carefully than in the past since a phone number automatically qualifies as personally-identifiable information (more on this in a subsequent blog post), but there are still ways of managing it.
Incidentally, Peter Steiner’s cartoon of the two dogs and the computer didn’t attract too much attention at the time, but it has since become a popular Internet meme and is reputedly the most reproduced cartoon in the history of The New Yorker. We can’t promise that people will still be talking about your online surveys in 25 years’ time, but if you want more advice on making them as successful as possible, contact us by phone or by emailing firstname.lastname@example.org.
* For the original cartoon see https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you%27re_a_dog