“A survey shouldn’t be a fallback for when you can’t do the right type of research.” – Erika Hall, Author of Just Enough Research
On the surface, running a survey seems easy. After all, all you have to do is:
- create an account on SurveyMonkey, Typeform, or any of the bazillion survey platforms on the internet;
- write questions;
- publish the survey;
- share it with the appropriate audience; and
- wait for the answers to come in.
Right?
However, surveys are one of the most dangerous research tools. Creating a survey that is both representative of the views of the target audience and captures the right information is a real challenge.
Surveys are good for evaluative research; they can help you to understand the composition of a given population, and get a more granular understanding of the patterns within that population (e.g. how many one-time buyers find the product too expensive).
How to Implement a Survey That Works
To implement a survey, you need to be very clear on:
- what you are trying to learn;
- the audience you’re trying to learn from; and
- the sample size that you need in order to be able to rely on the data you’re collecting.
With surveys, you are usually trying to reveal hidden patterns, or to sample a large population so that you can better understand its composition and preferences.
So, what challenges is your business facing? Where are you making assumptions? What information could help reduce uncertainty?
You should be able to summarize the goal of your survey in one sentence. List out the sub-topics you’d like to explore.
How to Recruit for Surveys
Who would you need to survey to get representative answers to your questions? Prospects? Users? Customers? Random strangers?
Recruiting from your user base should be pretty straightforward. Make sure you target the appropriate segment as precisely as possible. If you need to poll strangers or prospects, consider working with a research recruitment firm, or using ads to target the right folks.
Calculate the sample size ahead of time. This will help you figure out how many respondents you need. A confidence level of 95% means that the survey results have a 95% chance of being representative. This is an accepted standard.
The sample size you need—and the confidence level you aim for—will dictate your recruitment strategy and the speed at which you can get results. It’s important to consider that only a small percentage of the people you contact will actually take your survey. For example, via email, it’s possible that only five percent of the people you invite will complete your survey (Assuming a 30% open rate, and a 5% click rate).
Once you know what you are trying to learn, and from whom you’re trying to learn, it’s time to create your survey.
As a rule of thumb, you should only include questions that will lead to specific actions. If you don’t know how the answers will help you advance your business, then you’re probably wasting your users’ precious attention.
The shorter the survey, the more likely respondents are to complete it, and the more likely your data will be reliable. Try to aim for four or five questions at most. If you need to ask more questions, break up your survey into several steps.
Questions to Ask When You Implement a Survey
Write clear and short questions using language your audience is familiar with. Ask one question at a time. Don’t try to combine two or more questions into one. Avoid loaded or leading questions. Don’t ask people to predict the future, or to recall the distant past.
Use closed questions for validation or to collect discrete answers. Those could be:
- Checkboxes: best used when several answers could apply.
- Multiple choice: best used when the respondent needs to pick an answer within a set.
- Scales: best used when you want respondents to rate an item or statement on a numerical scale.
- Ranking: best used when you want to understand your respondents’ priorities.
For closed questions, be sure to consider all possible options from the respondent’s perspective ahead of time. If you’re not entirely clear, then add a fill-in option for extra answers.
Open-ended questions help you learn about customer needs that you might not have known existed. They take more time to analyze, but can help reveal deeper patterns and insights. Use them sparingly.
To ensure that your questions are clear and unambiguous, do a dry run with small sample, or with colleagues who haven’t yet been exposed to the project. How does their understanding line up with the information you’re trying to collect?
Think through your question sequencing carefully. It’s often a good idea to start with easy questions like ‘yes’ or ‘no’ questions. Oftentimes, people will be more likely to complete the survey once they feel they made a commitment.
Once you implement a survey and the questions have been validated, share it with the precise audience you had envisioned. Sharing surveys at large on social media only works if you’re hoping to learn from just about anyone.
Stopping the Survey
When you’ve collected enough responses to meet your sample size requirements, analyze the data. Don’t get fixated on any specific data point. Evaluate all patterns independently before making up your mind, or communicating the insights to a broader group.
Unfortunately, survey results often become ‘truths’ once they have been shared across an organization. It’s generally a good idea to try and disprove the results before you go all in on the insights.
– –
This post in an excerpt from Solving Product. If you enjoyed the content, you'll love the new book. You can download the first 3 chapters here →.