“We’ve been trained to look at patterns, look at a line that fits the data points, but when it comes to qualitative research, those anomalies make a lot of difference because that may be a node to understanding a completely different market, a completely different customer. Anomalies need to be understood and need to be unpacked.” – Ashwin Gupta, VWO Head of Growth
When you start really digging through your data, you will start to notice patterns and behaviors that you don’t instinctively understand. These are likely to include:
- users who sign up without ever using your product;
- users that max out their free trial without upgrading;
- users who drop out at critical points of product flows;
- users consistently using secondary product features; and
- users with atypical usage patterns.
To continue finding friction points, look at the entirety of the customer journey and your funnels.
Are there areas of hyper engagement? Big drop offs? Behaviors you’re not sure you understand?
Identify the biggest friction points. Your goal is to understand the differences in profiles, goals, and behaviors that make some users convert and others drop off.
Are you able to reach users right after they drop off or perform the task you’re hoping to explore? Timely surveys can help you break down the behavior that you are trying to understand.
How to Setup Surveys to Learn About the Friction Points
Set up a single question survey probing:
- Users who completed the action: “What made you decide to do [ Action ]?”
- Users who decided to drop off: “What made you decide not to do [ Action ]?”
The less friction involved in answering the question, the more likely it is that users will respond. For example, a survey delivered through an In-App message or a popup in your product will get more—and better—responses than a survey sent via email.
With good analytics and segmentation, you will be able to target the exact activities that users are doing inside your product. This will help ensure that your survey is both timely and relevant which, in turn, will help increase your survey response rate (benchmark of 2 to 4%) and the quality of the answers you’re getting.
A customer who just bought will give better feedback because they’re only recently removed from the buying process. They will also be more likely to give useful information about why they bought, what emotional event caused them to look for your product, and which parts of the product they find valuable.
Once you start to get responses to your surveys, compare the answers of the users who dropped off with the answers of those that completed the action. Are there noticeable differences? Does looking at the data through different lenses (personas, profiles, CLV, etc) reveal new patterns? Answers should help you to understand the success drivers, and identify friction points.
If the answers don’t lead to a conclusive theory about behavior, consider using the exact same targeting to recruit users for interviews.
Some Caveats to Consider
Survey fatigue happens when surveys aren’t relevant to user activities. To keep your response rate high, make sure you get the timing right, and keep your surveys short.
Conversely, research shows that if you can get someone to answer a survey question, the likelihood that they’ll answer following questions is higher than 90%. So, if the targeting makes sense, consider asking more questions.
As you capture insights, keep in mind that the responses you get will most likely be skewed towards dissatisfied users and customers, because these people are more likely to respond. For this reason, it’s often a good idea to triangulate your findings by also getting insights from customer interviews or analytics.
Once you have begun to understand a certain behavior, move on to the next biggest drop off.
– –
This post in an excerpt from Solving Product. If you enjoyed the content, you'll love the new book. You can download the first 3 chapters here →.