“In user tests, we actually like to test our competitors. Whether it’s their marketing or the actual product, that lets us learn a ton about competitors. But more importantly, it lets us learn about the customers, because we start to understand what value props resonate with customers, what do customers care about, what our customer’s problems are… So whenever we look at a competitor, it’s always with the lens of the customer.” – Marie Prokopets, FYI Co-Founder
When you have a sense of the unmet needs and the reasons why customers hire competing products, you can start to test how well those products actually meet customer expectations.
If you are on a budget, testing the products yourself (or having colleagues test the products) will help to generate learnings.
To take things further, consider doing competitive testing.
You can use services like UserTesting or Loop11 to organize remote user testing sessions. For a fee, these platforms will recruit participants based on advanced demographic criteria.
During the test, participants will be asked a series of questions and given tasks to perform using your prospect competitors’ products.
You’ll be able to review videos of their actions, hear their comments as they perform the tasks, and read their answers to your questions.
How to do Competitive Testing
To do competitive testing:
- Select the competitors you’d like to evaluate.
- Define three or four tasks around the Job you identified.
- Flesh out the profile of the people you’d like to test on (the proxy customers).
- Set up the remote user testing sessions.
Early on, two or three participants per competing product should give you enough information to go on.
The best way to identify proxy customers is to start with people that have bought the competing products by looking, for example, at product reviews: What were the reviewers’ profiles? What kind of work did they do? What were the characteristics of their organizations? Could you find more people like them?
To further your understanding of the market and the competition, follow the user tests with open-ended questions like:
- How do you currently [ Job to be Done ]?
- What products and services do you currently use to [ Job to be Done ]?
- How would you evaluate the product(s) you’re using?
- Have you ever switched tools or processes? Why?
Similar questions will help you learn the words that prospects use when they talk about the competition. You’ll also learn why they choose to use those products, whether they were using them before or have switched, and what they think about the competition.
Combining Tests With Net Promoter Score Surveys
You can also follow the tests with Net Promoter Score® (NPS) surveys to learn about participants’ sentiment towards those tools.
The basic NPS question is an 11-point scale (0 to 10) asking “How likely is it that you would recommend [ Product ] to a friend or colleague?”.
The most valuable information you can get from NPS surveys tend to be the responses to the open-ended follow up question:
“What is the primary reason for your score?”
By combining observations from competitive testing, NPS surveys, and open-ended answers you can get a feel for the strengths and weaknesses of each competitor. You’ll also be able to identify opportunities for improvements.
This post in an excerpt from Solving Product. If you enjoyed the content, you'll love the new book. You can download the first 3 chapters here →.