Challenges in Building a Marketplace — 4
Asking users the right questions
In my previous posts, I reiterated the importance of talking to your target users and getting the feedback that will help shape your product. However, it will be haphazard if you were to jump straight into showing your prospective users the product, get them to use it and subsequently share their experience.
When conducting a user test, you want to be asking the right questions — in terms of obtaining the answers that will validate or nullify your assumptions. You also want to be asking in the right manner — proper phrasing that do not lead your users to make statements based on yours or their assumptions. More importantly, you need to know the difference between what users say/think they want and what they really want.
Let us break down the sequence for conducting an interview.
Define your hypotheses
The purpose of spending effort to conduct user interviews is to gain a better understanding of:
- Your audience — their needs and preferences etc.
- Your product — whether what you think you are trying to save the world with is indeed THE solution
- How your audience reacts to your product
Hence, it is important to know the objective of each interview session. What is it you are trying to validate? What assumptions have you made along the way that can be verified or changed?
Zoom in on your ideal audience
Many products aim to solve a personal pain (however infinitesimally small it might be), and most people try to put their foot into the shoes of people who might face the same problem like themselves. Although that gives a starting point, the next step is finding out how many others experience a pain as deep as yours and if they are willing to pay for it.
Keeping the ideal user (who will use your product and pay for it as you expect him to) in mind, it is useful to think about his characteristics and situation that puts him in that situation. Then think about alternative users who might face the same problem.
With some luck, you will have an idea of your primary and secondary audience (and a whole bunch of surrounding assumptions), and head to the field to do some user research.
Prioritise and categorise questions
While writing down my list of assumptions and questions, I will usually draw small symbols to prioritise these questions. Rather than go down the list in sequential order, each of my assumptions can branch out in different ways much like a mindmap. Lastly, I will make a small mark with different coloured highlighters/pens so as to group the questions by themes.
As I go through the user interview, these indicators guide me towards a smooth session that does not deviate much from my original purpose. For example, putting a * for critical questions ensure that I do not miss out on them. Colour coding by themes also helps me see if there are any set of questions that particularly stands out, if there are any recurring themes.
During the Interview
Set the context
Firstly, it is important to determine the setting in which your user use your product. For example, user behaviour will differ when using a food ordering app before and after lunch. Defining the context helps to narrow your scope and increase validity for that particular situation.
Having said that, it is even more important to be aware of how your user gets into the scenario in the first place, and where he will be using your product. We can ask them to imagine “they were planning for a holiday” (for a vacation app) or “them having to file taxes” (for a SaaS for businesses), but all these comes with assumptions. For the former, we are assuming that the prospective users go on holidays, that they plan for holidays, that they even think about holidays to be able to conceive that imagination.
Listen, then rephrase
Say you got the above right, and asks the right question perfectly that will elicit a relevant response. Even then, you cannot ensure that you interpreted your interviewee’s answer correctly. If there are two observers to a particular answer, their interpretations can differ and be influenced by their subjective judgements (hence the need for inter-observer reliability in statistics).
What I find helpful is to paraphrase what I hear, back to my interviewee. If I have interpreted wrongly, he can attempt to clarify his answer. Even if he agrees with my interpretation, hearing me rephrase his answer can trigger other thoughts and use cases that he has previously missed.
Be adaptive, and open-minded
One of the beautiful parts of conducting a qualitative interview is that there are no boundaries to what you can ask. Be prepared to venture outside of your preset list of questions, especially when your interviewee repetitively speaks about a particular issue, or expresses extreme sentiments suddenly.
These are signs that he could be facing a more nagging pain, that you might be able to help resolve. You can also find recurring themes across different user interviews, which may well be the right ingredient for your pivot.
Compare notes and debrief
Like all activities, it is essential to evaluate. Sharing your findings with the team helps align everyone in the same direction. Different insights and inspirations can also arise from the sharing session.
What have you found out from this session? Has any assumptions been validated, and what new assumptions are there? Moving forward, what should your team be working on next?
Evaluation is not simply about concluding that chapter, but rather to bring the company forward. Be it iterating, implementing, or conducting more interviews, the goal for any subsequent steps is to eliminate assumptions and to find product-market fit.
Qualitative and quantitative measures
Most of the arguments above are made in relation to conducting qualitiative interviews. While qualitative research helps to seek out the nuances of user behaviours, consumer preferences and how they react to your product, the use of quantitative research can help derive trends and meaningful information.
When one user says that your product will not work well in a particular context but might function in other ways, that could be his interpretation. However, when many others sing to the same tune, it becomes a quantitative feedback that should not be ignored.
Each user can tell you that they want this and that feature or that it will be really cool if you added this or that, and you can end up confounded with a host of different requests. Yet, you will not build everything. It is such times when data (and hence information) from quantitative research can aid in your decisions.
At the end of the day, it has to be a balance of both qualitative and quantitative research. And while it is important to be asking questions, it is even more important to be asking the right questions.
This is the fourth part of a series — Challenges in Building a Marketplace. My previous posts can be found here: Part 1 (creating value for users), Part 2(audience segmentation), Part 3 (pre-revenue period). This serves as a form of documentation for my work at i1Machines. Give a ❤ if you found the points useful, and feel free to respond with your thoughts below.