Preventing Flawed MVP Development in Startups - The Mom Test

Recently, various hypothesis testing methodologies such as MVP, prototypes, pre-prototypes, and lean startups have been employed in startups to develop fail-proof products. However, when you try these processes yourself, you'll find that they require more resources and time than you might think. Is there a way to reduce even the resources involved in hypothesis testing? In this article, we introduce some well-known (obvious?) methods and how to effectively implement them to significantly reduce resources in a non-obvious way.

Testing hypotheses before MVP development

In startups, it is common to develop an MVP and test whether it resonates with the intended business as a prototype. However, there are some known methods to verify hypotheses with people before developing an MVP.

1. Surveys

Using Google Forms, Typeform, or Weavon surveys to test your hypothesis is quite effective. In fact, there are many startups that create these surveys and find participants for you.
However, if you have conducted a survey before, you may know that it can be challenging to determine which questions to ask and how to interpret the results. Moreover, it is questionable whether the obtained results are meaningful. How does knowing that a field is popular among male employees in their 20s help make business decisions for an early-stage startup?

2. Interviews

Conducting interviews with potential customers is crucial for startups. However, if you don't know the proper way to conduct an interview, there is a high possibility of gaining meaningless insights and answers obtained through leading questions. If you are asking questions like the ones below during an interview, you are likely conducting the interview incorrectly:

• Are you willing to use our product in the future?
• Do you prefer feature A or feature B?
• What features do you need? What do you think about this feature we are developing?
• Would you buy a product with such a feature?
• How much are you willing to pay for it?

Even if you work hard on surveys and interviews, have you ever experienced not knowing which direction to go, or having everyone say they would buy your product when it comes out, but no one actually buying it? Fortunately, many startups face similar problems. So, how can we solve these issues?

The Mom Test

This is a fairly well-known concept abroad, which is a principle that startups should follow when conducting interviews. There is a book called "The Mom Test." When entrepreneurs come up with an idea, they usually ask people around them if the idea is good. Naturally, the easiest person to ask is the entrepreneur's mother, who gives her opinion on the idea.

Most mothers, out of love for their child, give positive feedback regardless of whether the idea is good or bad. Entrepreneurs, encouraged by this support, pursue their idea, which often ends in failure. To prevent such occurrences, the author advises interpreting the conversation as follows:

1. Discard any compliments

People find it difficult to say harsh things to others. The same applies to interviews; people are more likely to give insincere compliments than critical feedback. Even if you think a compliment is genuine, you should discard it.
The reason is that only actions taken in the past have value as accurate data, while future actions are inaccurate and unpredictable even to the individual themselves. During the interview, they may be in a good mood or have read a newspaper article related to the conversation topic that morning. If you receive compliments during the interview, it is best to respond with gratitude and then ignore them.
Ignoring compliments is more difficult than you may think, especially when they come from authoritative figures or people in related fields. For startups, it can be particularly challenging to stay firm when receiving praise from venture capitalists (VCs).

2. Ask about actual actions taken rather than just opinions:

When someone says, "Of course, I would buy the product. I think I would definitely use it if it had feature A," you should be cautious of such responses. You must ask questions like, "Have you ever purchased a similar product in the past? Have you ever tried to solve the problem before? When was that?" to find out if the person has taken any actions to back up their opinion. Focus on the reasons behind their thoughts, why they think that way, and what problems they are trying to solve.

Some example questions to test:

Here are a few questions from the book. Test whether these questions are good or bad:

Do you think this idea is a good idea?
Would you buy a product with feature X?
How much would you pay for X?
What is your ideal product?
What is your biggest problem?
Tell me about a recent problem you've faced.
What else have you tried?
What are you doing now to solve that problem?

Add the answer and personal opinion:

Do you think this idea is a good idea?

Terrible question! Only the market can decide if an idea is good or not. Everything else is just an opinion. Even industry experts often struggle to determine if an idea is good or bad. To ask a better question, you should look for evidence of whether the other person has actually taken related actions concerning the idea and combine that evidence to draw conclusions.
For example, suppose you have an idea for a shared pet-walking service. You could ask about some actions they have actually taken:

"Do you walk your pets?"
"How many times a day do you walk your pets?"
"Have you ever been unable to walk your pets? If so, why?"
"What problems have you encountered?"
"Have you ever looked for someone else to walk your pet?"
"Have you ever spent money on getting your pet walked?"

Through a series of questions, you can learn that people struggle to walk their pets, the cause of the problem is XXX, they have actually taken action to solve it, and even paid for it.

2. Would you buy a product with the X feature?

Bad question. This is a question about opinions and hypotheses. If the person is positive, they might say yes, regardless of the product or feature, while a negative person might say no. Instead, ask if they have experienced the problem that the X feature aims to solve and if they have paid to solve it. If they say they haven't paid, does that mean the feature is worthless? In my opinion, it could be valuable or not, but it's worth testing.

3. How much would you pay for X?

Bad question, for exactly the same reason as question 2.

What is your ideal product?

This could be a good or bad question, depending on what follow-up questions you ask. The question itself is not great because it asks for opinions, but if you can learn why they need the ideal product and what problem it aims to solve, it could provide hints for discovering problems.

What is your biggest problem?

Great question! The goal of the interview is not to find solutions to problems but to identify the problems themselves. There are many ways to solve problems, and it's our responsibility as startup entrepreneurs to find them.

Tell me about a recent problem you've faced.

Great question! Just like question 5, the goal of the interview should be to deeply understand and identify problems, and this question can serve as the first step for hypothesis validation.

What else have you tried?

Great question! This question allows you to determine 1) whether the person is actively trying to solve the problem (and thus, it's a problem worth solving), and 2) that there are several ways to solve the problem, and these methods could be good or bad.

What are you doing now to solve that problem?

Great question! According to the author, this question can provide hints not only about the user's problem-solving workflow but also about pricing strategy. By checking the actual price of the actions taken by the person to solve the problem, you can find out how much the problem is worth.

We have looked at the Mom Test, a method for pre-validation before the MVP, which is more preliminary. With Waveon, you can easily develop MVPs and conduct hypothesis testing with minimal resources. They can also create user surveys for the Mom Test, so it's worth checking out. Thank you.