Testing value & growth hypotheses

XC Staff

It's not good enough for a Product Manager to say, "I think users want this feature." Instead, you need to ask, "What user value or growth outcome do we predict this feature will have?"

Let's break these terms down. Value is the benefit your user will receive from using your feature or product. Growth outcome is the way in which your user will discover your feature or product. The pieces of evidence you systematically gather to support or debunk your hypotheses serve as a your stepping stones in the development of a product or feature that is ultimately adopted and used.

Hypothesis-driven product management

Hypothesis-driven product management is the practice of treating the development of new products as a series of experiments. Instead of formulating requirements, we formulate hypotheses along with some validation criteria that state how strong of a signal we need to consider the hypothesis true. We use what we learn from each experiment to iterate on our ideas until we get where we want to be. Or until we determine that the product isn't viable and we cancel the effort.

Start with your riskiest growth and value hypotheses

The riskiest growth and value hypotheses are often called our leap of faith assumptions; if we get them wrong, our product will fail. By identifying our riskiest hypotheses early and testing them through experiments, we dramatically reduce the uncertainty associated with our product ideas. Instead of trying to get it right from the start, we try to get it right one small piece at a time in order of potential impact. For example, we can form hypotheses related to questions such as:

Problem/solution fit: Have we found a problem worth solving? Can we solve it?

Product/market fit: Have we built something the market wants?

Scale: Have we found a sustainable business model?

Experiments test our assumptions

A lean experiment is the smallest experiment we can run to quickly test our assumption. We start small and fast, and then increase the scale and scope of our experiments over time.

An experiment consist of three parts: a hypothesis, a test and validation criteria.

The hypothesis is falsifiable version of our assumption. For example, an assumption might be: "Humana associates need help understanding their Health Savings Account benefit." Phrased another way, the hypothesis might be: "We believe that Humana associates do not fully understand their Health Savings Account benefit."

The test is how we intend to validate our hypothesis. Remember to make sure you are only testing one variable at a time, otherwise you won't get reliable data.

The validation criteria are the signals we need to see to consider the hypothesis true.

Example experiment template for discovery

Hypothesis
We believe that Humana associates do not fully understand their Health Savings Account benefit.

Test
We will conduct research interviews with 7 randomly selected Humana associates.

Validation Criteria
We will know that our hypothesis is valid if we observe that the majority of interviewees cannot accurately describe key details about their Health Savings Account benefits.

Sample experiment template for a feature

In some cases, you'll be able to quantitatively measure the outcome of a test. This might be possible, for example, during the usability of testing of a feature. Here's a template to use as a guide in those cases:

Hypothesis
We believe that will result in [x].

Test
We will do/make [y].

Validation Criteria
We will know that our hypothesis is valid if we observe/measure [z].

Validation criteria

You'll need to determine how much evidence is enough for you to consider a hypothesis valid or invalid. Here are some tips to consider: