How to evaluate and implement startup ideas using Hypothesis Driven Development

Posted on February 1, 2017

This article was originally published on swombat.com in January 2011.

So you’ve come up with an interesting idea. You think it might work. You’ve sketched it out using various tools like the Business model generation  canvas, a business plan, an Excel financial model, etc. You’re still positive on the idea and think it’s probably worth giving it a shot.

What next?

One common approach is to visualise launch day and work back from there. Figure out what you need to launch some kind of initial product, and then start casting the spells and incantations required to get there (mostly in the form of code or application specifications).

A slightly better approach, which those who have tried the above method usually end up using next, after they built something that took them 6 months but was utterly useless to anyone, is to build the bare minimum that you need in order to get some users, any users, to use the system on a regular basis. This is better, and gets you feedback much more quickly than the previous method.

Even better is to aim not for something you can get people to use, but instead try to build something you can charge for immediately, even if the price is lower. This is often favoured by [Lean Startup](http://en.wikipedia.org/wiki/Lean_Startup) afficionados who haven’t quite taken the lean methodology the whole way yet.

What do all these methods have in common? They present the unfolding startup as a series of tasks to be completed to get somewhere.

Here’s a better approach.

Hypothesis-Driven Development

A startup idea is not a plan of action. A startup idea is a series of unchecked hypotheses. In essence, it is a series of questions that you haven’t completely answered yet. The process of progressing a startup from idea to functioning business is the process of answering these questions, of validating these hypotheses.

Let’s consider a theoretical startup to illustrate this. Let’s say we’re looking at building “Heroku for Django”. The initial three questions for most web startups will be in the form:

  • Can I actually build it?
  • Can I get people to know about it?
  • Can I make money from it?

Often, this is the order in which they will arise, if you have some experience of web startups but are fundamentally a builder type. Making money is the last concern. “If I can get lots of passionate users who are willing to pay something, then it will probably be alright.”

To apply Hypothesis-driven development properly, you will want to order your questions by priority before proceeding. This is especially essential once you break down the questions into sub-questions and end up with dozens upon dozens of questions.

The best way to prioritise the questions is by uncertainty. An initial order for these three questions might then be:

  1. Can I make money from it?
  2. Can I get people to know about it?
  3. Can I build it?

Your own prioritisation may vary, but if you’re technical, “Can I build it?” will probably be last on the list. Of course you can build it. If you couldn’t, you would probably have discarded the idea before even getting to this stage.

Before trying to answer the questions, you first break them down into sub-questions (please note this breakdown is nowhere near exhaustive enough, it’s just an example):

  1. Can I make money from it?
    1. How much will it cost me to serve the smallest users?
      1. Which cloud platform is best for this?
      2. How many instances will I need at a minimum to run the platform?
    2. How many users will I need just to break even?
      1. How much will I be able to charge per user?
      2. What proportion of paid vs free users will I have?
      3. How well will users convert from free to paid?
  2. Can I get people to know about it?
    1. What channels are there to get the message out?
    2. How much will each of these channels cost me?
      1. How competitive are the Ad-words for this?
    3. Do I have enough contacts to get the initial, core users so the service will be useful to real users?
  3. Can I build it?
    1. What are the hardest bits of technology I’ll need to put together?
    2. Can the scope be cut down so that I have a chance of building a version 1 with extremely limited resources?
    3. Which features can be put off until later?

You should keep expanding this list until you can start to see what the burning uncertainties are. These will be unique to your startup idea and to your skills and available resources. Two people evaluating the same idea will probably come up with different key questions. Once you’ve got those key questions (the ones which make you think “Hmm, I really don’t know this and it’s really important.”), shift those to the top.

Then, start working through the questions, one by one or even in parallel. Most of the time, the answer will not be found in code, but in good old-fashioned research, planning, and the dreaded Excel spreadsheets. You don’t need to answer all these questions with 100% certainty, but you should be clearly aware of the limits of your answers, and when the answer is really critical, you should make an effort to answer it as fully as possible. You can’t know everything, but you gotta know what you don’t know and how much it can hurt you.

At some point, if the idea has answered enough questions, the next most important question will require you to build something – be it a paper prototype, a landing page, or something else. When it’s the most important question, do it, and do just enough to answer the question. Later, if your idea is really good, you will probably, at some point, start to do the really expensive stuff: building a real application.

At that point, with so many critical questions having been answered, the likelihood that you build something that nobody uses or pays for should be low.

Of course, you may succeed by shooting from the hip and just going for it without a second thought, but more experienced entrepreneurs will usually look before they leap.

Thoughts from 2017

When I wrote this article, Eric Ries had been blogging about his Lean Startup ideas for a little while, but the book was still 9 months away from publication. It is obvious that this approach is one of the core components of the Lean Startup Methodology. That said, it extracts one of the key points and condenses it in a more readable format, which I think is valuable to be able to point people to.

All of the ideas in this article are still very much current – even more so, perhaps. Lean Startup, or Hypothesis Driven Development, is still the best way to devise a plan for building a new company: find the most risky assumptions, test them and update them until they are no longer risks, then find the next most risky assumptions and repeat until you have built a functioning company.