Measuring Product Development Progress

As usual, this is written fast and I appreciate help to make it better - please point out typo's unclear sentences or arguments.

New product development is often a highly dispersed process. It usually starts with some sort of inspiration: A new technology has become available, there is a big media story in some magazine about a start-up that is doing something similar to your thoughts, a friend takes about exactly the issue you were thinking about., etc.

On the way to create a new product, there are basically three problems:

  • How to define what progress is
  • How do you measure your progress (and yourself) 
  • When do you decide to invest to build the product

How to determine whether it's a problem worth solving?

What you want to find is a problem, a price people are willing to pay to solve that problem, how many people have that problem. That is your market size and the cornerstone of you unit economics.
Normally, then this happens: you find a problem but you do not ask for a price. At home, you then do some more or less random math how much this could be worth to the customer or take the costs and add some margin. 

The fundamental problem with this is: you have not found a market size. You have found a problem size. A market is a mechanism that clears supply and demand based on a price. Without a price you don’t know the demand size. You also don’t know whether you should supply that demand. Or - reductio ad absurdum -  if I have a problem and you are selling me a product at a price of negative 10 USD I will buy quite a lot. Not sure, how your business is gonna go with that though. 


“Do you buy shirts from time to time?"

12 - 15

“How many do you buy a year?”

No, sometimes they don’t fit and going to the store is annoying because the one I like closes at 6 when I am still at work

“Do you always find well fitting shirts in the right design fast?”

“Could we solve that problem by taking your measurements. We send you a sample of the cloth so you can touch and feel it. Whenever you need a shirt we will send you a perfect fit shirt with the fabric you chose. 

Yes, that would be great.

Personally, I’d use this product/service at about 40 USD per shirt. Do the economics work at 40 USD per shirt? Definitely not. Am I part of the market: yes, at 40 USD (loss making for the supplier). No at anything higher (smaller market size for supplier).  Or, you find some crazy technology or distribution that makes this work. 

The point is, you need to put a price on the problem in the first encounter with the customer. Anything else is pointless. 

Other things you want to know about your customer

Problem features - like how often and when it occurs, what is the measure of it being solved?

Customer profile - who is the person that will pay the price? Job title in B2B? Stage in life for B2C other things like that

Other - when did you talk, how could you reach him. If you decide to build the product this is your initial sales pipeline.

Execution Example

The trick is simple. You define what you need to know. You put that into a google form. After each customer interaction you are required to fill out this form. This forces you to gather the complete information that matter for the market size and the unit economics that drive your business. 

So, let’s say we want to develop a tool for Digital Consultants to make their proposal writing easier. Also, we might want to sell insurance to them to protect them against a customer suing for not having fulfilled their proposal. 

Next we identify the questions that matter. Below you find the screenshots from a questionnaire you find in full here: - check it out!

Note, that I don't mean you send this questionnaire to the customer. Probably, it is more effective if you conduct the interview verbally and fill this out. In other cases, maybe you can actually mail this to people. This will depend on your product.

Product Interviews

Here you want to capture things like the name, when you talked and who of your team talked. 

Obviously you add whatever is relevant here. For example: how did you source this customer? Through linkedIn outreach? A friend?



Problem Questions

These are obviously key questions. If you cannot define them well, you need to clarify your (implicit) assumptions. This will also change as you start to do these interview. This will also be a key input for your marketing message. Because your product has only one job, solving these problems. So white papers, adds, direct emails will be based upon this.

Problem identification

Price questions

As we have discussed upfront - the price is the indicator whether the problems you have identified can actually be solved. You need two inputs to ask this well: the unit of the product (on-time vs. subscription vs. per unit of problem solved) and the price. This depends intrinsically on what you plan to build. 

This is how they could look like for our example:

Product Management - Ask for a Price

How do you measure success of product discovery

Above, we defined the three problems: 

  • How to define what progress is
  • How do you measure your progress (and yourself) 
  • When do you decide to invest to build the product

Essentially, success is identifying a problem really well. Narrowing it down to it's core and - most importantly - attaching a price point to it. Together that gives you a definition of progress - point 1. 

How do you measure progress and yourself: Well, once you have identified the questionnaire and include the questions on price you can pretty much measure yourself (or your team) by the number of fulfilled questionnaires. Obviously, you can still cheat yourself and there will be other inputs. But in general, everybody agrees that talking to the potential customer is the most important. Now, you have a number for this because you have defined what "talking to the potential customer" really means. 

Lastly, when you decide to invest to build the product: well, that comes down to the number of data points (filled out questionnaires) that you are confident with. Obviously, you can talk to nobody and just build and you could talk to thousands of customers. That trade-off is yours to make but at least this is somewhat quantified by now.