In Greek mythology, Procrustes (in translation ‘he who stretches’) was a rouge blacksmith and bandit from Attica, owning a house by the side of the road where he offered hospitality to passing strangers.
His guests were invited in for a pleasant meal and a night’s rest in his very special bed — a bed that allegedly had the unique property that its length exactly matched whomsoever lay down in it.
What Procrustes didn’t disclose to his guests was that it was not the bed that had an adjustable length; rather the guests themselves were forced to fit into this ‘one-size-fits-all’ bed by either being stretched on the rack if they were too short or having their limbs chopped off if they were too long.
There is some similarity between Procrustes’ bed and 21st century enterprises. Due to corporate mass standardization everything has to be ‘one-size-fits-all’, including the key performance indicators. KPIs appropriate for mature businesses are the arbitrary standard to which exact conformity is forced.
Dangerously simple, this practice of evaluating early stage ventures using financial KPIs designed for the mature business has, so far lead to nothing but frustration on the part of all involved and to corporations being disrupted by much more nimble and flexible startups.
The main point to make here is that companies should try as much as possible to separate their innovation activity from the running of the core business. And if companies have accounting systems providing vital information about the state of the core business, so should the innovation departments.
The only catch is that the KPIs used by innovation departments cannot be the same as the ones used in the core business. The core business is executing on a known business model, whereas the innovation departments are involved in searching for sustainable business models. These two activities require different KPIs.
Innovation accounting should comprise of three sets of KPIs:
- Reporting KPIs: designed to measure the progress of incubated new ventures from ideation through to product-market fit.
- Governance KPIs: helping paint a better picture for the company’s board as to whether or not to continue investing in particular ventures.
- Global KPIs: these examine the overall performance of the innovation department within the context of the larger business.
Reporting KPIs
Each large enterprise should devise it’s own reporting KPIs based on the challenges it faces, the business it’s in and maturity stage of the business. Reporting KPIs are meant to offer insight on the progress an incubated venture is making from ideation, problem-solution fit; and later product-market fit. The focus at this stage is on the number of experiments being conducted, number of customer interviews in context, number of prototypes and minimum viable products, number of hypotheses validated/invalidated and time spent within each business model stage. Some of these will be vanity metrics that may not provide information that is actionable for the organization. However, there are some actionable KPIs that organizations can track:
Cost per learning (or cost per failure): Every product release is followed by string of learnings coming from customer feedback. A focus on managing the costs per learning will inspire a culture of experimentation. For example, if there is an idea that has an estimated development price tag of $100,000. Innovation managers can take one of two approaches in this case. They can build one product costing $100,000 and hope that the market will actually like it; or build 10 prototypes costing $10,000 each; iteratively improving the product with each release. General management principles tell managers they should focus on managing the number of failed initiatives, so by going down the first path with one release the manager can report to the board that he only failed once. Whereas in the second case he might report that he failed 8 out of 10 times — not so good when one is aiming for a promotion.
Time-cost per learning: This KPI is related to the one above and focuses on how long each learning cycle takes. Companies should minimize the time cost per innovation cycle as much as possible, ideally reaching a continuous learning state where market input is immediately transformed into a customer deliverable. In the long run, mismanaging the cost of failure and the time it takes for each failure is going to bring down the entire enterprise faster than failing to manage the rate of failure.
Validation velocity: The ultimate goal on innovation is to reach product market fit, as indicated by finding a sustainable business model. According to Steve Blank and Alex Osterwalder everything that initially goes on a business model canvas are assumptions and hypotheses that require validation for the business to succeed in the marketplace.
Under conditions of uncertainty, the speed at which this validation process happens is important for the venture to have a competitive edge in the markets.
Thus, measuring the validation velocity for each incubated venture will ensure an optimum time to market is reached (with a validated business model). In physics, velocity is expressed as the ratio of relative distance covered by a moving object over a period of time. In business, validation velocity can be a defined as the amount of validated business assumptions a team has completed over a given time frame.
It is important to note that validation doesn’t solely imply talking with customers (customer development) and it can take many other forms and shapes: from rapid prototyping to analyzing analog competitive offerings. This process can also involve measuring the time/cost of the incubated venture’s movement from ideation, problem-solution fit and product-market fit. With passing time validation activities gradually lower the investment risk in the incubated venture, so this brings us to the next reporting KPI.
Governance KPIs
Similar to the entrepreneurial world where a startup needs to present progress in order to access a new investment round, in the enterprise environment a new venture needs to present progress to be kept on the fast track of investment. Governance KPIs are related to reporting KPIs in that they help management to decide whether to keep investing in an incubated venture. Also, like reporting KPIs, governance KPIs during the search phase of business model discovery are not financial. Rather, these KPIs need to be relevant set of measurements painting a better picture of progress towards product-market fit.
In line with the Lean Analytics framework proposed by Alistair Croll and Benjamin Yoskovitz, we believe that incubated ventures have to answer a series of related governance question in order to justify further investment. We outline these below with the caveat that the exact metrics used will be determined by the nature of the business, the customers and the product idea:
Problem-Solution Fit KPIs: In this business model stage, ventures have to demonstrate that they have found a real market need that faces an addressable market. They also have to show that they have found a solution to the market need that customers are willing to pay for.
Product-Market Fit KPIs: In this business model stage, ventures have to demonstrate that they have built the right product that the market is happy with and users keep using. They also have to show that they have figure out a business model in which the behavior of customers fuels growth, the revenue model and related costs are sustainable; and other aspects of the business model (e.g. channels), are also sustainable. The discovery of product-market fit and a sustainable business model is what justifies further investment in the business venture. As such, an important metric here is the knowledge to assumption ratio.
Knowledge-to-assumption ratio: The ultimate goal of a business venture that is searching for sustainable business model is reducing the amount of non-validated vs. validated assumptions in their business model. The closer to 1 this ratio is, the better. This because a ratio of 1 means that all assumptions within the business have been validated.
Indeed, for startups, a knowledge-to-assumption ratio equaling 1 will result in a very favorable investment valuation. In other words, an entrepreneur keeps more of his company if the investment risk is minimal (knowledge-to-assumption ratio close to 1), whereas investors demand more equity from risky ventures.
As such, from a governance perspective, the knowledge-to-assumption ratio informs management about how close a business venture is to product-market fit and whether further investment is justified.
It is important to bear in mind though that not all assumptions are alike. Some assumptions have a greater degree of uncertainty than others, and some have a bigger impact on the overall business models than others. More on critical assumptions and the prioritization of assumptions can be found here.
Barebones NPV: As the knowledge to assumption ratio gets closer to 1, another good governance KPI that can then be used to determine further investment will be what Prof. McGrath calls BareBones NPV. This KPI is basically designed to tell an investment manager if he should continue investing in the development of a particular venture based on the results that venture has already had so far.
Global KPIs
Knowing how a particular venture performs or if you still need to invest in it is not enough for evaluating the overall performance of an Innovation Department. The corporate C-suite needs a set of KPIs designed to help them answer the question of whether the innovation department is successful.
Corporate finance is based on the notion that managers should be engaged in activities that increase shareholder wealth. As such, global KPIs will focus on evaluating the returns on the investment made in the innovation department. These global KPIs are more financially and market oriented than the operational and governance ones. Financial KPIs may not work to evaluate the progress of an individual venture towards product-market fit. However, over time the overall investment into the innovation department, should produce financial returns and make a contribution to the overall growth of the organization:
Innovation contribution: This KPI examines the percentage from the overall corporate of revenue (or profit) coming from ventures launched by the innovation department in the past X years.
Cohort performance: For this KPI to be relevant the innovation department should use innovation cohorts similar to what startup accelerators or MBA programs are using. After a given number of years the financial profitability of each cohort should be measured. For example, how much money did the cohort Q1 2010 bring the corporation compared to Q2 2010? In business terms, the innovation department should be improving with regards to its overall contribution to company revenues with each cohort.
Innovation conversion: This refers to the percent of old customers that are converting to new offerings designed to replace the old ones (or being complementary to existing offerings — customers which are using more than one product/service from the corporation’s portfolio). Even though enterprise think of themselves as being able to defend market share from competitors and potential new entrants, history has shown us that this is not necessarily the case. Countless markets being disrupted by smaller new entrants. Unless the corporations are learning how to disrupt their own markets, they will be left vulnerable to external disruption.
Procrustean KPIs should be avoided by large enterprises. It is important for management to recognize that with innovation they are involved in searching rather executing on a known business model. As such, KPIs that are used in the core business cannot be transported for evaluating innovation. Indeed, innovation activities should be separated from the day-to-day core business of the enterprise.
Once this is done, reporting and governance KPIs can be used to track the progress of each venture being incubated in the innovation department. The innovation department itself can be evaluated using global KPIs that focus on its contribution to revenue, profits and company growth.
This is a guest post by Dan Toma & Tendayi Viki (authors of The Corporate Startup).