Exploration versus Exploitation

Building successful software products is hard. This is not an opinion. It is a fact, supported by numerous studies.

Startups are case-studies in product development and – according to a WallStreet Journal article – 75% of all startups fail within 3 years. The number one reason – product/market fit problems. This just means that either the team tried to solve the wrong problem or the problem was right, but they built the wrong product.

Product failure happens in the walls of more mature, established companies as well. Google cancels ~ 40% of its products. Statistical bureaus quote a 50-90% new product failure rate. Standish group estimates that at least 50% of all enterprise software projects are canceled and of those that are not cancelled, 75% of the features developed are never used. Ronny Kohavi – General Manager of Microsoft’s Experimentation Platform said “60-90% of ideas do not improve the metrics they were intended to improve”.

“Product” is a hip term these days and everyone wants to be a “product” guy or gal. Software engineers are “product” engineers. designers are “product” designers. Big data and BI folk are creating data “products”. But the ugly truth is building successful software products is hard, really hard.

Failure mode #1: Team is exploring when they should be executing or executing when they should be exploring. They are confused about how mature the product is and are applying the wrong management & development approach as a result

Product teams are either “exploring” & discovering new products or “exploiting” and optimizing existing products. We could take a more complex view, but this binary distinction works well enough. Which of these two phases we are in (discovering new products or evolving existing ones) fundamentally determines how we manage, market, fund, develop, and measure the progress of our product development efforts.

When you are first designing and building a product, you are an explorer. You dont know your customer, her problem, or how you will solve it. You are “searching”. The goal is knowledge creation. Have you found a customer problem worth solving. Have you found a novel solution. Is there a business model wrapped around your product that makes sense. You have a bunch of guesses and what you need is to test them with your user base. You must be comfortable with iteration and failure (by “failure” I mean “invalidating a hypothesis or guess”). You dont want to build software for scale, quite yet. You dont want to build a perfect software framework. Your user experience should be minimalist. You want to build just enough to test a thesis and no more. Your goal is to achieve product/market fit. The ideal organizational design is a small, highly integrated, relatively flat cross-functional team. The ideal process model is iterate, iterate, iterate. Efficiency is meaningless as a metric. Flow of new information and time to validation of the thesis underlying the new product are the key success metrics.

When you are evolving an existing product, things are different. You know your customer (at least a little), you basically understand the problem they have that you are solving, and you have an approach (e.g. customer = affluent females, problem/need = want luxury items at below retail prices, approach = a market place for semi-used luxury items). You want to make your product better, bring it to more customers, derive more value from it for the firm. You are “executing”. You are dealing with issues of scale, you are ruthlessly prioritizing the experiences and features that will deliver the most value. You have an existing set of customer life-cycle metrics that you can use to instrument and improve product engagement, retention, & monetization. Efficiency and ROI metrics like NPV and IRR become yardsticks to measure success. Tests are about improving yield.

If you apply execution approaches when you should be seeking knowledge or apply discovery approaches when should be mercilessly executing, then things will go off the rails. I have seen this again and again. Mis-managing the balance between search & execution in a product portfolio or wrongly applying search & execution to an individual product, is one of the fastest ways to fail.


Thoughts after being acquired (2 of 10)

Empirical approaches to Product Development are easy to talk about but hard to do  (deterministic approaches die hard)

After Napster was acquired by BestBuy, I spent a lot of time thinking about the meaning of ‘product development’. BestBuy outsources the majority of its software development efforts to Accenture – a large consulting organization-  and, as a result, a contracting based metaphor permeates how many people at BestBuy think and talk about product development. “The business” and its needs are treated as separate from the people who are building the products. Cost, time to market, and technical feasibility dominated most early stage product ideation discussions that I was part of. “Should we do it?” was usually superseded by “Can we do it?” and that was very quickly followed by “How quickly can we do it?”  I have spent most of my professional career arguing that the best, most successful products occur when singular teams are responsible for marrying business value, customer delight, and technical excellence and that separating those concerns with rigid organizational boundaries results in sub-optimal solutions. So, I spent the next couple of months clarifying my understanding of product development and, with many of my colleagues at Napster, communicating our perspectives to our new partner. Read More

Thoughts after being acquired (1 of 10)

I have spent the last 20 years of my professional career building web-based products and services and growing businesses on the web. During that time I have worked for early stage startups,  fortune 500 behemoths and everything in between. Most recently I worked as the  CTO of Napster, a late stage startup. I had a lot of fun there. We rebuilt all of the consumer facing applications, beat waterfall and a “business vs. technology” mindset out of the organization, and generally introduced elements of risk taking and excitement back into the culture.  But probably the most interesting and valuable part of my Napster experience was  the acquisition of Napster by BestBuy in October of 2008. Read More