Exploration versus Exploitation

Building successful software products is hard. This is not an opinion. It is a fact, supported by numerous studies.

Startups are case-studies in product development and – according to a WallStreet Journal article – 75% of all startups fail within 3 years. The number one reason – product/market fit problems. This just means that either the team tried to solve the wrong problem or the problem was right, but they built the wrong product.

Product failure happens in the walls of more mature, established companies as well. Google cancels ~ 40% of its products. Statistical bureaus quote a 50-90% new product failure rate. Standish group estimates that at least 50% of all enterprise software projects are canceled and of those that are not cancelled, 75% of the features developed are never used. Ronny Kohavi – General Manager of Microsoft’s Experimentation Platform said “60-90% of ideas do not improve the metrics they were intended to improve”.

“Product” is a hip term these days and everyone wants to be a “product” guy or gal. Software engineers are “product” engineers. designers are “product” designers. Big data and BI folk are creating data “products”. But the ugly truth is building successful software products is hard, really hard.

Failure mode #1: Team is exploring when they should be executing or executing when they should be exploring. They are confused about how mature the product is and are applying the wrong management & development approach as a result

Product teams are either “exploring” & discovering new products or “exploiting” and optimizing existing products. We could take a more complex view, but this binary distinction works well enough. Which of these two phases we are in (discovering new products or evolving existing ones) fundamentally determines how we manage, market, fund, develop, and measure the progress of our product development efforts.

When you are first designing and building a product, you are an explorer. You dont know your customer, her problem, or how you will solve it. You are “searching”. The goal is knowledge creation. Have you found a customer problem worth solving. Have you found a novel solution. Is there a business model wrapped around your product that makes sense. You have a bunch of guesses and what you need is to test them with your user base. You must be comfortable with iteration and failure (by “failure” I mean “invalidating a hypothesis or guess”). You dont want to build software for scale, quite yet. You dont want to build a perfect software framework. Your user experience should be minimalist. You want to build just enough to test a thesis and no more. Your goal is to achieve product/market fit. The ideal organizational design is a small, highly integrated, relatively flat cross-functional team. The ideal process model is iterate, iterate, iterate. Efficiency is meaningless as a metric. Flow of new information and time to validation of the thesis underlying the new product are the key success metrics.

When you are evolving an existing product, things are different. You know your customer (at least a little), you basically understand the problem they have that you are solving, and you have an approach (e.g. customer = affluent females, problem/need = want luxury items at below retail prices, approach = a market place for semi-used luxury items). You want to make your product better, bring it to more customers, derive more value from it for the firm. You are “executing”. You are dealing with issues of scale, you are ruthlessly prioritizing the experiences and features that will deliver the most value. You have an existing set of customer life-cycle metrics that you can use to instrument and improve product engagement, retention, & monetization. Efficiency and ROI metrics like NPV and IRR become yardsticks to measure success. Tests are about improving yield.

If you apply execution approaches when you should be seeking knowledge or apply discovery approaches when should be mercilessly executing, then things will go off the rails. I have seen this again and again. Mis-managing the balance between search & execution in a product portfolio or wrongly applying search & execution to an individual product, is one of the fastest ways to fail.


Focusing on “new product development”


I am going to restart this blog. Going to talk about – primarily – succeeding with new product development efforts and marketing optimization approaches. My intent is to reflect operations theory, microeconomics, machine learning, and statistical analysis against the world of building new products and to see what ideas and insights emerge.

To tackle the topic :  “succeeding with new product development efforts” we will have to push against two key themes of this blog:

  1. Are we building the right software? – aka. product/market fit
    • Are we solving an acute and pervasive customer problem or need?
    • Are we doing it in a novel, differentiated way?
    • Do we understand and know our core customer segments?
    • Do we have simple ways of getting our software in their hands?
    • Do we have a validated get/keep/grow strategy that works economically?
    • How are we capturing value for the firm (monetizing our software)?
    • etc
  2. Are we building the software right? – aka. development flow/time to market
    • Are we able to evolve our software quickly in the face of our evolving ?understanding of the market and how to delight it?
    • How quickly can we move from “concept to cash” (deliver to market)?
    • Do we understand the economics of our development cycle?
    • How do conceptualize technical debt?
    • How do we make decisions about product/software development issues?
    • What kinds of organizational designs & process models are we using?
    • etc

I have taken a new position

I am now working at a group buying company, BuyWithMe. We are competing in an industry that is exploding. The economics are very interesting and the market is organizing very quickly (Groupon raising over 100 million, Living Social raising over 50 million). We, right now, are in position # 3. I am in the process of building a marquis staff of experienced and talented individuals that, like me, have spent years working building large-scale web applications using a consumer focused/lean approach. If you are interested in joining what is becoming  to 2010 what social networks were in 2006, come join me. Anyone who has worked with me knows that I value intelligence, commitment, and humility. I am going to re-start my discussion of my learnings post Napster being acquired by BestBuy. But in the meantime, I am going to talk about cutting edge product development approaches and how we are leveraging empirical or lean start-up approaches to building our portfolio at BuyWithMe.com. We are building our organization with fast feedback and consumer focus in mind. We will chase product/market fit mercilessly. Come and join me!! I haven’t been this excited in quite a long time.

Thoughts after being acquired (2 of 10)

Empirical approaches to Product Development are easy to talk about but hard to do  (deterministic approaches die hard)

After Napster was acquired by BestBuy, I spent a lot of time thinking about the meaning of ‘product development’. BestBuy outsources the majority of its software development efforts to Accenture – a large consulting organization-  and, as a result, a contracting based metaphor permeates how many people at BestBuy think and talk about product development. “The business” and its needs are treated as separate from the people who are building the products. Cost, time to market, and technical feasibility dominated most early stage product ideation discussions that I was part of. “Should we do it?” was usually superseded by “Can we do it?” and that was very quickly followed by “How quickly can we do it?”  I have spent most of my professional career arguing that the best, most successful products occur when singular teams are responsible for marrying business value, customer delight, and technical excellence and that separating those concerns with rigid organizational boundaries results in sub-optimal solutions. So, I spent the next couple of months clarifying my understanding of product development and, with many of my colleagues at Napster, communicating our perspectives to our new partner. Read More

Thoughts after being acquired (1 of 10)

I have spent the last 20 years of my professional career building web-based products and services and growing businesses on the web. During that time I have worked for early stage startups,  fortune 500 behemoths and everything in between. Most recently I worked as the  CTO of Napster, a late stage startup. I had a lot of fun there. We rebuilt all of the consumer facing applications, beat waterfall and a “business vs. technology” mindset out of the organization, and generally introduced elements of risk taking and excitement back into the culture.  But probably the most interesting and valuable part of my Napster experience was  the acquisition of Napster by BestBuy in October of 2008. Read More