The Lean Startup

Reading objective: to refresh my ideas on Lean Startup to make companies more successful in achieving product/market fit and get a common nomenclature.

Introduction

Discusses his first business failure, and how the right team, technology, and so forth were not enough to save it from failing because they were addressing the wrong market.

This is in sharp contrast to the popular vision of an entrepreneur who works hard and perseveres and eventually is successful. Says this is "the product of selection bias and after-the fact rationalization".

Ries then brings up IMVU, and how they do almost everything differently. Focus on cycle time, getting an MVP out quickly, and learning from customers. Shipping early, charging early, not necessarily taking customers' feedback, but instead being more likely to experiment on them.

Origins of the Lean Startup

Technical person from the beginning of his career; thought technical issues were the key problems to solve, but did not bring success when solved.

At IMVU, Steve Blank was one of the advisors, where he started talking about his notion of customer development.

Eric had innovated on the technical side, and could see things were working, but struggled to explain to others why these practices were working. So he studied Lean Manufacturing and found many concepts would apply to innovation. So, coupled with customer development, this forms the Lean Startup.

The Lean Startup Method

entrepreneurs are everywhere
entrepreneurship is management
validated learning
build-measure-learn
innovation accounting

Why Startups Fail

Startups do not yet know who their customer is or what their product should be. So the traditional management methods and planning are not sufficient to lead to success.

The "Just Do It" because there is too much chaos approach doesn't work either.

"The passion, energy, and vision that people bring to these new ventures are resources too precious to waste."

Part 1: Vision

Overview: makes the case for a new discipline of entrepreneurial management.

Chapter 1: Start

Entrepreneurial Management

Basically: we need new ways of managing startups, old methods are not as successful. While we have excess capacity, we waste too many resources chasing innovation.

Roots of the Lean Startup

Takes name from Toyota (Ohno, Shingo) manufacturing approach. Key tenets that are common:

knowledge / creativity of individual workers
shrinking batch sizes
JIT production and inventory control
acceleration of cycle times
value-creating vs. waste
build quality into products

Key measure of progress in a startup is validated learning.

People generally feel that Lean Startup-like activities are less productive because they are not "doing their job" (coding, etc.) Validated learning is often much less tangible than lines of code written. So we should measure productivity differently if we want to change behavior.

"Because startups often accidentally build something nobody wants, it doesn't matter much if they do it on time and on budget. The goal of a startup is to figure out the right thing to build—the thing customers want and will pay for—as quickly as possible."

Analogy of startup to car. Trying to improve engine of growth. Feedback between driver and steering wheel. Says startups are more like driving a car than planning out a rocket's trajectory. Hence, they must be more driven by feedback.

Defines vision: a true north, a destination in mind: creating a thriving and world-changing business.

Defines strategy: business model, product road map, point of view about partners and competitors, and ideas about the customer will be.

Defines product: the end result of a strategy.

Products change constantly through optimization (tuning the engine). Strategy less frequently changes (pivot). Overarching vision rarely changes.

Analogy: if you were commuting, you would not give up and go somewhere else if a road to your work was blocked. You would find a different way to achieve vision.

Balance: many different activities to coordinate. "Even the smallest startup faces the challenge of supporting existing customers while trying to innovate. Even the most established company faces the imperative to invest in innovation lest it become obsolete."

Gives example of manager who says: "didn't hit goals, but learned a lot." Says would be unlikely to be successful, but that is what we want to encourage in entrepreneurship.

Chapter 2: Define

Who, Exactly, is an Entrepreneur?

Entrepreneurs are everywhere - entrepreneurs, etc.

If I'm an Entrepreneur, What's a Startup?

Defines startup: a human institution designed to create a new product or service under conditions of extreme uncertainty.

I like Blank's "temporary" addition to this, something along the lines of: a startup is a temporary organization to search for a validated business model.

The SnapTax Story

Intuit created app that would just file taxes for people in CA with W-2s. Illustrates that large companies can innovate in similar fashion to small startups.

Brings up Christiansen's sustaining innovation vs. disruptive innovation.

Small, internal, team that was shielded from outside influences to be able to innovate.

A Seven-Thousand-Person Lean Startup

Intuit story: successful and fended off larger competitors.

Problem: innovations were not getting the ROI they thought was possible.

Even large companies need to innovate now because technology disrupts at an increasing rate.

Intuit now measures:

number of customers using products that didn't exist 3 years ago
% of revenue coming from offerings that didn't exist 3 years ago

Primary advantage is building capability to experiment.

Chapter 3: Learn

Or: how not to achieve failure (successfully executing a plan that creates a product nobody wants).

Validated Learning at IMVU

All founders had experience, and didn't want to repeat their failures. But what would they build, and for whom?

Brilliant Strategy

Create an instant message (IM) solution that had 3-dimensional avatars.

Key assumptions:

people wouldn't want to add yet another IM platform
should be an add-on to existing IM platforms
users would want to use their existing friend lists
should be as usable on different types of computers, etc. to get maximum virality

Six Months to Launch

Built a really buggy product because it was the best they could do with the strict time limit they set for themselves. Were worried about looking technically incompetent, but still released to get feedback since they had failed like this previously.

Launch

Then nobody used the product! Was relieving but frustrating.

Tracked new users each day to see how many of them would get through the product funnel. Tweaked marketing and onboarding copy to try to increase numbers of people to use product.

Muses that having quantitative revenue goals forced them to take pain early when they realized how hard it was to get paying customers (friends and family in the initial subset of people.)

They basically figured out their entire strategy was wrong.

Talking to Customers

Only teenagers, heavy IM users, or tech early adopters would even talk with them.

Teenager sets up avatar and has fun, then they say to add the IM add-on, and they say: "what?" They walk her through it and then say: "can you invite one of your friends?" Then they say: "no way! not sure it's cool yet, and if it sucks they think I suck."

They added a single-player mode, to much the same effect.

Then in desperation, made Chat Roulette-style "chat now" button to connect with a stranger, and people liked it.

But couldn't convince them to add strangers to existing buddy lists.

In short, all of their basic assumptions were basically wrong, which they only found out after talking with customers.

Throwing My Work Away

Needs to throw away several thousand lines of code since they are building a new network. Feels bad about this. Consoles self by saying that they needed something to test with customers otherwise would not have gotten this feedback.

Still, wonders if feedback could have been obtained faster.

Value vs. Waste

Waste is anything the customer does not find valuable. However, when the customer is not known, it is very easy to create waste. Therefore, anything they did that did not contribute was a form of waste.

Ries agonized over all of the debates and prioritization and extra features built.

Wondered: could they have learned what they learned without supporting any IM networks?

Where Do You Find Validation?

Astutely notes that it is easy to say that you have learned, but not have much to show for it.

adopted the view that our job was to find a synthesis between our vision and what customers would accept; it wasn't to capitulate to what customers thought they wanted or to tell customers what they ought to want.

Numbers generally were flat until they were more aligned with customers' needs, and then they started improving.

Changed perspective from how much they were building to how much they were learning for their efforts.

Eventually when they realized that people wanted to make new friends, they could align their product development to this idea and it was more productive.

The Audacity of Zero

Easy to sell to investors, employees, families when there are no numbers. Really hard when the numbers are low. So unless validated learning is accepted as the measure of success, likely to be disheartening even though you are building what is a viable, scalable business.

Lessons Beyond IMVU

Too many people focus on the tactics of the IMVU story and cargo cult or say it won't work for them because the conditions in their business environment are different.

Great quote:

The question is not "Can this product be built?" In the modern economy, almost any product that can be imagined can be built. The more pertinent questions are "Should this product be built?" and "Can we build a sustainable business around this set of products and services?"

The answer to these questions is to use the scientific method to validate learning.

Chapter 4: Experiment

Many startups have questions like:

what customer opinions to listen to?
which features are critical / which are ancillary?
what can be changed safely / what might anger customers?
what to work on next?

Many startups just ship a product to see what happens. The result is that they succeed–at seeing what happens. They don't necessarily gain validated learning.

If you cannot fail, you cannot learn.

Think Big, Start Small

Example of Zappos starting with large vision of having people buy shoes online, and testing by taking pictures and posting online, and buying shoes locally and sending them to people. Basically all manual work, but they got to learn a lot about the viability of the market / product and what issues they would have.

Clear outcome: either enough customers would purchase shoes online to be a viable business or they would not.

For Long-term Change, Experiment Immediately

Discusses HP manager who wants to increase company volunteer impact. How might use LS principles?

Defines value hypothesis: tests whether a product or service really delivers value to customers once they are using it.

"What could we see in real time that would serve as a proxy for the value participants were gaining from volunteering?"

Surveying might work, but bias rules the day. Instead, experiment with small batches to see retention, which is a more accurate measure of engagement.

Defines growth hypothesis: tests how new customers will discover a product or service.

Gives idea of concierge MVP to test best possible experience. Give this test to early adopter types, the type most likely to want to volunteer and tell friends. Then test to see how many volunteer, volunteer again, and tell friends.

If it fails, then we get qualitative feedback from those people about why it didn't work. Much faster way of learning. And we can still do other learning or experimenting at the same time. Or we quickly invalidate and need to pivot, which is a good thing to learn sooner rather than later.

An Experiment Is a Product

A successful experiment turns into the final product, and has customers all along the way. It has value all along the way.

Gives example from Kodak, who are trying to transform into leaner enterprise. Four key questions to answer:

do customers recognize that they have the problem you are trying to solve?
if there was a solution, would they buy it?
would they buy it from us?
can we build a solution for that problem?

Product development tends to lean toward the last step.

Story of online wedding photo galleries. Had hypotheses, built small, embarrassing version. Creating an album was harder than predicted. Customers asked for features that the team had backlogged. Was valuable for validating that customers wanted this kind of functionality.

Used KISSInsights to survey customers and found they really wanted to be able to sort albums before sharing with others. So held off releasing, which was a departure from status quo that was warranted.

The Village Laundry Service

Experiment to see if Indian people would send their laundry to a service (only 7% of people had laundry machine due to cost.)

Drove a truck around with washing machine (for show) and asked people what they would look for in a service.

People wanted service but were suspicious of washing machine in truck, so they switched that part and got more information.

As a result built final machine and had good adoption.

Part 2: Steer

Overview: dives into the Lean Startup method in detail, showing one major turn through the core Build-Measure-Learn feedback loop.

Key is to focus energy on minimizing the total time through the build-measure-learn feedback loop.

Value hypothesis and growth hypothesis are the riskiest assumptions, so-called leap of faith assumptions.

Once leap of faith assumptions are validated, next step is to enter the Build phase as quickly as possible with a minimum viable product.

Defines minimum viable product: that version of the product that enables a full turn of the Build-Measure-Learn loop with a minimum of effort and the least amount of development time.

Notes that MVP has limited feature set but may contain extra overhead like measurement and showing to customers.

Then Measure phase to see whether product development efforts are leading to real progress. Recommends innovation accounting for tracking.

Finally, shortly discusses pivot / persevere.

Talks about running Build-Measure-Learn in opposite order: figure out what you want to learn, then what you need to measure, then what you need to build.

Chapter 5: Leap

Strategy is based on assumptions

Every business has assumptions. First challenge for entrepreneur is to build an organization that can test these assumptions systematically. Second challenge is to perform the testing without losing sight of the overall vision.

Key to validate leap of faith assumptions first because otherwise the whole effort might be doomed.

Says many leap of faith assumptions are comparisons to other companies or situations.

Analogs and Antilogs

Discusses Randy Komisar's Getting_to_Plan_B.

iPod example. analog: Walkman. antilog: Napster.

The iPod makers were willing to say that people would want to listen to music on the go. However, the iPod long-term strategy depended on people being willing to pay for music.

Value and Growth

Value creating vs. value destroying. Must be value creating to be sustainable. Otherwise you get success theater.

Defines success theater: using the appearance of growth to make it seem that you are successful.

Genchi Gembutsu

Toyota phrase translated as: "go and see for yourself".

Get Out of the Building

"Metrics are people" … behind the numbers are real humans whose behavior is changeable.

The truths that startups need to find exist only outside of the building.

Story of Intuit founder calling people and asking if they had frustrating with paying bills by hand.

Design and the Customer Archetype

Goal of early conversations is not to gain answers. Instead, to clarify at a basic, coarse level that we understand our potential customer and what problems they have.

Defines customer archetype: a brief document that seeks to humanize the proposed target customer. AP: similar to personas, IMO.

Analysis Paralysis

One danger of talking with customers is being impatient and just wanting to skip this step.

The second is analysis paralysis. Solution to this is the MVP.

Chapter 6: Test

Discusses Groupon MVP of skinned WordPress blog and manually doing everything.

MVP helps learning as quickly as possible. Reiterates that it is not the smallest product imaginable, though, it is simply the fastest way to get through the BML loop with the minimum amount of effort.

Its goal is to test fundamental business hypotheses.

Why First Products Aren't Meant to Be Perfect

Early products are used by early adopters. These earlyvangelists are accepting, and even prefer, an 80% solution. Prefer, because they like being early adopters and being the first to use something. Polished solutions sound mainstream.

When in doubt, simplify the MVP.

Most entrepreneurs approach a question like [“how many customers will sign up for a free trial given what we believe is enough information?”] by building the product and then checking to see how customers react to it. I consider this to be exactly backward because it can lead to a lot of waste. First, if it turns out that we’re building something nobody wants, the whole exercise will be an avoidable expense of time an money. If customers won't sign up for the free trial, they’ll never get to experience the amazing features that await them. Even if they do sign up, there are many other opportunities for waste. For example, how many features do we really need to include to appeal to early adopters? Every extra feature is a form of waste, and if we delay the test for these extra features, it comes with a tremendous potential cost in terms of learning and cycle time. The lesson of the MVP is that any additional work beyond what was required to start learning is waste, no matter how important it might have seemed at the time.

The Video MVP

Instead of spending massive resources building technical solution, Dropbox creates a video of their ideal solution. Beta waiting list went from 5000 people to 75000. Validates critical business assumption, since email signups are a form of currency.

The Concierge MVP

Food on the Table example of basically doing everything by hand and then only automating things that will clearly help make the experience better or more possible to do because of large overhead. Pretending that the tech is already there and doing Wizard of Oz style theatrics.

Defers doing things until they are necessary to be done to deliver value to customers (prevents waste of overbuilding).

Seems inefficient, but necessary to do the learning required to get early customers and learn what their needs are.

Gives example of Aardvark and how the learning was much less expensive to do with humans as the technology.

The Role of Quality and Design in an MVP

Quality: what a customer will pay for. Corollary:

If we do not know who the customer is, we do not know what quality is.

Gives example of the hack of "teleporting" avatars instead of animating them. Users actually raved about this and said it was "better than the Sims".

Speed Bumps in Building an MVP

Legal issues - consult lawyer, still might be overblown, IMO.

Fear of competitors - overrated, in Ries's opinion.

The reason to build a new team to pursue an idea is that you believe you can accelerate through the Build-Measure-Learn feedback loop faster than anyone else can. If that's true, it makes no difference what the competition knows. If it's not true, a startup has much bigger problems, and secrecy won't fix them…. The only way to win is to learn faster than anyone else.

Brand risk - launch under a different brand name and experiment under the radar until more established.

From the MVP to Innovation Accounting

Risk of bad news from MVP. Solution: commit to iterating for a set period of time before giving up.

Chapter 7: Measure

Startup's job:

rigorously measure where it is right now
devise experiments to learn how to move the real numbers closer to the ideal reflected in the business plan

Most products do not have zero traction, growth, customers. But must stay out of the land of the living dead (AP: I call this startup purgatory.)

While it is a great story to say you persevered and succeeded. Rarely hear about those that persevered and failed.

Innovation Accounting

How to know if the changes you are making are making product better? If you are fulfilling needs of customers better?

An Accountability Framework That Works Across Industries

"Enables startups to prove objectively that they are learning how to grow a sustainable business."

How Innovation Accounting Works - Three Learning Milestones

Use MVP to establish real data on where the company is right now
Tune from baseline toward the ideal
Pivot or persevere

Establish Baseline

Construct an MVP or series of MVPs that test the key business assumptions.

Tuning the Engine

"Every product development, marketing, or other initiative that a startup undertakes should be targeting at improving one of the drivers of its growth model."

Before doing, should state what changes are expected to happen after implemented. If they don't happen, should judge it as a failure.

Pivot or Persevere

Should see the numbers in the model rise from baseline toward ideal. Otherwise, time to reconsider and pivot.

Innovation Accounting at IMVU

Had low sales, assumed it was product quality, so set out to improve it. Had monthly board meetings and would calculate statistics every month. Month after month the product improvements were not affecting the baseline significantly. Eventually tightened feedback loop and found same result.

Improving a Product on Five Dollars a Day

Used Adwords to bring in new customers to the funnel every day. Funnel metrics basically seemed flat.

Cohort Analysis

Defines cohort analysis: instead of looking at cumulative totals or gross numbers, look at the performance of each group of customers that comes into contact with the product independently. Each group is a cohort.

Power of cohorts is it shows change in product over time. But in their case it was not showing any major changes over time.

Key issue was their product was not aligned with what their customers wanted because they didn't know what their customers wanted.

Sign of a successful pivot: the new experiments you run are overall more productive than the experiments you were running before.

Optimization vs. Learning

"If you are building the wrong thing, optimizing the product or its marketing will not yield significant results."

Discusses engineering death spiral (compared to marketing death spiral that Blank often discusses.) Problem is not the engineering team, it is the overall problem of executing a plan that does not make sense.

Vanity Metrics: A Word of Caution

Top line numbers that give impression that the business is succeeding more than it is.

Actionable Metrics Versus Vanity Metrics

Example of Grockit and how they started off strong (good vision, early prototyping, worked in Agile fashion), but then spun their wheels trying to make product improvements that would move the needle. Says the one thing that was their advantage was that they were disciplined, so they could switch what they were doing.

Cohorts and Split-tests

Discussion of how these are used to test assumptions / improve product.

Kanban

Standard Kanban, similar to Ash Maurya's product development board. Limiting WIP, and having doneness criteria of being validated (or the feature is subsequently removed because it adds insufficient value).

Hypothesis Testing at Grockit

Lazy registration was an assumption they thought they needed. When tested, had no discernible effect on user signup behavior.

AP: I have spent a lot of time on this for a previous project, so can attest to its complexity.

The Value of the Three A's

Actionable

Must demonstrate clear cause and effect.

Accessible

Reports must be simple. Express units in terms of people, not hits.

As the gross numbers get bigger, accessibility becomes more and more important.

Should be as clear and unambiguous as possible.

Auditable

Need ability to check with real customers when the numbers appear suspect.

Whenever possible, draw conclusions from primary data, not intermediate systems for traceability reasons.

Chapter 8: Pivot (or Persevere)

Critical to change when our assumptions are invalid.

Innovation Accounting Lead to Faster Pivots

Votizen example. Not getting very good results on key metrics (Registration-ARR). Even after optimization, did not tick up enough.

After zoom in, much better results, but not enough revenue to sustain business. Demonstrates validated learning.

Had several other pivots until success, but at increasing speeds.

A Startup's Runway is the Number of Pivots it Can Still Make

Pivots Require Courage

Don't rely on vanity metrics
Ensure you have clear hypotheses, otherwise no failure possible
Be courageous if you want to achieve your vision

The Pivot or Persevere Meeting

Wealthfront example.

Failure to Pivot

IMVU is moderately successful, but thinks what got them here will get them there. Does not realize that their customer segment needs to change to appeal to a broader audience to continue growing.

Catalog of Pivots

Defines zoom-in-pivot: refocusing the product on what previously had been considered just one feature of a larger whole.

Defines zoom-out-pivot: sometimes a single feature is insufficient to support a whole product.

Defines customer segment pivot: keeping the functionality of the product the same but changing the audience focus.

Defines customer need pivot: knowing our customers, our current offering does not serve their needs, but we can identify other value needs.

Defines platform pivot: instead of selling an application to one customer at a time, create a platform that they can build on. Or the opposite.

Part 3: Accelerate

Overview: Explore techniques that enable Lean Startups to speed through the build-measure-learn feedback loop as quickly as possible, even as they scale.