Managing The Design Factory Outline

Title: Managing the Design Factory: A Product Developer’s Toolkit

Author: Don Reinertsen

Length: 288 pages

Published: 1997

ISBN-10: 0684839911

ISBN-13: 978-0684839912

Introduction

JIT acknowledged as most significant change since assembly line

Only possible by questioning assumptions and reinventing manufacturing from scratch

While product development is unpredictable, there are principles we can follow

Danger of “best practices”: “the practice can get disconnected from its intent and its context and may acquire a ritual significance that is unrelated to its original purpose.”

“The most effective way to deal with this problem is to try to retain a degree of thoughtfulness.”

Use things in this book as tools, not as rules or rituals

There are thinking tools and action tools, both important to product developers

Reinertsen says the thinking tools are more important – why, not how

Part I: The Design Factory

1 - Into the Design Factory

Analogizes product development to a bank account that pays no interest and loses money

We should try to view design as manufacturing because:

  • we have thought about manufacturing a lot
  • manufacturing works with mostly known quantities, repeatable
  • we have gotten useful tools that we can apply to design

Must begin with asking: “What is the objective of the Design Factory?”

Objective: to make a profit (see The Goal)

Manufacturing : cooking food :: design : creating recipes

“Our designs are not important because they are new, or because they are beautiful, or because they are innovative; they are only important if they make money. Our designs will only make money if we create recipes to turn material, labor, and overhead into valuable functionality better than our competitors do. The goodness of our recipe is important, but it is perishable.”

“Thus, we need good recipes, and we need them before our competition gets them.”

Discusses design-in-process inventory (“During the time that the recipe is incomplete we are holding an investment that is earning no money.”)

DIP is worse that WIP because WIP typically has faster cycle time and the holding costs of design are higher due to perishability

DIP typically not managed because of GAAP bias

Design has an exponentially expensive cost of change, whereas manufacturing has linear CoC

Design also has information coming in throughout process, especially late in process, when it happens to be most expensive to change

Manufacturing is a repetitive, ideally low-variance process. Design is a unique, probably variant process.

“…while we make no money by reinventing the wheel, we can make money by manufacturing the same wheel many times in a row.”

“…risk, and the variability associated with it, are inherent and desirable characteristics of design processes.”

Using manufacturing tools to reduce variability will probably backfire in design world. Some variance is needed.

Final difference is expanding work – Herbert Simon explains this in The Sciences of the Artificial

Engineering work expands to fill the time allotted to it, not because engineers are bad, but because engineers stop when they run out of time

Summary - similarities and differences between manufacturing and design factory

Similarities: goal is to make money, use input resources to produce output, have inventories

Differences: DF: CoC increases dramatically over time, have less information at beginning of process, variability higher and important, expandable tasks (“rubber finish line”)

Part II: Thinking Tools

2 - Making Profits Not Products

Many companies try to change their development process, but cannot explain why they want to change it

Imitation and obedience might work sometimes, but are not the most effective strategy

“The great risk in behaving this way is that you are constantly being blown hither and yon by each shift in the managerial winds.”

Economic model can be viewed as black box with inputs that we can tweak to try to make money

Can systematically change inputs and observe output changes to improve our profitability

Models give us ways of thinking about process, useful for teams and project managers to get more concrete context and make better decisions

Project models

Done from the perspective of the development team

Quantify what overruns of expense, cost, and performance, and for a one-month change in schedule will do to profitability

Better to have simple model that everyone understands and agrees with than to have opaque model – inputs are everything

Predict what a one-month change of schedule by analyzing market and potential competition to project delay scenarios

Benefits of using project model: make better decisions using data, faster decision making, higher buy-in on team decisions

Application models

Done from the perspective of the customer

“When we design an application model we are trying to understand how specific product attributes affect the customer economics.”

Hence, the application model allows development team to focus on efforts that are important to the customer

Process: identify application economic drivers (AEDs), convert to dollars, model the overall economics, and translate to business rules

Initial identification usually done internally by talking to sales, marketing, application engineering, and field service

From this, we can make an initial attempt to quantify the dollar impact each of the AEDs have

Then, we talk to the customer directly, changing our dollar estimates and even adding AEDs based on the feedback we get

Different customers weight different factors differently, so stratify cost estimates

Look at one-year, five-year, and lifetime costs

Then convert this into something the team can use, similar to how project models show costs of percentage change

Two caveats: not all customers of a product will have the same economic models, and customers do not necessarily understand their economics

Models of Process Economics

Done from the perspective of the development company

Similar to project models, but applied to enterprise level

If you have a large number of products, break them into groups by cost of delay and revenue

Basically showing process effectiveness and showing where the biggest gains can be realized

Practical tips for models

Use cross-functional team to generate models for buy-in

Financial people also important to include

Avoid unnecessary complexity

Input data is the key error component, and sales data is the most likely to be off

Consider what the best format for using the model will be

Prefer using an imperfect model and iterating rather than trying to come up with a perfect model

3 - Entering the Land of Queues

Discussing queueing theory terms and concepts

“In the stochastic world of queueing theory, we will experience delays even when we have excess capacity in the system.”

While manufacturing factories wouldn’t dream of loading past 85 to 90%, but development is often past 100%

“We will always need excess capacity to optimize the economics of a development process.”

“Queues are not inherently bad; they are good or bad depending on their relationship to the critical path of a program.”

Discusses cumulative flow diagrams and how they show inventory in queues Queueing theory proves most instinctive project management wrong

Levers to improve queues

increase capacity - more people (full-time, overtime, part-time, contractors) - higher productivity per person - tools - increase support - reduce non-value add activities - training - cross-train workers - recommends making people more general, since specialists cause queues - interesting implications for respect and compensation

manage demand

  • control the number of products in the system
  • control feature content per project
  • create a relief value to relieve demand (drop requirements, etc.)
  • reuse design content from existing projects

reduce variability

  • measure variation
  • reuse design solutions
  • process standardization
  • smaller batch sizes

using control systems

  • remove queue from critical path
  • reservation systems
  • capacity planning to prevent “predictable queues”
  • managing batch size
  • queue monitoring

Should not intermingle large batch transfers with small ones

Little’s Law - average number of units in process is equal to the arrival rate times the average time a customer spends in the store

“Process queues give us instant information about the health of our process.”

4 - It’s All About Information

“The optimum design process must generate valuable information cost-effectively.” –> 1) efficient 2) high-value

Information theory

Economic benefit is essentially ROI – value / cost

Efficiency is cost per unit of information

Value is the monetary value per unit of information

Information is inversely proportionate to the probability of an event occurring

  • interesting, this goes along with my thoughts on model generation and how disproving your model gives you information

The information content of a test is maximum when the probability of failure is fifty percent

AJP: this could apply to TDD, other testing, and architectural tests (“does this feature work in our architecture?”)

Information is better earlier – use small batch sizes to avoid delays in information

Model for this is economic order quantity problem

Small batch sizes control the size of the total error population of the product, enabling better completion date prediction, easier to fix defects since they were added recently

Iterations also generate information early

Instead of trying to do things perfectly the first time, put something out there and improve on it

Discussing decision trees and how information fits into this (reminds me of real options)

Should seek to generate information about possible failures by focusing on high-risk areas and where it has the highest economic significance

Design failures only useful when they generate new information

Must communicate failures to get the full value from them

Expert knowledge comes from failures, because they are lower probability and contain more information

Create checklists to prevent failures that will not add information because they are preventable

Information theory balances top-down and bottom-up by focusing on where the presumed risks are

5 - Just Add Feedback

Cannot predict the behavior of a system simply by understanding the behavior of its components

This “implies that we will only be able to verify system-level behavior by testing at the system level, not…individual components.”

Discusses how poor role definitions lead to too many meetings because no one knows who is authorized to make a decision

  • AJP: hmm, where have I seen this before…

“…there is no communication tool that is as powerful as correct partitioning of the system.”

“We discover that the optimum final structure for the system may not be the optimum structure for building a system.”

Closed-loop solutions “can tolerate changes in the external environment well and they continue to perform well even when some of their elements are not performing well.”

However, closed-loop solutions have more difficulty in troubleshooting, stability (chaos), accuracy

Open-loop solutions focus on making process accurate as possible, closed-loop solutions focus on the feedback loop

Discusses vector arithmetic and how it leads to law of large numbers with averaging

The variability of a system will approach the variability of the most variable component

Part III: Action Tools

6 - Choose the Right Organization

Organizations are systems, and as such some of the properties of systems apply to them

Should focus on the interfaces rather than the functional blocks (relationships over people)

Coupling, or level of feedback, will have a strong impact on the organization

“It is only when external and internal coupling are properly balanced that we get an organization that responds well to changes in its environment.”

Common organization structures

functional (vertical):

  • pros: efficiency through economies of scale
  • cons:
    • slow
    • poor communication internally and externally

autonomous (horizontal):

  • requires versatile resources
  • pros: speed
  • cons:
    • teams disconnected from organization
    • inefficiency
    • lack of expertise

hybrid:

  • pros:
    • performanc
    • low cost.

Divide responsibilities in detail, which is key for making decisions quickly

First, reduce need to communicate by using well-partitioned architecture, well-defined responsibilities on the team, and dedicated team members

Next, focus on improving communication

Use colocation to improve communication, and realize that it doesn’t need to be all or nothing

Discussion on communication methods is a bit outdated

7 - Design the Design Process

Most companies create their design process through evolution, but this is too slow

The design process outlines the method of using budget, manpower, and technology to create recipes

The design process defines which activities should take place, the sequence, and who should do them

Again, design is a one-time, unique process

Analogizes design process to English language

Standardization of low-level elements enables flexibility for high-level elements

Cannot have flexibility at every level–this would result in chaos

Modular processes use set inputs and desired outputs, and give choices for projects to make on the implementation of the processes

Similar to design principles of keeping your interfaces set so that you can change underlying implementation

Pattern languages focus on principles that should be present in good design processes

Work arrival subprocesses - large batch of work causes problems in estimation, motivation, accuracy of requirements

These times can be improved internally by changing the way that we work

Must make trade-off in engineering between fixed cost of introducing product and holding cost of design-in-process inventory

Uses game theory to show that mixing project sizes is the safest bet

Coupling technology and product development has risks, but can result in efficiency as opposed to speed

Control queues by having small batch sizes

Need to optimize subprocesses by economics

Key design principles: degree of overlap, managing information profiles, decentralizing control and feedback, and location of batch queues

Discusses how focusing on different optimizations leads to different processes

Reiterates that large batch sizes should not precede small batch sizes

Discusses evolving the process through “lessons learned”

Should be done as soon after the lessons are learned

They should also be balanced and try to include independent people

Make these sessions solution-oriented

Note observations, and pick a small subset for subsequent action

Review your review sessions to continually improve this process

8 - Product Architecture: The Invisible Design

Key architectural principles - modularity, segregating variability, and interface management

Modularity:

  • good for development expense, cycle time
  • bad for gross margin (higher unit costs or performance weaknesses)
  • These benefits are only realized if the system is partitioned correctly
  • Module interfaces can be a source of poor performance (look at operating systems)
  • There is a strong interaction between organization, architecture, and development process

Consider partitioning the system to concentrate key economic risks

While components get a lot of attention, interfaces determine performance, riskiness, and ultimately, value of a system

Managing interfaces:

  • Be sure all interfaces are defined. Ask: “what is the worst thing subsystem A could do to subsystem B?”
  • Ensure that each interface has a single point of control
  • Have a mechanism for freezing or stabilizing the interface
    • locking too early locks in system-level trade-offs
  • establish adequate design margins in the interfaces

Discusses how many organization-wide improvements should be done with team whose sole purpose is the improvement

Changing architecture to pursue specific economic objectives: development expense, unit cost, performance, and schedule

To reduce development expense:

  • maximize reuse of existing designs (components generally must be designed with this in mind)
  • displace customization outside the system boundary (consider macro languages, plugins)
  • obtain nonstrategic subsystems from vendors

To reduce unit costs:

  • tightly couple subsystems within the design
  • specify subsystems to minimize manufacturing costs
  • seek scale economies for key components

To improve performance:

  • build the design around the key performance-limiting element
  • focus on system-level optimization
  • tightly couple systems within the design

To improve speed of development:

  • fundamental shift: use a concurrent approach
  • reuse subsystems
  • stabilize architectural interfaces early
  • place design tasks with high schedule variability off the critical path

People in all departments should make decisions regarding architecture, they see different constraints

9 - Get the Product Specification Right

Starts with strategy - how are you using your strengths to provide value?

Select the right customer

Market segmentation means not serving some customers so you serve the most profitable ones

Both engineering and marketing must be involved to correctly assess cost and profitability

Result of market segmentation should be a definition of target customer and a rough definition of customer needs that will be satisfied

Understand the customer. Ask:

  • What do customers want?
  • Why do they want it?
  • How do customers make their decisions?

High ROI customer understanding tools: - customer interviews - observation of customers - and focus groups

Read the book for the pros and cons and tips for using these, I mostly skimmed it

A specification is a critical input for the design process

A specification is a control device for the project

Specifications should be brief – control the important things, not everything

Controlling everything dilutes the product by missing important things and taking up design time by making things more complex

Consider a catalog page for high-value requirements

Can use qualitative or quantitative means of determining what is important to customers

Have a product mission: Why should a customer buy this product instead of the competitor’s?

Need to have a good process to create a good specification

Should reduce delays in program and have development team involvement

Recommends fluid specifications that eventually are frozen for development purposes

On the other hand, realize that specifications should be in motion if the target market is in motion

Most companies treat specifications as open system – but this makes is prone to error

Seeing the specification as flawed to begin with is more constructive

This will cause us to work continuously through the product development lifecycle

Use customer-driven process instead of specification-driven process

“Its advantage is that customer-driven processes will almost never generate a product that nobody will buy.”

In order to do this, we need specific mechanisms to get quick, accurate feedback

Marketing people are very busy in a customer-driven process

  • AJP: in a lean startup, everyone is very busy in this process

Methods include: using a customer partner, customer advisory boards or user groups, or proxy customers on the design team

“The specification should never be the only tool telling the design team what is important to the customer.”

Again, discusses how to change the specification process to pursue specific economic objectives:

To reduce development expense:

  • thorough requirements that include downstream groups, like manufacturing
  • specification change process - will be strict about increasing work, liberal about reducing work

To reduce unit costs:

  • eliminate all unnecessary elegance from the design
  • the cost goal should be broken down between subsystems and owned by the entire design team
  • also demands careful market segmentation
  • specification often subjugated to cost realizations by team

For high performance:

  • intense contact with specific customers
  • models of application economics and performance benchmarks used for making tough trade-offs For development speed:
  • start the design before we have a complete specification
  • this is especially useful when requirements are changing rapidly anyway

“Too often companies do a poor job at these front-end activities in the development process. They move rapidly to create a list of features without spending much time selecting and understanding their customer. Such actions only create the illusion of rapid progress; companies that shortcut these steps pay the price later.”

10 - Use the Right Tools

“Technology impacts development processes in three ways: it accelerates information flows, it improves productivity, and it reduces delays.”

Information flow examples - pharmaceutical companies using databases to search for trends while trials going instead of at the end, using build servers

Productivity comes from automating easily automatable tasks that are boring to humans

Could see no increase or even drop in productivity due to steep learning curve or trying to do more than we would have manually

Reducing delays comes from reducing process queues (see chapter 3)

Implementation Principles

Technology changes process - if you are not prepared for your process to change, you are not getting the most out of your technology change

Pay attention to economics - never use something for the sake of using something new

Of course, this depends on having a business-level cost of delay approved by the finance department…

Technologies: automating design computations and decisions, prototyping and testing, improving communication, simplifying information storage and retrieval

Reinertsen notes that one company improved technology for bidding and significantly reduced this cost and improved speed

Dates himself, “It is likely that E-mail will become as ubiquitous as the telephone for most development teams.”

Design reuse can be improved with better technology

11 - Measure the Right Things

“…the purpose of controlling a process must be to influence economic outcomes.”

Can only control what most leverages our economic outcomes

“This simple concept is perhaps the most frequently violated principle in control system design for product development.”

Discusses project control triangle: scope of work, resources, schedule

Since product development has inherent variability, we must consciously choose which side of the triangle will bear the project’s variability

Most companies unconsciously choose to fix scope and resources, at the expense of making schedule variable

Discusses how setting other things as variable might be better for software, medical devices

Understand how to decentralize control correctly by stratifying events by frequency and where in the organization decisions will be made

  • basically, empower people by letting them make decisions at the right levels; provide decision rules, rather than performance standards

Choose metrics that are well-suited to control objectives: simple, relevant, leading indicators

Leading indicators can be seen at the process queues

Other leading indicators include turnover rates, manning level, specification freeze percentage

Project-level controls are similar to previous sections, focused on specific economic objectives:

To reduce development expense: limit delegation, use expense budgets, and do piecewise funding of projects

One key of budgets is to see things in aggregate so we don’t waste time when buckets overrun

To reduce unit costs: limited delegation of authority (not very effective), using cost budgets, and design reviews (do them early)

For high performance:

  • limit authority
  • performance budgets
  • design reviews
  • decision rules
  • design defect tracking systems
  • customer-driven process For development speed:
  • control schedule changes
  • time budgets (suggests internal and external to prevent Parkinson’s law)
  • schedule review meetings
  • use priority systems
  • monitor queues

Business-level controls measure the functioning of the overall development process

Expense-focused controls measure the relative efficiency of our development process (ROI)

Cost-focused controls can be internal (cost-performance ratio of the product) or externals (equivalent competitors)

Performance can be measured in terms of market share gains, sales growth, or price premiums in the marketplace

  • using technical measures can be misleading – might be missing entire markets with wrong focus

Speed can be measured internally by the fuzzy front-end, the development process, and the volume ramp of the product

Cycle time can also be measured externally by comparing to competitors

“It does not matter if we run faster than the competition as long as we win every race. We can do this by starting running before they do.”

“The key message of this chapter is that allocation of scarce time and attention to project control must be done with a keen understanding of the underlying economic leverage points.”

“Don’t spend resources controlling things that don’t influence the overall economics of the business.”

12 - Manage Uncertainty and Risk

Discusses making decision trees to weigh risks

Not trying to eliminate risk – that would be impractical given the nature of product development

Factors we can influence include: decrease the magnitude of the downside, reduce the probability of the downside, increase the magnitude of the upside

Market requirement <–> Product Specification <–> Product design [Market risk] [Technical risk]

Most product failures are caused by market risk: differing needs, trouble articulating needs, needs may change with time

Ironically, companies spend millions of dollars testing to technical specifications, and little to see if the specifications meet market requirements

Risk management strategy: obtain information to place intelligent bets, determine if the bets have paid off, backup plan in case we have lost

Can manage market risk by:

  • using a substitute product
  • simulating the risky attribute
  • making the design flexible
  • moving fast

AJP - interesting that he says that market risk is higher than technical risk and is overlooked, but then devotes more space to technical risk

Manage technical risk

Have a buffer between projects so key participants are engaged at the beginning

AJP - how does this fight the S-curve often displayed on projects or prevent overloading of resources?

Manpower always is a management issue – when a product is a priority, we will see few problems

For most projects the failure modes relate to integration, which means they lie in the interfaces rather than in the modules

Advocates a meeting similar to planning poker where team estimates risk in different modules

Risks should only be taken if they are economically justifiable

If taking a risk is economically justifiable, we should resolve the uncertainty and create a backup plan

Can resolve uncertainty with a model or simulation, or with a prototype

These are good because you see things as they will be, and they can have an energizing effect on the organization

Reduce system integration risk by: increasing reuse of interfaces, provide extra design margin in interfaces, increase amount of subsystem testing, force system integration earlier in project

Backup plans protect against failure on a key economic parameter

As such, we seek to offset damage in one area by compensating with one of the other three parameters

World-class testing - tests have a very high information content

Many companies misunderstand testing because they fail to distinguish between design testing and manufacturing testing

Goes into the four economic things again

Cheap testing

  • rarely the most important objective in design testing, but if it is:
    • eliminate duplicate testing
    • test at the most efficient subsystem level
    • automate many testing processes
    • avoid overtesting the product

Low unit cost impact

  • again, rarely the most important objective in design testing, but if it is:
  • eliminate product features that have been added solely to make the product easier to test (works in theory, not in practice)
  • use testing as a tool to fine-tune product costs

Maximizing performance - try to maximize test coverage (need a keen understanding of the actual application of the product) - enhance the validity of the tests (testing should generate the same type of failures that are showing up in the field) - when we find competitor products that fail our internal tests, it is likely that our tests are not valid

Fast testing - increase parallelism - use reliability prediction - decrease testing batch sizes - monitor and manage queues in our test processes Continuously improve your testing process - set explicit objectives, people to accomplish them, a plan, resources, and a schedule

Part IV: Next Steps

13 - Now What Do I Do?

Directed mostly at senior management

Do your math - understand the economics of your development projects and processes

Use decision rules - Use hard facts to make decisions. Apply intuition to model inputs and sanity-checking outputs, not to extrapolating outputs.

Pay attention to capacity utilization - grasp the economics of queues

Pay attention to batch size - lower it

Respect variability - uncertainty is what creates information, and information creates value of product development. Create processes that function in a variable world.

Think clearly about risk - see it as a process of placing rational bets on the future. Avoid punishing failure, instead communicate failure

Think systems - don’t optimize the parts, optimize the whole by setting the right context

Respect the people - sometimes “problem is system, not people” is misleading. Instead, “success is equally dependent on creating a workable system and populating this system with excellent people.”

  • “If we truly believe that such an attitude toward people is correct and profitable, we will devote our scarcest resource, management time, to developing our people.”

Design the process thoughtfully

Pay attention to architecture - obtain cross-functional reviews before product architectures are adopted

Deeply understand the customer - understand why the customer wants the product, not just what they want

Eliminate useless controls

Get to the front lines

Avoid slogans