An Unlikely Origin of Agile Thought

Perhaps the most cited reference defining and supporting Agile thinking is the 1986 Harvard Business Review article by Takeuchi and Nonaka, “The New New Product Development Game” (the first “new” is for the new process, the second “new” is for new products created by the process).  It is this article that is credited with tying the word “scrum” to product development methodology and describing for the first time several principles which have become associated with Agile.  But does the research described in the article actually support Agile principles and is the domain of new product development suitable to generalize to software development?  This essay makes a detailed comparison of all the described characteristics.  If that is TLDR, the points are summarized in a table at the end.

The article describes common elements found among 6 product development efforts conducted by Japanese consumer product companies in the mid-1970s.  All the products studied were hardware products including copiers, cameras, a car, and a personal computer.  The development cycles for the products varied from 1 to 3 years.  Teams were given only a high-level vision of the product and it was expected that the bar for success was set very high (which the authors called “built-in instability”). 

It seems evident that these product development efforts share little if any of these key attributes with software product development.  The products are complex physical devices with the constraints, time required, and costs associated with establishing and revising manufacturing processes as the products evolve.  As well, typical software development efforts begin with a much more clearly defined set of goals and desired features than was provided in the studied cases.  Finally, in Agile software development we work at understanding the minimum product that will suffice while in the studied products, teams were encouraged to achieve the maximum capability possible.

The focus of the article is 6 common process elements identified among the product development efforts.  They were:

1. In each of the cases, development was done by a hand-picked product task force, a cross-functional team, where team members came from all the needed functions including engineering and business and were typically top performers in each of their areas.  In effect, these were all-star teams from across all functions of the company.  As well, it was expected that the team would be disbanded at the conclusion of the development project.  By contrast, while Agile principles assume motivated team members, they do not assume that they must all be top performers.  Also, while Agile teams typically include a single customer representative, they don’t include the broad variety of business stakeholders described in the article.  And finally, for Agile teams to achieve full maturity and become highly functioning, it is assumed that the team will persist intact over time rather than being dissolved at the end of the project.
2. Another common element among the product approaches was the use of “self-organizing teams.”  In the article, that phrase is defined in terms of several attributes:
   • First, the teams start from having “no knowledge,” i.e., that they are not working from established product designs or approaches.
   • Next, the team is given minimal product vision, so the requirements and product features are defined by the team itself rather than a sponsor or customer representative.
   • Next the team continuously raises the bar on the requirements as the design evolves, challenging themselves to make the product better and better
   • And finally, all team members are co-located to facilitate easy face-to-face communication.
   • While the phrase “self-organizing teams” has been adopted by Agile approaches, the definition used for the phrase shares very little with the article’s definition.  Agile teams are expected to bring prior knowledge and best practices to the effort, the customer representative defines the needed features – the “what” and the team defines the “how”, and the team is not asked to continually challenge itself with higher requirements instead only responding to the customer representative’s interpretation of the need.  Actually, the only shared element of the definitions is the use of co-location as a means for enabling close team communication.
3. Development occurred in overlapping phases which was called the “sashimi system.”  While looking superficially iterative in one illustration, a subsequent example shows that the process is still a very “waterfall” in approach with the phases representing different levels of engineering design with different outcomes rather than shorter design/build iterations.  The approach greatly enlarged the teams since at any given point in time there could be multiple concurrent engineering phases but largely separate engineering teams.  While the overlapping phases enabled quicker delivery and communication across engineering teams and disciplines at different phases, it created the risk of needing greater project management planning (e.g., team communications, contingencies, dependencies).  As with the previous common elements, this seems to be strongly divergent from Agile approaches.
4. Teams practiced what the authors called “multi-learning” which they defined as a combination of extreme cross-functional-training, emphasis on learning by doing, group as well as individual learning, and individual participation motivated by internal competition and peer pressure. In the studied companies, individual team members were expected to both be experts in their own field of specialty and to develop significant skills in another, entirely different field (e.g., an engineer learning marketing).  While Agile principles don’t specifically mention ongoing learning or training, one might find support for learning from the Agile principle of “continuous attention to technical excellence.”  However, the approaches taken in the studied companies seem to be motivated by the intent to drive radical organizational change, rather than supporting individual technical excellence.
5. Based on the assumption of hand-picked teams, management for these teams could exercise “subtle control” over them. With subtle control teams were largely self-governing with infrequent checkpoints with their sponsors.  Teams were encouraged to make direct user contact for gathering market information and needs.  Motivation for the team was supported by group reward processes and peer pressure, although encouraging acceptance of mistakes.  Finally, the subtle approach also including not trying to directly control supply-chain partners (could also be thought of as project dependencies), but rather provided them with requirements and then let them solve those needs on their own.  While Agile doesn’t assume hand-picked teams, it does assume highly motivated team members and so further assumes that teams can be “self-organizing.”  However, with Agile, checkpoints are significantly more frequent (weeks instead of months).  Agile assumes controlled rather than free access to end users and the market, using representatives as proxies for that engagement.  Where the studied companies rely on peer pressure to motivate team members, Agile assumes the motivation exists and so expects close cooperation instead.  And finally, where the studied companies used an arm’s distance relationship with partners, Agile expects close collaboration.
6. In the final common process, the organizational transfer of learning, studied companies used two methods.  First, they documented best practices and established standards to be used in subsequent projects across the company.  Second, after the release of the product and at the conclusion of the project, they dispersed team as a means of transferring the knowledge to other groups within the company.  Both approaches are opposed in Agile thinking, instead encouraging retention of tacit rather than documented knowledge by the team and keeping the team together to affect that retention.

The article closes by describing some of the conclusions the authors reached about the approach used:

• On the positive side, with each of the studied efforts, there was some expectation that product would transform the company, and that it was a deliberate part of the strategy that the approach was intended to facilitate.  Agile approaches on the other hand assume evolutionary rather than revolutionary impact on the product and company (e.g., through frequent releases).
• On the negative side, the authors noted that success of the projects relied on massive overtime (100-150%) by team members.  By contrast, one of the Agile principles states that teams have “sustainable development, able to maintain a constant pace.”
• The authors described a risk that they thought the approach wouldn’t scale due to co-location requirements.  With teams already substantially larger than typical Agile teams, this may not be a practical concern.  But it does raise the issue that Agile methods may have limits to their scalability as well.

In conclusion we can see that with only a couple of superficial exceptions, the New New Product Development Game is nearly opposite to Agile thinking.  If it were not for the adoption of the rugby metaphor through the use of the word “scrum,” it would be easy to dismiss any thought that the approaches described in the article formed any basis for Agile thinking.  And neither the article nor  Nonaka and Takeuchi’s subsequent book The Knowledge Creating Company use the word “scrum” even once!

Figure 1 – Summary comparison of New New Product Development Game and Agile Principles