In Search of Excellence

In 1982, at the time of publication of In Search of Excellence, America was looking to Japan as the rising economic force. American businesses were studying Japanese management techniques and looking to learn from their successes.[1]

In Search of Excellence went against this trend, by focusing on American companies and studying what made the most successful American companies successful.

In Search of Excellence did not start out as a book, as Tom Peters explained when interviewed in 2001 to mark the 20th anniversary of In Search of Excellence. In the same interview, Peters claims that he and Waterman were both consultants on the "margins" of McKinsey, based in the San Francisco office.[2]

In 1977 McKinsey director Ron Daniel launched two projects; the first and major one, the Business Strategy project, was allocated to top consultants at McKinsey's New York City corporate HQ and was given significant resources, but could not manage to effectively implement strategy.

Peters states that directly after graduating with a PhD from Stanford, and returning to McKinsey, Daniel handed him a "fascinating assignment."[3] Motivated by the new ideas coming from Bruce Henderson's Boston Consulting Group, Peters "was asked [by Daniel] to look at 'organization effectiveness' and 'implementation issues' in an inconsequential offshoot project nested in McKinsey's rather offbeat San Francisco office."[3] While Daniel's first project was focused on Business Strategy, this second project was concerned with Organization, which Peters defined as involving "the structure-and-people side."[2] This "Organization" project was seen as less important, according to Peters in a Fast Company interview.

Despite being described as "marginal," the project "had an infinite travel budget that allowed [Peters] to fly first class and stay at top-notch hotels and a license from McKinsey to talk to as many cool people as [he] could all around the United States and the world."[2] Peters admits that "There was no carefully designed work plan. There was no theory that I was out to prove. I went out and talked to genuinely smart, remarkably interesting, first-rate people."[2] In addition to Karl Weick and Einar Thorsrud, Peters notes that Douglas McGregor's theory of motivation known as Theory X and Theory Y was directly influential on the direction of the project.

In a 1978 article, "Symbols, Patterns and Settings," Peters argued that "shifting organizational structure" and "inventing new processes" - structure and system, respectively - were only two tools of organizational change. Peters then outlines eight "mundane" tools that every manager has at their fingertips. He described this article as a "tentative presentation" and "the first public expression of these ideas."[3]

In 1979 McKinsey's Munich office requested Peters to present his findings to Siemens, which provided the spur for Peters to create a 700-slide two-day presentation. Word of the meeting reached the US and Peters was invited to present also to PepsiCo, but unlike the hyper-organised Siemens, the PepsiCo management required a tighter format than 700 slides, so Tom Peters consolidated the presentation into eight themes. These eight would form the chapters of In Search of Excellence.

In 1980, Waterman joined Peters, and, along with Waterman's friend Tony Athos and Richard Pascale - both academics - came together at a two-day retreat in San Francisco to develop what would become known as the 7S Framework, the same framework that would organize In Search of Excellence. In June 1980, Peters published an op-ed in the Manager's Journal section of the Wall Street Journal titled "The Planning Fetish."[4] In this article, he "stressed the importance of execution and dismissed the whole idea of strategy."[5] As strategy was McKinsey's main operation at the time, this was seen as a "frontal assault" on the company, leading Mike Bulkin, the head of the New York office, to demand that Daniel fire Peters.

The primary "innovative" theme that under-girded what would become In Search of Excellence was that "structure is not organization." This also happened to be the title of a 1980 journal article authored by Bob Waterman, Tom Peters, and Julien Phillips in which they argue that the "picture of the thing is not the thing...An organizational structure is not an organization."[6] This article also introduced what would become the McKinsey 7S Framework.

In December 1981, Peters left the company, after agreeing to a fifty percent royalty split with McKinsey. Later co-author Waterman stayed at the firm for three more years, but received no royalties from In Search of Excellence

The book consisted of three parts:

1. The Saving Remnant - introduced the background and methodology for the book
2. Toward New Theory - introduced their underpinning ideas about business and management
3. Back To Basics - coverage of the 8 key "themes" found in Excellent companies

In the first chapter of the book, Peters and Waterman introduced the background for the book and their research method.

Initial investigations

They started out by interviewing leaders of successful organisations to get a feel for what themes emerged.

They then refined their initial observations, using the approach described below.

The McKinsey 7S framework

Peters and Waterman were concerned with how organizations were organized and managed. They wondered whether structure follows strategy, as Alfred Chandler had suggested.

To address this, their perspective was that:

"...any intelligent approach to organizing had to encompass, and treat as interdependent, at least seven variables: structure, strategy, people, management style, systems and procedures, guiding concepts and shared values (i.e. culture), and the present and hoped-for corporate strengths or skills."

They used these seven "variables" to create a visual framework, which became known as the McKinsey 7S Framework.

The interviews and resulting insights

They then used their 7S framework as a lens through which to evaluate organizational excellence. They conducted in-depth interviews with leaders at 43 "excellent" companies using this lens.

They then reduced the set of emerging insights from these interviews down to eight "themes."

Companies covered

Forbes.com provides a list of companies covered in the book, and their subsequent financial performance.[7] This list includes only publicly traded companies.

In Search of Excellence is a critique of a "rationalist" approach to management.

The second chapter of the book, "The Rational Model," introduces and then critiques the rationalist approach.

An example of the rationalist mindset is reproduced below:

"Professionalism in management is regularly equated with hard-headed rationality … It doesn’t teach us to love customers. It doesn’t instruct our leaders in the rock-bottom importance of making the average Joe a hero and a consistent winner … It doesn’t show, as Anthony Athos puts it, that ‘good managers make meanings for people, as well as money." (p. 29)

By contrast, a more "social" form of management takes into account the realities of what really motivates people. This set of real human motivations is explored in the third chapter, "Man Waiting for Motivation."

The fourth chapter puts these concerns into a historical context, exploring the evolution of management theories between 1900 and the time of publication of the book in the early 1980s. The latest era of management is characterised as more "social" than "rational," meaning that real human motivations drive business goals and activities. It also more "open" than "closed," meaning that outside forces such as market pressures can shape the evolution of structure and organisation within a firm.

This leads to an increasing concern with the ongoing evolution of an organization, and the role of culture in maintaining and shaping an organization.

Ultimately, these chapters would be seen in today's terms as advocating for leadership over management. Leaders articulate values and purpose, and achieve buy-in to vision and values from employees.

These chapters set the foundation for the rest of the book, which address eight core themes for the book.

Peters and Waterman found eight common themes which they argued were responsible for the success of the chosen corporations. The book devotes one chapter to each theme.

1. A bias for action: a preference for doing something - anything - rather than sending a question through cycles and cycles of analyses and committee reports
2. Staying close to the customer - learning his preferences and catering to them.
3. Autonomy and entrepreneurship - breaking the corporation into small companies and encouraging them to think independently and competitively
4. Productivity through people - creating in all employees the awareness that their best efforts are essential and that they will share in the rewards of the company's success.
5. Hands-on, value driven - insisting that executives keep in touch with the firm's essential business.
6. Stick to the knitting - remaining with the business the company knows best.
7. Simple form, lean staff - few administrative layers, few people at the upper levels.
8. Simultaneous loose-tight properties - fostering a climate where there is dedication to central values of the company combined with tolerance for all employees who accept these values.

Before and after In Search of Excellence, Peter Drucker was probably the preeminent management theorist.

"'It is frustratingly difficult to cite a significant modern management concept that was not first articulated, if not invented, by Drucker,' said James O’Toole, a University of Southern California management professor."[8]

However, while Peters and Waterman reviewed management theory over the preceding eight decades within In Search of Excellence, they did not consider Peter Drucker's contributions to management theory within their review.

According to the Los Angeles Times, Peters and Waterman had not read Drucker's work:[8]

"Thomas J. Peters, the co-author of the hugely successful In Search of Excellence, says he earned two advanced degrees, including a Ph.D in business, without once studying Drucker."

It seems unlikely that Peters and Waterman were completely unaware of Peter Drucker, if for no other reason than they cite Drucker three times in the index of In Search of Excellence.

Peters himself wrote that he read Drucker's The Effective Executive in 1968:[9]

"But the truth is that, though [Drucker's] consulting was carried out in the stratospheric confines of CEO-world, his books and articles were very comprehensible and accessible to the likes of LTJG Thomas J Peters, USN, in 1968, when Peters, age 25, left Vietnam and was assigned to a forces management team in the Pentagon. No, the Stanford Graduate School of Business from which Peters got an MBA six years later, in 1972, did not assign as much as a single word of Drucker's work. But in 1968 Peters read (devoured!) his first management text, Drucker's The Effective Executive — and was profoundly influenced by it. LTJG Peters was hardly alone!"

Peters shared in an interview that when writing In Search of Excellence, he was "pissed off" at Peter Drucker:[2]

"But I did have an agenda. My agenda was this: I was genuinely, deeply, sincerely, and passionately pissed off! ... Who was I pissed off at? At Peter Drucker, for one. Today, everybody acts as if Peter Drucker has always been one of those who gets it. Go back and read Concept of the Corporation. Peter Drucker may be an Austrian, but he’s more German than the Germans when it comes to hierarchy and command-and-control, top-down business operation. Take a look at the business bible according to Peter Drucker, and you’ll see. Organizations are about organization! You vill be in your place! That was the received order of the day. So, in my mind, Peter Drucker was the enemy. A good enemy, but still the enemy."

Apparently Peters had construed Druckman as advocating a Taylor-style reductionism:

"...what Peter Drucker had sold to General Motors ... all went back to Frederick Winslow Taylor and scientific management. The theory that Taylor devised came to be known as "the one best way." In essence, Taylorism said that every job could be reduced to a simple, repeatable, mechanical set of activities that even the stupidest, most unwilling worker could do."

After publication of In Search of Excellence, Peters re-read a wider body of Drucker's work, and commented:[8]

"'I had considered at least a little of what Bob Waterman and I had written in In Search of Excellence to be new,' Peters said ... 'But upon rereading [Drucker's] "The Practice of Management" book, to my amazement . . . I found everything we had written (was) in some corner or other of that book.'"

In 2005, on the occasion of Drucker's passing, Peters summarized Drucker's contributions to the theory of management:[9]

"Peter Drucker did arguably

(1) "invent" modern management as we now think of it;

(2) give the study and craft of management-as-profession credibility and visibility, even though biz schools like Harvard had been around for a long time; and

(3) provide a (the first?) comprehensive toolkit-framework for addressing and even mastering the problems of emergent enterprise complexity.

And he did something else incredibly important: He popularized the study of appreciation of modern management. Doubtless Mr Drucker would have been appalled to be described as a "popularizer" — after all, that was one of his abiding and biting criticisms of me."

Peter Drucker presaged and covered similar perspectives to Peters and Waterman's approach to management theory, for example in Drucker's The Practice of Management (first edition in 1954).

Jim Collins noted in Built to Last (p. 254) that a number of companies featured in both In Search of Excellence and in Built to Last, such as HP and P&G, "were tremendously influenced by Drucker's writings."

The omission of any reference to Drucker's theory of management in In Search of Excellence was a serious gap, undermining Peters and Waterman's claim that In Search of Excellence helped develop a new approach to management theory.

This omission doesn't however change the validity of the points raised by both Drucker and Peters and Waterman.

Peters and Waterman organized their work into eight "themes."

There may be issues with their chosen way of structuring this information for presentation, including atomicity and surfacing.

1. Atomicity

Sometimes the one "theme" consisted of two quite different points.

For example, "Hands-on, value driven" is about two different things:

1. Having a core set of guiding values at the core of the organisation
2. Leadership being "hands-on", for example through "management by walking around" or through open-door policies.

These are related, because leaders being hands-on is a way to institutionalise and embed the guiding values. But conceptually these are two different points, and arguably are better discussed as such.

Similarly, "Simple form, lean staff" is about two different things:

1. Simple form is about the organisational structure - what branches, divisions or companies
2. Lean staff is about a flat structure.

These are different points. So much so that the chapter on "Simple form, lean staff" does not discuss "lean staff" at all. It is covered only in an overview of the chapter, elsewhere in the book.

"Atomicity" would reduce each of the book's "theme's" to making a single major point.

2. Surfacing.

Sometimes there is an idea within a "theme" that arguably deserves to be surfaced as a main theme in itself.

For example, the theme "A bias to action" in In Search of Excellence also includes the sub-idea that excellent organizations adopt an experimental approach. Arguably this deserves to be raised to the level of a major "theme" in and of itself in the book - as it is in Jim Collins' Built To Last (in his chapter on experimentation, "Try A Lot of Stuff and Keep What Works").

In December, 2001, Fast Company printed an article, crediting Tom Peters as author, entitled "Tom Peters's True Confessions". Most of the "confessions" were humorously self-deprecating remarks (In Search of Excellence had been "an afterthought... a hip-pocket project that was never supposed to amount to much"). One of them, however, used the term "faked data:"

This is pretty small beer, but for what it's worth, okay, I confess: We faked the data. A lot of people suggested it at the time. The big question was, How did you end up viewing these companies as "excellent" companies? A little while later, when a bunch of the "excellent" companies started to have some down years, that also became a huge accusation: If these companies are so excellent, Peters, then why are they doing so badly now? Which I'd say pretty much misses the point.

[In] Search [of Excellence] started out as a study of 62 companies. How did we come up with them? We went around to McKinsey's partners and to a bunch of other smart people who were deeply involved and seriously engaged in the world of business and asked, Who's cool? Who's doing cool work? Where is there great stuff going on? And which companies genuinely get it? That very direct approach generated a list of 62 companies, which led to interviews with the people at those companies. Then, because McKinsey is McKinsey, we felt that we had to come up with some quantitative measures of performance. Those measures dropped the list from 62 to 43 companies. General Electric, for example, was on the list of 62 companies but didn't make the cut to 43 -- which shows you how "stupid" raw insight is and how "smart" tough-minded metrics can be.

Were there companies that, in retrospect, didn't belong on the list of 43? I only have one word to say: Atari.

Was our process fundamentally sound? Absolutely! If you want to go find smart people who are doing cool stuff from which you can learn the most useful, cutting-edge principles, then do what we did with Search: Start by using common sense, by trusting your instincts, and by soliciting the views of "strange" (that is, nonconventional) people. You can always worry about proving the facts later.[10]

BusinessWeek ran an article about Fast Company's article. As related by BusinessWeek, the article was actually written by Fast Company founding editor Alan M. Webber, based on a six-hour interview with Peters. Peters reviewed and approved the article prior to publication, but the actual phrase "we faked the data" was Webber's, and Peters had not actually used these words during the interview. BusinessWeek quoted Peters as saying "Get off my case. We didn't fake the data." According to BusinessWeek, Peters says he was "pissed" when he first saw the cover. "It was his [Webber's] damn word," he says. "I'm not going to take the heat for it."[11]

As early as 1984 it was apparent, to certain analysts, that the book's choice of companies was poor to indifferent. NCR, Wang Labs, Xerox and others did not produce excellent results in their balance sheets in the 1980s.

Rick Chapman titled his book on high-tech marketing fiascoes, In Search of Stupidity, as a nod to Peters's book and the disasters that befell many of the companies it profiled. He notes that "with only a few exceptions... [the excellent companies were] large firms with dominant positions in markets that were senescent or static."

In an article in Fast Company, Peters remarked that the criticism that "If these companies are so excellent, Peters, then why are they doing so badly now," in his opinion "pretty much misses the point."[10]

The research methodology employed by the authors of this book is also severely criticized by Phil Rosenzweigh in his book "The Halo Effect" [12] as the "Delusion of Connecting the Winning Dots". Rosenzweigh opines that it was not possible to identify the traits that make a company perform simply by studying already-performing companies which Peters and Waterman did.

Almost 40 years after its original publication In Search of Excellence remains a widely read classic, and an influential book for leaders and managers.

A 2002 panel of experts convened by Forbes rated In Search of Excellence as the most influential business and management book from the decades between 1980 and 2000.[13]

Inc magazine reported that by March 1999, In Search of Excellence had sold over 4.5 million copies.[14] Forbes in 2016 ranked the best-selling business books ever [15] - based on sales tracking data that was not collected before 2004 - and put Strengths Finder 2.0 at number 1 with 4.5 million copies sold. Since In Search of Excellence had equalled this level of sales in 1999, and presumably made further sales in the succeeding 17 years, there is an argument to be made that In Search of Excellence is either the best-selling business book of all time or the equal best-selling business book of all time.

Tom Peters identified several key contributions from the book, that changed the field of future management books.

First was the reorientation from a focus on strategy to recognising the importance of "soft" aspects of business like culture and people.

"The whole point of In Search of Excellence is hard is soft and soft is hard. There’s more to life than business strategy ..."[16]

Second was the use of research and specific case studies to bring ideas about management to life:

"[We] wrote case studies. And that is what everybody and his and her brother does today, but it was totally new at that time. Meaning that Peter Drucker had said extraordinary things and valuable things, but he never mentioned company names–nobody mentioned company names. We stuck our necks way out and gave the cases from IBM and McDonald’s and various other places."[17]

In Search of Excellence opened up the way for further research and publication around excellence in business. Jim Collins's books Built to Last: Successful Habits of Visionary Companies and its sequel Good to Great are widely-known influential later works in this genre.

Third, Peters reflected that In Search of Excellence brought management ideas to a broader audience than previous theorists like Peter Drucker had reached.[9]

The ideas in the book remain influential today.

The book essentially boils down to five main tenets:

1. Maintain vision and values at the core of the organisation
2. Be customer-centric
3. Respect and develop employees
4. Be nimble and adaptive, with flat management structures and loosely coupled business units.
5. Continue to build core capabilities

The authors advocated an agile, customer-centric organization with a core commitment to values.

From that perspective, many of the points made in the book are timeless rather than ephemeral - and they stand strong today.

In Search of Excellence is not widely regarded as being great at predicting future success for individual "excellent" companies.

However, the set of "excellent" companies studied, as a whole, still outperformed the market. A 2002 analysis in Forbes found that

"Over a five-, 10- or 20-year period, the Excellence index--an unweighted basket of the 32 public companies among Peter's and Waterman's 43--substantially outperformed the Dow Jones Industrial Average and the broader S&P 500. Since October 1982, when the book was published, the companies on the authors' list earned an average total return of 1,305%, or 14.1% annually. This return outdistanced the DJIA companies, which earned an average annual return of 11.3%, and the S&P, whose companies earned an average annual return of 10.1%."[18]