The McNamara fallacy (also known as the quantitative fallacy), named for Robert McNamara, the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations (or metrics) and ignoring all others. The reason given is often that these other observations cannot be proven.
McNamara fallacy – WikipediaBut when the McNamara discipline is applied too literally, the first step is to measure whatever can be easily measured. The second step is to disregard that which can’t easily be measured or given a quantitative value. The third step is to presume that what can’t be measured easily really isn’t important. The fo[u]rth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.
— Daniel Yankelovich, “Interpreting the New Life Styles”, Sales Management (1971)
But when the McNamara discipline is applied too literally, the first step is to measure whatever can be easily measured. The second step is to disregard that which can’t easily be measured or given a quantitative value. The third step is to presume that what can’t be measured easily really isn’t important. The fourth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.
If you want to lose the Vietnam war, the first thing you do is discard what cannot be easily measured.
the biggest threat facing your team, whether you’re a game developer or a tech founder or a CEO, is not what you think | by Doc Burford | Medium
When McNamara was told he wasn’t considering what the people he was literally bombing may have felt about his actions, he wrote that down, then erased it, and said that since the feelings of the Vietnamese people couldn’t be measured, they didn’t matter.
In Hearts and Minds, the best Documentary on the Vietnam War that I’ve ever seen, the most heartwrenching scene occurs near the end, when a farmer is grieving, breaking down at the misery inflicted upon him.
“Take it back to the United States,” a farmer pleads, “tell them what happened. My daughter is dead.”
“I can’t measure it,” said McNamara, “so it must not be important.
”Fuck you, McNamara. May you and Kissinger burn forever.
the biggest threat facing your team, whether you’re a game developer or a tech founder or a CEO, is not what you think | by Doc Burford | Mar, 2024 | Medium
Above all, measurement may become counterproductive when it tries to measure the unmeasurable and quantify the unquantifiable.
Concrete interests of power, money, and status are at stake. Metric fixation leads to a diversion of resources away from frontline producers toward managers, administrators, and those who gather and manipulate data.
Muller, Jerry Z.. The Tyranny of Metrics (p. 8). Princeton University Press.
Measurable outcomes may be the least significant results of learning.
Linda McNeil
Table of Contents
Metric Fixation
The metric fixation is the seemingly irresistible pressure to measure performance, to publicize it, and to reward it, often in the face of evidence that this just doesn’t work very well.
Muller, Jerry Z.. The Tyranny of Metrics (p. 4). Princeton University Press.
The problem is not measurement, but excessive measurement and inappropriate measurement—not metrics, but metric fixation.
Muller, Jerry Z.. The Tyranny of Metrics (p. 4). Princeton University Press.
The most characteristic feature of metric fixation is the aspiration to replace judgment based on experience with standardized measurement.
Muller, Jerry Z.. The Tyranny of Metrics (p. 6). Princeton University Press.
Concrete interests of power, money, and status are at stake. Metric fixation leads to a diversion of resources away from frontline producers toward managers, administrators, and those who gather and manipulate data.
Muller, Jerry Z.. The Tyranny of Metrics (p. 8). Princeton University Press.
A key premise of metric fixation concerns the relationship between measurement and improvement. There is a dictum (wrongly) attributed to the great nineteenth-century physicist Lord Kelvin: “If you cannot measure it, you cannot improve it.” In 1986 the American management guru, Tom Peters, embraced the motto, “What gets measured gets done,” which became a cornerstone belief of metrics.3 In time, some drew the conclusion that “anything that can be measured can be improved.”4 When proponents of metrics advocate “accountability,” they tacitly combine two meanings of the word. On the one hand, to be accountable means to be responsible. But it can also mean “capable of being counted.” Advocates of “accountability” typically assume that only by counting can institutions be truly responsible. Performance is therefore equated with what can be reduced to standardized measurements.
When proponents of metrics demand “transparency” they often insinuate that probity requires making explicit and visible as much information as possible. The result is the demand for ever more documentation, ever more mission statements, ever more “goal-setting.”
The key components of metric fixation are:
- the belief that it is possible and desirable to replace judgment, acquired by personal experience and talent, with numerical indicators of comparative performance based upon standardized data (metrics);
- the belief that making such metrics public (transparent) assures that institutions are actually carrying out their purposes (accountability);
- the belief that the best way to motivate people within these organizations is by attaching rewards and penalties to their measured performance, rewards that are either monetary (pay-for-performance) or reputational (rankings).
Muller, Jerry Z.. The Tyranny of Metrics (pp. 17-18). Princeton University Press.
Metric fixation is the persistence of these beliefs despite their unintended negative consequences when they are put into practice. It occurs because not everything that is important is measureable, and much that is measurable is unimportant. (Or, in the words of a familiar dictum, “Not everything that can be counted counts, and not everything that counts can be counted.”)
Muller, Jerry Z.. The Tyranny of Metrics (p. 18). Princeton University Press.
Whenever reward is tied to measured performance, metric fixation invites gaming.
Muller, Jerry Z.. The Tyranny of Metrics (p. 19). Princeton University Press.
Because the theory of motivation behind pay for measured performance is stunted, results are often at odds with expectations. The typical pattern of dysfunction was formulated in 1975 by two social scientists operating on opposite sides of the Atlantic, in what appears to have been a case of independent discovery. What has come to be called “Campbell’s Law,” named for the American social psychologist Donald T. Campbell, holds that “[t]he more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” In a variation named for the British economist who formulated it, we have Goodhart’s Law, which states, “Any measure used for control is unreliable.” To put it another way, anything that can be measured and rewarded will be gamed. We will see many variations on this theme.
Trying to force people to conform their work to preestablished numerical goals tends to stifle innovation and creativity—valuable qualities in most settings. And it almost inevitably leads to a valuation of short-term goals over long-term purposes.
Muller, Jerry Z.. The Tyranny of Metrics (pp. 19-20). Princeton University Press.
Because belief in its efficacy seems to outlast evidence that it frequently doesn’t work, metric fixation has elements of a cult. Studies that demonstrate its lack of effectiveness are either ignored, or met with the assertion that what is needed is more data and better measurement. Metric fixation, which aspires to imitate science, too often resembles faith.
Muller, Jerry Z.. The Tyranny of Metrics (p. 20). Princeton University Press.
Easy Data
Nike invested a material amount of dollars (billions) into something that was less effective but easier to be measured vs something that was more effective but less easy to be measured. In conclusion: an impressive waste of money.
Nike: An Epic Saga of Value Destruction | LinkedIn
Metrics are McNamara Fallacied to hell. Money can be easily measured, so there’s no reason to try to measure things that can’t be measured, like “what is people’s sentiment around Valve?” or “how can we get people to trust Valve to make quality games?”
No matter what I do, how hard I try, it all comes back to this: a lot of people got into the business to do the business. They found the numbers that were easy to follow, easy to measure, and they started chasing that. They bred snakes to make snake-death bounties, but the result was just creating more snakes than ever.
A common refrain in every failure, from Marvel to Warner Brothers to Boeing, was this: “Leadership doesn’t know what they want,” and “leadership doesn’t trust the people who know what they’re doing to do their jobs.” It’s a deadly combination — people who try to use easy data to justify making decisions when they don’t know the first thing about a product, because they’re too busy numberfucking and datafucking to try to make number bigger, results in every one of these companies getting worse.
Artificial Simplicity
There is a pattern to the universe and everything in it, and there are knowledge systems and traditions that follow this pattern to maintain balance, to keep the temptations of narcissism in check. But recent traditions have emerged that break down creation systems like a virus, infecting complex patterns with artificial simplicity, exercising a civilizing control over what some see as chaos. The Sumerians started it. The Romans perfected it. The Anglosphere inherited it. The world is now mired in it. The war between good and evil is in reality an imposition of stupidity and simplicity over wisdom and complexity.
Yunkaporta, Tyson. Sand Talk: How Indigenous Thinking Can Save the World (p. 3). HarperCollins.
At the same time, the two categories are different in kind: domestication is a continual process of simplification (simple is safe), whereas wildness is a continual tendency towards complexity. As I have indicated, domestication shortens every food chain, always inserting human beings and their food animals and excluding as many other species – wild ‘pests’, ‘vermin’ and ‘weeds’ – as possible.
Nick Totton. Wild Therapy (second edition): Rewilding our inner and outer worlds (pp. 36-37). PCCS Books Ltd. Kindle Edition.
What develops is not just an organism, an individual, a child, but an immensely complex system. Research on interacting brains, bodies, and environments has long concluded that each of us is as complex as the universe, indeed probably more so (Marcus & Freeman, 2015; Swaab, 2014). This is why narrow silos can be dangerous when we deal with learning and development. Wrong ideas here can do real harm.
Narrow specializations in narrow academic silos have brought us a great deal of progress in science. But times are changing. When we face highly complex problems, narrow expertise can become dangerous. Narrow experts tend to underestimate and undervalue what they don’t know (Harford, 2011; Jenkins, 2006; Weinberger, 2012). They tend to think that their methods answer complex questions that, in reality, go well beyond their area of expertise. And they tend to engage in “groupthink” as they converge in their narrow echo chambers, advancing paradigms that are not tested against the results of other silos.
Interest is a quality that tends to elude measurement – except when it is deliberately (and perversely) reduced to ‘behaviours’. The third diagnostic criterion has always been about interests; within ABA and its variants, that has been reframed as about behaviours. Behaviourism is a reduction of dimensions which creates an illusion of scientific worth by focussing only on what we can ‘know for sure’.
Welchism
The prevailing power dynamic of our economic age, Welchism has at its heart the conviction that companies must prioritize profits for shareholders above all else, that executives are entitled to enormous wealth and minimal accountability, and that everyday employees deserve nothing more than their last paycheck. Welchism ascribes moral worth to material success, bestowing millionaire CEOs with the veneer of virtue, almost entirely irrespective of their actions. It thrives on downsizing, dealmaking, and financialization. And the Welchist worldview adopts a Darwinian attitude toward the labor market, a smug conviction that those who don’t make it are to blame for their own misfortune, that the poorest among us ultimately deserve their fate. The closest historical analog to Welchism is probably imperialism. The empires of yore had a comparable multinational reach to today’s biggest corporations, a similar willingness to confer absolute power upon their rulers, and the same tendency to exploit their subjects. Yet unlike imperialism, which has largely faded into history, Welchism still thrives today. Forty years after Welch took power, his warped worldview is still shaping our economy in ways large and small.
There is capitalism in America before Jack Welch, and after him. His career serves as a line of demarcation, a split between the past and the present. Look at the trend lines for any number of key economic indicators—wages, mergers and acquisitions, manufacturing jobs, union representation, executive compensation, corporate tax rates—and it’s clear that right around 1981, the year Welch took over, things started to go off the rails.
