In the name of efficiency, we have increasingly disabled the ability of healthcare, industry, and education to carry out their missions effectively.

Labaree, David. The Ironies of Schooling (p. 45).

The covid-19 pandemic showed a lot of things that were wrong in American society, including terrible leadership, a frail social safety net, and a lack of investment in public goods.  But one that has particularly struck me is the way our social institutions have been become dominated by the logic of efficiency over the logic of effectiveness.  In the name of efficiency, we have increasingly disabled the ability of healthcare, industry, and education to carry out their missions effectively.  An effective system of public education needs be more loosely coupled than the efficiency model imposed by the educational standards movement. The link between teaching and learning is tight if you’re teaching to the test but it’s necessarily much looser when you’re trying to induce students to engage with the issues and skills and knowledge domains that they will need to be productive workers and civic minded citizens. 

Labaree, David. The Ironies of Schooling (pp. 10-11).

Public education is another key example of this problem.  The pressure to raise educational standards has put the emphasis on creating a structure for schooling that seeks to increase the efficiency of educational instruction through the media of curriculum standards and high stakes testing.  So you set standards in a form that is easily measurable, establish tests to find out how individual students measure up to this standard, and then and then assume that rising scores means improved teaching and learning.  What this ignores is the Goodhart’s Law:  Once a measure becomes a target it is no longer an effective measure.  When you focus on teaching to the test, scores rise.  But in the process, teaching and learning get reduced to what is easiest to measure rather than what is important for producing informed and capable members of society.  We’re increasing the efficiency of producing high scores while undermining the effectiveness of real teaching and learning.

Labaree, David. The Ironies of Schooling (p. 46).

The core problem with this strategy is that effectiveness depends on a certain degree of inefficiency.  For example, to be effective, a system of production or medicine needs a cushion of excess capacity in order to tide it over during difficult times.  Both need a store of supplies that is considerably in excess of what is required under more routine circumstances.   And both need a certain amount of redundancy:  multiple suppliers of the same goods, multiple hospitals providing the same service.  For a system of production, health care, or national security to be resilient in the face of extreme demands, we have to be willing to subsidize the kind of excess capacity that we will need in a crisis.

Labaree, David. The Ironies of Schooling (pp. 46-47).

In the last 50 years, our public sector has been hard-wired to the ethic of efficiency, in which prudent capacity building is seen as reckless waste and where major responsibilities of government are outsourced to private providers.

Labaree, David. The Ironies of Schooling (p. 48).

But what makes sense to those with power, in terms of efficiency, can lead to terrible outcomes for individuals, with many historical examples of the worst of outcomes when efficiency over autonomy becomes institutionalized and systematized.

Trust Kids!: Stories on Youth Autonomy and Confronting Adult Supremacy by Carla Bergman
In Defense of Inefficiency – YouTube

Measurable outcomes may be the least significant results of learning.

Linda McNeil

Goodhart’s Law

Goodhart’s law is an adage often stated as, “When a measure becomes a target, it ceases to be a good measure”. It is named after British economist Charles Goodhart, who is credited with expressing the core idea of the adage in a 1975 article on monetary policy in the United Kingdom:

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

Goodhart’s law – Wikipedia

McNamara Fallacy

When McNamara was told he wasn’t considering what the people he was literally bombing may have felt about his actions, he wrote that down, then erased it, and said that since the feelings of the Vietnamese people couldn’t be measured, they didn’t matter.

In Hearts and Minds, the best Documentary on the Vietnam War that I’ve ever seen, the most heartwrenching scene occurs near the end, when a farmer is grieving, breaking down at the misery inflicted upon him.

“Take it back to the United States,” a farmer pleads, “tell them what happened. My daughter is dead.”

“I can’t measure it,” said McNamara, “so it must not be important.

Fuck you, McNamara. May you and Kissinger burn forever.

the biggest threat facing your team, whether you’re a game developer or a tech founder or a CEO, is not what you think | by Doc Burford | Mar, 2024 | Medium

Metrics are McNamara Fallacied to hell. Money can be easily measured, so there’s no reason to try to measure things that can’t be measured…

the biggest threat facing your team, whether you’re a game developer or a tech founder or a CEO, is not what you think | by Doc Burford | Mar, 2024 | Medium
202. The McNamara Fallacy – YouTube

Metric Fixation

The metric fixation is the seemingly irresistible pressure to measure performance, to publicize it, and to reward it, often in the face of evidence that this just doesn’t work very well.

Muller, Jerry Z.. The Tyranny of Metrics (p. 4). Princeton University Press.

The problem is not measurement, but excessive measurement and inappropriate measurement—not metrics, but metric fixation.

Muller, Jerry Z.. The Tyranny of Metrics (p. 4). Princeton University Press.

The most characteristic feature of metric fixation is the aspiration to replace judgment based on experience with standardized measurement.

Muller, Jerry Z.. The Tyranny of Metrics (p. 6). Princeton University Press.

Concrete interests of power, money, and status are at stake. Metric fixation leads to a diversion of resources away from frontline producers toward managers, administrators, and those who gather and manipulate data.

Muller, Jerry Z.. The Tyranny of Metrics (p. 8). Princeton University Press.

A key premise of metric fixation concerns the relationship between measurement and improvement. There is a dictum (wrongly) attributed to the great nineteenth-century physicist Lord Kelvin: “If you cannot measure it, you cannot improve it.” In 1986 the American management guru, Tom Peters, embraced the motto, “What gets measured gets done,” which became a cornerstone belief of metrics.3 In time, some drew the conclusion that “anything that can be measured can be improved.”4 When proponents of metrics advocate “accountability,” they tacitly combine two meanings of the word. On the one hand, to be accountable means to be responsible. But it can also mean “capable of being counted.” Advocates of “accountability” typically assume that only by counting can institutions be truly responsible. Performance is therefore equated with what can be reduced to standardized measurements.

When proponents of metrics demand “transparency” they often insinuate that probity requires making explicit and visible as much information as possible. The result is the demand for ever more documentation, ever more mission statements, ever more “goal-setting.”

The key components of metric fixation are:

  • the belief that it is possible and desirable to replace judgment, acquired by personal experience and talent, with numerical indicators of comparative performance based upon standardized data (metrics);
  • the belief that making such metrics public (transparent) assures that institutions are actually carrying out their purposes (accountability);
  • the belief that the best way to motivate people within these organizations is by attaching rewards and penalties to their measured performance, rewards that are either monetary (pay-for-performance) or reputational (rankings).

Muller, Jerry Z.. The Tyranny of Metrics (pp. 17-18). Princeton University Press.

Metric fixation is the persistence of these beliefs despite their unintended negative consequences when they are put into practice. It occurs because not everything that is important is measureable, and much that is measurable is unimportant. (Or, in the words of a familiar dictum, “Not everything that can be counted counts, and not everything that counts can be counted.”)

Muller, Jerry Z.. The Tyranny of Metrics (p. 18). Princeton University Press.

Whenever reward is tied to measured performance, metric fixation invites gaming.

Muller, Jerry Z.. The Tyranny of Metrics (p. 19). Princeton University Press.

Because the theory of motivation behind pay for measured performance is stunted, results are often at odds with expectations. The typical pattern of dysfunction was formulated in 1975 by two social scientists operating on opposite sides of the Atlantic, in what appears to have been a case of independent discovery. What has come to be called “Campbell’s Law,” named for the American social psychologist Donald T. Campbell, holds that “[t]he more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” In a variation named for the British economist who formulated it, we have Goodhart’s Law, which states, “Any measure used for control is unreliable.” To put it another way, anything that can be measured and rewarded will be gamed. We will see many variations on this theme.

Trying to force people to conform their work to preestablished numerical goals tends to stifle innovation and creativity—valuable qualities in most settings. And it almost inevitably leads to a valuation of short-term goals over long-term purposes.

Muller, Jerry Z.. The Tyranny of Metrics (pp. 19-20). Princeton University Press.

Because belief in its efficacy seems to outlast evidence that it frequently doesn’t work, metric fixation has elements of a cult. Studies that demonstrate its lack of effectiveness are either ignored, or met with the assertion that what is needed is more data and better measurement. Metric fixation, which aspires to imitate science, too often resembles faith.

Muller, Jerry Z.. The Tyranny of Metrics (p. 20). Princeton University Press.

Further Reading


Posted

in

by

Tags: