Back to News & Commentary

How Can Smart, Ethical Individuals Form Dumb, Amoral Government Agencies?

Jay Stanley,
Senior Policy Analyst,
ACLU Speech, Privacy, and Technology Project
Share This Page
September 6, 2013

Yesterday I set forth some reflections on our giant national security establishment, and how we should think about it. I argued that one very useful way of conceptualizing it is not as a result of politics or personality, but as an institution, which to the first approximation is best thought of as a mindless, amoral, and self-perpetuating primitive life form.

And by saying that, I do not mean to disparage any of the individuals who make up those bureaucracies. I live in Washington and am friends with many government workers who are excellent, thoughtful human beings.

But when you gather many human beings into an institution, that institution tends to take on a life of its own. Most of the individuals who make up the gigantic national security state are reasonably intelligent, and many of them no doubt are exceptionally so. But when you aggregate thousands of intelligent human minds together in a bureaucratic organization, the ironic result is that the collective is sometimes dumber than its individual parts. By the same token, there is no particular reason to think that bureaucracies attract a disproportionate number of amoral or immoral individuals—they surely form the same bell curve as any other group of humans when it comes to characteristics such as empathy, sensitivity, and conscience. But the collective set of such humans can exhibit a marked quality of amorality, as exhibited for example by the willingness of security bureaucracies to do horrifying things such as continue to detain people at Guantanamo who are known to pose no threat to the United States.

Complexity theorists have a concept called emergence, which refers to the fact that when large numbers of individuals interact, behaviors can emerge from the collectivity that do not appear to be the predictable result of any characteristics evident in the individuals in isolation. A classic example is the V-shaped flocking formation of birds. A computer programmer trying to create that behavior in a flock of virtual birds could tie herself in knots, but it turns out that if each “bird” is programmed with just a few simple rules (“don’t crowd your neighbors, but steer toward their average heading and position”) a flock of simulated birds will behave in extremely complex ways strikingly similar to the movements of real birds. That complex flocking behavior emerges out of simple rules in ways that are not apparent on the surface.

Perhaps when individually complex and intelligent human beings are placed together into bureaucratic organizations, they exhibit “emergent dumbness” and “emergent amorality.”

Why would it be the case that thousands of individuals acting together in a bureaucracy are dumber and more evil together than most or all of them probably are individually? No doubt legions of sociologists and political scientists have written tomes on this subject, but without having plumbed that literature I suspect the explanations include such things as:

  • The ideology of the bureaucracy. Max Weber observed that a bureaucrat is ultimately “responsible only for the impartial execution of assigned tasks and must sacrifice his personal judgment if it runs counter to his official duties.” Insofar as the vision of a bureaucracy as a machine is embraced by its participants, they will suppress their wide-ranging human intelligences and limit their judgment and discretion accordingly. Indeed as Weber points out, they are expected to do so.
  • Groupthink. The well-documented human tendencies toward conformism and “groupthink,” which can cause people to give up their critical faculties when faced with a group consensus.
  • Diffusion of responsibility. This is the tendency of people not to tackle a problem when they are surrounded by other people, each of whom assumes somebody else in the crowd will surely address it so they don’t have to.
  • Pournelle’s Iron Law of Bureaucracy. This brilliant “law” has two parts. First, “in any bureaucratic organization there will be two kinds of people: those who work to further the actual goals of the organization, and those who work for the organization itself.” Second, in every organization, it’s the people in the second category who always end up running things, while “those dedicated to the goals the bureaucracy is supposed to accomplish have less and less influence, and sometimes are eliminated entirely.”
  • Abstraction. When we say that most people are “ethical,” what we mean is that they are socialized to behave well to those who surround them—to the concrete human beings they encounter in the flesh. But we humans are generally not good at abstract thought, and it may be that it is the nature of bureaucracies to separate rulers from their victims, so that cruelty and bad policies become remote and abstract, breaking down the ethical training that has been carefully socialized in most modern human beings.

Whatever the reason or combination of reasons, it seems clear to me that government (and corporate) bureaucracies are capable of cruelty and stupidity—and that is not to impugn anybody’s motives. When I point out for example that the cybersecurity issue is being exaggerated by the security establishment to advance its powers and budgets, I’m honestly not impugning anyone’s sincerity; I’m just pointing out that the behavior is predicted by this theory. Thinking of what is happening in terms of this or that leader’s personality in the end will largely distract us from the kinds of systematic remedies that are needed to rein in the out-of-control, Sorcerer’s Apprentice nature of the national security agencies.

(Second in a series)

The third post in this series is here.

Learn More About the Issues on This Page