Artificial Intelligence at Any Cost Is a Recipe for Tyranny

This post was adapted from a presentation at an AI Now symposium held on July 10 at the MIT Media Lab. AI Now is a new initiative working, in partnership with the ACLU, to explore the social and economic implications of artificial intelligence.

It seems to me that this is an auspicious moment for a conversation about rights and liberties in an automated world, for at least two reasons.

The first is that there’s still time to get this right. We can still have a substantial impact on the legal and policy debates that will shape development and deployment of automated technologies in our everyday lives.

The second reason is Donald Trump. The democratic stress test of the Trump presidency has gotten everyone’s attention. It’s now much harder to believe, as Eric Schmidt once assured us, that “technology will solve all the world’s problems.” Technologists who have grown used to saying that they have no interest in politics have realized, I believe, that politics is very interested in them.

By contrast, consider how, over the last two decades, the internet came to become the engine of a surveillance economy.

Silicon Valley’s apostles of innovation managed to exempt the internet economy from the standard consumer protections provided by other industrialized democracies by arguing successfully that it was “too early” for government regulation: It would stifle innovation. In almost the same breath, they told us that it was also “too late” for regulation: It would break the internet.

And by the time significant numbers of people came to understand that maybe they hadn’t gotten such a good deal, the dominant business model had become so entrenched that meaningful reforms will now require Herculean political efforts.

How smart can our “smart cameras” be if the humans programming them are this dumb?

When we place “innovation” within — or atop — a normative hierarchy, we end up with a world that reflects private interests rather than public values.

So if we shouldn’t just trust the technologists — and the corporations and governments that employ the vast majority of them — then what should be our north star?

Liberty, equality, and fairness are the defining values of a constitutional democracy. Each is threatened by increased automation unconstrained by strong legal protections.

Liberty is threatened when the architecture of surveillance that we’ve already constructed is trained, or trains itself, to track us comprehensively and to draw conclusions based on our public behavior patterns.

Equality is threatened when automated decision-making mirrors the unequal world that we already live in, replicating biased outcomes under a cloak of technological impartiality.

And basic fairness, what lawyers call “due process,” is threatened when enormously consequential decisions affecting our lives — whether we’ll be released from prison, or approved for a home loan, or offered a job — are generated by proprietary systems that don’t allow us to scrutinize their methodologies and meaningfully push back against unjust outcomes.

Since my own work is on surveillance, I’m going to devote my limited time to that issue.

When we think about the interplay between automated technologies and our surveillance society, what kinds of harms to core values should we be principally concerned about?

Let me mention just a few.

When we program our surveillance systems to identify suspicious behaviors, what will be our metrics for defining “suspicious”?

Know the eight signs of terrorism

This is a brochure about the “8 signs of terrorism” that I picked up in an upstate New York rest area. (My personal favorite is number 7: “Putting people into position and moving them around without actually committing a terrorist act.”)

How smart can our “smart cameras” be if the humans programming them are this dumb?

And of course, this means that many people are going to be logged into systems that will, in turn, subject them to coercive state interventions.

But we shouldn’t just be concerned about “false positives.” If we worry only about how error-prone these systems are, then more accurate surveillance systems will be seen as the solution to the problem.

I’m at least as worried about a world in which all of my public movements are tracked, logged, and analyzed accurately.

Bruce Schneier likes to say: Think about how you feel when a police car is driving alongside you. Now imagine feeling that way all the time.

There’s a very real risk, as my colleague Jay Stanley has warned, that pervasive automated surveillance will:

“turn[] us into quivering, neurotic beings living in a psychologically oppressive world in which we’re constantly aware that our every smallest move is being charted, measured, and evaluated against the like actions of millions of other people—and then used to judge us in unpredictable ways.”

I also worry that in our eagerness to make the world quantifiable, we may find ourselves offering the wrong answers to the wrong questions.

Terror alert levels

The wrong answers because extremely remote events like terrorism don’t track accurately into hard predictive categories.

And the wrong question because it doesn’t even matter what the color is: Once we adopt this threat-level framework, we say that terrorism is an issue of paramount national importance — even though that is a highly questionable proposition.

Think about how you feel when a police car is driving alongside you. Now imagine feeling that way all the time.

The question becomes “how alarmed should we be?” rather than “should we be alarmed at all?”

And once we’re trapped in this framework, the only remaining question will be how accurate and effective our surveillance machinery is — not whether we should be constructing and deploying it in the first place.

If we’re serious about protecting liberty, equality, and fairness in a world of rapid technological change, we have to recognize that in some contexts, inefficiencies can be a feature, not a bug.

Bill of Rights first ten articles

Consider these words written over 200 years ago. The Bill of Rights is an anti-efficiency manifesto. It was created to add friction to the exercise of state power.

The Fourth Amendment: Government can’t effect a search or seizure without a warrant supported by probable cause of wrongdoing.

The Fifth Amendment: Government can’t force people to be witnesses against themselves; it can’t take their freedom or their property without fair process; it doesn’t get two bites at the apple.

The Sixth Amendment: Everyone gets a lawyer, and a public trial by jury, and can confront any evidence against them.

The Eighth Amendment: Punishments can’t be cruel, and bail can’t be excessive.

This document reflects a very deep mistrust of aggregated power.

If we want to preserve our fundamental human rights in the world that aggregated computing power is going to create, I would suggest that mistrust should remain one of our touchstones.

Add a comment (17)
Read the Terms of Use

reloquent

As a practicing software engineer, I recommend highly against allowing people like myself to define public policy via the products we build. We are in the business of building tools. We are not in a position to fully understand all of the implications of the algorithms we implement. That reluctance on my part sits on top of my discomfort with the quantity of personal data that must be collected to allow these systems to work at even a rudimentary level.

Anonymous

I agree, however it is bound to happen. Engineers working on this system should be familiar with the bill of rights.

Mentifex

Strong AI is already here, hiding in plain sight.

Anonymous

The Fourth Amendment was designed to prevent domestic spying and warrantless surveillance. Maybe it's fatal flaw - which could be amended - is that police, government officials and their contractors have embraced non-confrontational blacklisting.

Longterm, this type of blacklisting is the most evil tactic used by U.S. officials today. Any citizen can be blacklisted, for any reason, using searches and surveillance illegal under the Fourth Amendment. Once a citizen is deemed suspicious and placed on initially low level watchlists with local officials, those blacklists are then shared with state, federal and sometimes even international blacklists for a "non-crime" and "non-wrongdoing".
The "crime" may be dating the local cop's daughter or a legal First Amendment exercise on Facebook.

Since the police officer, official or their contractors never confront the blacklisted citizens, they can make up anything at all or proceed with a flawed theory. This non-confrontation - which is illegal under the supreme law of the land - robs citizens of "legal standing" in court. It's impossible to seek judicial relief if you aren't even aware of the abuses yourself and there are no statistics documented - warrantless surveillance can seem harmless to voters and the general public since the equation omits the real "harm" from the debate. Many times police, officials and contractors believe themselves it's harmless with the attitude "if you aren't doing anything wrong or illegal, you have nothing to worry about". That is the most bogus line ever created, if you merely sign a petition (legal under the First Amendment) affecting police or officials in any way (ex: police body cameras) - you may be blacklisted for life and placed o federal or international blacklists for your non-crime and non-wrongdoing. Once on a lifetime blacklist, over the years and decades it devolves into defamation which can harm your reputation. income and job opportunities. There is no incentive to remove anyone's dossier from the government databases.

Until the Fourth Amendment actually prevents unconstitutional searches and warrantless surveillance - it's totally useless as an amendment since the harm far outweighs the good.

Anonymous

Movies like "Minorty Report" and the "Lives of Others" were created to warn Americans of these dangers - instead we have been apathetic to unAmerican and illegal practices like the Bush Preemption Doctrine.

Anonymous

"Mission-Creep" is a major threat to civil liberties that the ACLU might want to focus on.

For example: What if we declared victory on our "War on a Tactic" after 9/11 (a tactic that predates our Bill of Rights)? Would the Department of Homeland Security shrink or close in proportion to the threat or would they manufacture new threats? Would state Fusion Centers shrink or close in proportion to the reduced threat?

Facts: Since 9/11 the terrorism-conviction rate based on unconstitutional searches and unconstitutional watchlists is less than 1/10 of 1%. Any legitimate program should be netting a terrorism-conviction rate of at least 50%. In other words if you watchlist or blacklist 1 million individuals, subjecting them to unconstitutional investigations and scrutiny, we should have a terrorism-conviction rate of at least 500,000 individuals if the program is legitimate.

The tactic of terrorism predates both the Bible and our Bill of Rights. The Framers of the Constitution were being terrorized by the 18th Century Red Coats and the U.S. Constitution was designed specifically to deal with this tactic.

The tragic events of 9/11 was merely a way for these agencies to exploit a tragedy to grab powers they always wanted. Based on real statistics we have won this war on a tactic, now let's tackle the mission-creep!

Craig Ziegler

Same thing, frustrated with the never ending inability to even remotely be able to deal with the existing intelligent beings........we substitute a more manageable artificial version, which begs the question who determines that standard by which we judge, or agree, that something makes the grade, and can therefore go forth with our best interests at heart. Hmm heart. Now that you are witnessing the absolute pinnacle of what derives when a organic system is subjected to the life, cultural, and educational refinement and fulfillment of the organism..........the best we can hope for is something akin to Donald Trump. Every technology, since the beginning of rudimentary society and culture, has first and foremost been weaponized. If you think that will not be the case for artificial intelligence, then dream on you little dreamers. I have no dog in the fight. I am old, no children, no relatives at all.......I saw this age coming a long way off and took precautions, since I knew I would never be able to abide this outcome, I get upset when your children are harmed, can you imagine if they were mine. So I checked out. But I will say, even I am surprised at the breadth, depth and pace the onset of this brave new world has become..........corporations with human rights, insane. Good luck with your hunt for the perfect existence, peering down form the bastion walls, at the wild things..............TTFN Craig

Anonymous

The ACLU should litigate an injuction today to outlaw "Employment Tampering" by Executive Branch agencies and contractors. It is the primary weapon used against innocent Americans with Bill of Rights guarantees. The best way to do that us with a 10+ mile restraining order, it's the only way to rein in America's Stasi.

Anonymous

The ACLU should debate: is the constitutional oath of office the greatest and most valuable "check & balance" against tyranny? It empowers subordinate government officials to refuse unconstitional and illegal orders from disloyal supervisors.

It's likely the Framers of the Constitution would have been on the side of the Bush whistleblowers and against the entire leadership of the mulitary and intel community during the Bush years. They also would be appalled by using the Espionage Act to go after those loyal to their constituonal oath. They would likely have been outraged by judges not checking the abuses of the political branches. This would be a great ACLU debate issue.

Anonymous

Any system is only as legitimate as the humans that run it. Recently, many law enforcement officers (hopefully not most officers) supported a presidential candidate that claimed he could commit premeditated homicide in New York City with no fear of criminal prosecution. In Virginia, law enforcement officers, in uniform, supporting a candidate for Governor - which is supposedly a federal crime under the "Hatch Act" (enforced by the federal Office of Special Counsel). Just this week a sheriff that violated Article VI of the U.S. Constitution, which he swore an oath to uphold, received a full pardon for violating his supreme loyalty oath. These are some, not all, of the humans running these surveillance and so-called intellgence systems.

Pages

Stay Informed