The Potential Chilling Effects of Big Data

Last week I posted about “Big Data” and how it is being used to discover new facts about people, to sift and sort them based on subtle patterns, to flag them as “risks” in this field or that, to predict their behavior, and to manipulate them for maximum profit.

Of course, humans are not sheep, and we don’t sit still when things like this happen to us. We perceive what is happening, and we change our behavior in response. We react. The effects of Big Data on privacy and society will be a game of three-dimensional chess, not checkers.

Humans are intensely social animals, keenly aware of when and how we are being observed and perceived by others. And our awareness of this game will only be intensified as big data is democratized—as it is used and adopted by individuals for self-analysis or other purposes. We’re likely to see the emergence of easy-to-use applications that make data mining available to all, and the spread of big data intuitions as people grasp what’s possible.

All this will only make us more conscious of our role as subjects of these techniques.

One of the powerful things that big data analytics can do is to count up large numbers of tiny actions and behaviors, and draw big-picture conclusions from them. Anyone who has observed the amazing effectiveness of Bayesian spam filters (see this interesting explanation of how they work; personally I use SpamBayes) cannot but be impressed by the power of mindless number crunching to ferret out meaning. Perhaps a more cautionary example is Facebook, which keeps track of your clicks to rate how interested you are in your various Friends. A developer has written an applet that will let you see the numerical scores Facebook has calculated for how much you stalk each of your Friends (as the site says, “This is really interesting, but may be embarrassing to you”).

It’s a small thing, but after I heard about that function, I started getting kind of self-conscious each time I clicked on various Friends’ updates. Not for any particular reason, it’s just that now I knew I was being watched in that way.

Given all the uses to which big data could be put, this is just the tip of a potentially very large iceberg. Over time, as the ramifications of big data analytics sink in, people will likely become much more conscious of the ways they’re being tracked, and the chilling effects on all sorts of behaviors could become considerable. Ultimately, if we imagine current trends pushed to their limits, we get the nightmare scenario: the consistent tracking and uncovering of such unconscious or semi-conscious behaviors across our lives, combined with our innate social self-consciousness, turning us into quivering, neurotic beings living in a psychologically oppressive world in which we’re constantly aware that our every smallest move is being charted, measured, and evaluated against the like actions of millions of other people—and then used to judge us in unpredictable ways.

Of course, as I said humans are not passive and will undoubtedly come up with many creative ways of subverting such a system. But that is an arms race we don’t want to enter. I have no idea how likely such a scenario is, but regardless it’s just another reason why we need good privacy protections.

Update:
Subsequent post on Big Data: Revolution or Overhyped Fad?

Add a comment (8)
Read the Terms of Use

LM412

I do not think we can trust Big Data. My personal response has been to conclude that, though I probably cannot prevent them from collecting my data and developing a profile of me, I CAN do my best to make their data as inaccurate as possible. I sometimes browse web sites I am not interested in. On retailer web sites, I click on products that I am not interested in. Most of my on-line accounts have incorrect demographic information. When shopping with a friend, we purchase items on each other's shopping list.

LM412

I do not think we can trust Big Data. My personal response has been to conclude that, though I probably cannot prevent them from collecting my data and developing a profile of me, I can do my best to make their data as inaccurate as possible. I sometimes browse web sites I am not interested in. On retailer web sites, I click on products that I am not interested in. Most of my on-line accounts have incorrect demographic information. When shopping with a friend, we purchase items on each other's shopping list.

Anonymous

My response has been to conclude that, though I probably cannot prevent Big Data from collecting information about me, I can do my best to make the information as inaccurate as possible. I sometimes browse web sites I am not interested in. On retailer web sites, I click on products that I am not interested in. Most of my on-line accounts have incorrect demographic information. When shopping with a friend, we purchase items on each other's shopping list.

Anonymous

I am not spamming you! I am trying to comment, and I do not understand what your spam filter finds objectionable! I cannot find an e-mail address for your site administrator either!

Anonymous

Thanks, interesting piece., vis a vis the comments: I think adding noise is hard unless you make it look like a signal--otherwise it is just noise and ripe for filtering.

Jay Stanley, ACLU

Thanks for your comment, LM412 (apologies if our commenting functionality is confusing, we're in the process of improving it). Most people are honest and reflexively provide true information about themselves when asked, but their trusting attitude is all too frequently not reciprocated when theat information is used and abused, and we have no obligation to provide true information about ourselves in many online contexts.

That said, it's true such strategies won’t be successful in many cases, and overall it’s too bad that we have to feel like we’re being watched and resort to those kinds of fake-out strategems, which in the offline context are the kinds of things people did in totalitarian countries like East Germany, just to protect their privacy. While doing what we can as individuals, as a society we shouldn’t accept that for our offline existences.

Anonymous

How can ACLU justify using Facebook, possibly the LEAST respectful corporation regarding personal privacy rights?

Shawn Earnest

I support NSA/NCSC, and D.O.D. Fusion-Center listing of any citizen. I also support federal military only martial law just as many other citizens do as to the fact most to all Americans are spoiled and are abusive, and abuse others constitutional rights. Main Abusers: The Domesticate-Terrorists, (Gang Banger, Gangster, or Gangstas as they call themselves, Sovereign Citizen Members are all the same), in the USA known as Gangs and the police are not allowed to round them up, cause of the constitution, the ACLU and crews.

I am disable, a veteran, brain damage, and I know I have no constitutional rights and to prove that I do have such I need thousands if not millions of dollars to prove that I do court, stupid. Since I have no rights, since I am not middle class, since I am happy, I am not middle class then neither should any of you who are most likely far more lacking in equality for all humane beings than myself, obviously.

Sign Up for Breaking News