The Potential Chilling Effects of Big Data

Last week I posted about “Big Data” and how it is being used to discover new facts about people, to sift and sort them based on subtle patterns, to flag them as “risks” in this field or that, to predict their behavior, and to manipulate them for maximum profit.

Of course, humans are not sheep, and we don’t sit still when things like this happen to us. We perceive what is happening, and we change our behavior in response. We react. The effects of Big Data on privacy and society will be a game of three-dimensional chess, not checkers.

Humans are intensely social animals, keenly aware of when and how we are being observed and perceived by others. And our awareness of this game will only be intensified as big data is democratized—as it is used and adopted by individuals for self-analysis or other purposes. We’re likely to see the emergence of easy-to-use applications that make data mining available to all, and the spread of big data intuitions as people grasp what’s possible.

All this will only make us more conscious of our role as subjects of these techniques.

One of the powerful things that big data analytics can do is to count up large numbers of tiny actions and behaviors, and draw big-picture conclusions from them. Anyone who has observed the amazing effectiveness of Bayesian spam filters (see this interesting explanation of how they work; personally I use SpamBayes) cannot but be impressed by the power of mindless number crunching to ferret out meaning. Perhaps a more cautionary example is Facebook, which keeps track of your clicks to rate how interested you are in your various Friends. A developer has written an applet that will let you see the numerical scores Facebook has calculated for how much you stalk each of your Friends (as the site says, “This is really interesting, but may be embarrassing to you”).

It’s a small thing, but after I heard about that function, I started getting kind of self-conscious each time I clicked on various Friends’ updates. Not for any particular reason, it’s just that now I knew I was being watched in that way.

Given all the uses to which big data could be put, this is just the tip of a potentially very large iceberg. Over time, as the ramifications of big data analytics sink in, people will likely become much more conscious of the ways they’re being tracked, and the chilling effects on all sorts of behaviors could become considerable. Ultimately, if we imagine current trends pushed to their limits, we get the nightmare scenario: the consistent tracking and uncovering of such unconscious or semi-conscious behaviors across our lives, combined with our innate social self-consciousness, turning us into quivering, neurotic beings living in a psychologically oppressive world in which we’re constantly aware that our every smallest move is being charted, measured, and evaluated against the like actions of millions of other people—and then used to judge us in unpredictable ways.

Of course, as I said humans are not passive and will undoubtedly come up with many creative ways of subverting such a system. But that is an arms race we don’t want to enter. I have no idea how likely such a scenario is, but regardless it’s just another reason why we need good privacy protections.

Update:
Subsequent post on Big Data: Revolution or Overhyped Fad?