Simple Dataset About American Colonists Shows Power of Metadata

In the best tradition of educators who manage to be both entertaining and enlightening, Duke sociology professor Kieran Healy has posted “Using Metadata to Find Paul Revere”—a fascinating demonstration of just how revealing metadata can be when subject to certain quite simple but powerful number-crunching techniques. Using simple information about 260 colonists in the years before the American Revolution (what organizations they belong to), he shows step by step how the lowest analyst at the “Royal Security Agency” could use that data to build powerful insights into what might be going on among the rebellious colonists.

The scariest thing about this is just how small and simple the starting data set is. Healy concludes:

I must ask you to imagine what might be possible if we were but able to collect information on very many more people, and also synthesize information from different kinds of ties between people! For the simple methods I have described are quite generalizable in these ways, and their capability only becomes more apparent as the size and scope of the information they are given increases. We would not need to know what was being whispered between individuals, only that they were connected in various ways. The analytical engine would do the rest!

In other words, this demonstration has just show us a hint of what an organization like the NSA can probably do with metadata.

More evidence that (as we have argued at greater length elsewhere) those downplaying the intrusiveness of metadata are way behind the times.

Add a comment (1)
Read the Terms of Use


The very WORD 'colonists' means they were rebellious. After all, one of my British friends made that fact painfully clear in what started as a joke on my part but ended with me learning that when British people call them "colonists" they're NOT giving them a compliment.

I asked him if he knew what our Thanksgiving was for, how it had started and what it meant. I mostly wanted to prove I actually know something he DOESN'T know but THAT sure failed.
He replied "Well of course I do. That's when you celebrate the birth of whatever freedoms the colonists' thought they were missing in England by going to the New World."
I don't think he believes they WERE missing any freedoms in England, but he was right about Thanksgiving even though he had ideas about what happened that I'd never heard of and haven't learned from anyone since he mentioned it.
His tone of voice, which was only slightly less polite than it usually is, made me crack up laughing.

He thinks American society is "too out of control" for his taste. I won't even tell you what he thinks of Edward Snowden.
He's part of the British Imperial Society, at least in theory. He comes from an Aristocratic family, but it's mostly figurative now, especially since the fall of the House of Lords.
Of course it didn't HELP that his very first visit to America happened to coincide with the 2000 election and the damn fiasco with the Florida votes and all that NONsense surrounding it.
He said he gathered we're "not really that different from England with the idea of ruling from one family line" (referring to the Bush family and how much they kept getting elected to political offices)
A few years later he would ask if Bush Junior was "the president you Americans have when you're not having a president." Which is funnier now than it was then since I VOTED for Bush Junior - but didn't want to tell him so. I still HAVEN'T told him that.

He did say that who we vote for makes a difference to the rest of the world, though I still can't understand that part.

Sign Up for Breaking News