The Brave New World of Discrimination

In recent years, the Internet has played an important role in the continuing battle against discrimination. Because of the ability to share information instantly and broadly, discriminatory abuses and practices have now received an unprecedented level of public exposure in social media. Out of this exposure has sprung movements, such as #blacklivesmatter and other grass roots organizations, that have brought a fierce new urgency to the struggle against discrimination and renewed the prospect of meaningful change in a time of often overwhelmingly negative news.

But alongside the potential for bringing about social progress, the Internet also holds the possibility of contributing to unlawful discrimination. An example of this potential negative impact is a patent recently acquired by Facebook that could conceivably permit loan servicers to gain access to the credit ratings of a loan applicant’s social network and then use that information to determine whether the applicant qualifies for a loan. The patent combines the possibility of serious invasions of privacy with the realistic prospect of illegal lending discrimination.

It isn’t hard to conceive of how this could harm, if not ruin, lives. For example, a responsible individual with a good credit rating may find herself penalized by the credit ratings of individuals in her social network, which might factor in her neighbor’s credit scores. Given the long, proven correlation between individual and neighborhood-level credit scores and race, such an approach could functionally exclude creditworthy people of color from receiving fair credit simply because they know or live in the vicinity of other people of color who, because of the history of discrimination in lending, themselves may have lower credit scores. The possibility of this happening is not far-fetched, as we have explained to the Federal Trade Commission, and would only further entrench racial segregation and other forms of financial discrimination.

The fact that this outcome would be the result of the operation of a computer algorithm rather than the result of intentional bias is both legally and factually irrelevant. Both the Equal Credit Opportunity Act and the Fair Housing Act, the two laws most likely to be implicated by unfair denials of credit, recognize that policies that have an unfair impact may be illegal regardless of intent or lack of intent. And that is as it should be. It would be little consolation to a person denied a home mortgage or a business loan because of her online friends that the denial was the result of a computer program rather than malice. That person would still be denied the better educational and employment opportunities that she sought by purchasing a new home or the possibility of lifting her family into a more financially stable position through opening a new business.

Such a denial would be contrary to the core intent of laws designed to assure fairness and equality, which urge treating people as individuals rather than judging them because they are a member of a group. No matter how hardworking and financially careful individuals are, the actions of their online friends or their neighbors could scuttle all of their dreams. And most disturbing of all, it is likely that they would never know the reason why.

Although the patent has not yet been and may never be used, it harbors the potential to further damage the financial well-being of people still feeling the harsh effects of earlier financial discrimination. It would be wise and fair for anyone considering implementing  this patent to keep this harmful impact in mind and for financial regulators to monitor the use of the program carefully.

And beyond its own applications, the patent highlights the larger potential for algorithms and big data to perpetuate discrimination. Recent studies, such as the one finding that Google ads for high-paying jobs were more likely to show up for men than women, show that even if algorithms are not explicitly designed to discriminate, they can have negative effects on groups who have historically been discriminated against. Particularly when it comes to access to employment, housing, education, and credit —areas that the civil rights laws have recognized as particularly important — we must be vigilant to ensure that automated decision-making does not replicate existing societal discrimination.

View comments (8)
Read the Terms of Use

Steady Pie

Nothing wrong with discrimination so long as it's voluntary.

If Facebook wants to sell your information to a third party, they need to place that in their terms and conditions. If you don't like that aspect of their privacy policy, then don't bloody accept the terms and conditions and go elsewhere.

That such information *could* be used to discriminate is irrelevant. It would be irrelevant even if the third party was openly using race/sexual orientation/etc to choose customers. This notion that because you "serve the public" you therefore lose the right to refuse service is completely absurd. No transaction has been entered into prior to you closing the deal - you should be able to refuse service for quite literally whatever reason you see fit. Companies should *absolutely* be able to use whatever information voluntarily obtained to assess risk - perceived or real.

I say this as an Atheist who would probably be subject to discrimination in certain areas if my strongly held beliefs on this subject were made reality. The ACLU is wrong on most issues relating to discrimination. It really bothers me because the ACLU does a lot of good work on drug prohibition, executive overreach, foreign policy, and mass surveillance.

Anonymous

Wow, how to lose the fight before it begins. This has NOTHING to do with race.

This is a 4th amendment atrocity and a blatant disregard for justice.

By making this a "race issue" you automatically discredit the movement to some would be allies because you're not even focusing on the real issue.

Kimberly

I'm afraid to death I'm being Sting Rayed from Harris Corporation.Can you help me get information on this practice?And what I can do because I'm disabled.

Corporate Cracker

Careful Americans, you all know we identify at a basic level with clicks, groups, and friends. Those connections are based on commonalities and beliefs. Rich white Americans will always identify with other rich people. Ghetto people will always identify with ghetto people. Race has little to do with it except that most "ghetto" people are black or other "minority". Get used to it, until the "minorities" claim to be the majority, rich whites will continue to hire and advance other rich whites. Sad but we honkys can't see far enough past the money to realize we are to become a minority. Unless we can close the bean train. Ooooohhhh, hahahahaha!

Anonymous

If you are looking for Managed VPS Hosting that is isolated from other users on the server, We provides you with everything for which you have been waiting for. Our hosting is built on a blazing-fast SSD latest hardware That's Highly tuned for optimum performance. The hosting includes the latest WP version, plugins, themes, automated daily backups, anytime money back, 99.97% uptime and 24x7 support.

https://nestify.io/vps-hosting/

Arianna

This is quite nice sharing with great knowledge, I like this and will discuss with father after mine hershey’s chocolate world bus tour package.

Arianna

I will share this article with my father and will get his views about this as will be free from my east coast bus trip.

Arianna

I will must discuss on this topic with others as will be free from my tours to yosemite park.

Stay Informed