Machines Misbehaving: 62 Scientific Studies Showing how Big Tech’s Algorithms can Harm the Public

Might it be time to create an “FDA for algorithms?”

Photo by James Lee on Unsplash

In the United States, there is currently no federal institution that protects the public from harmful algorithms.

We can buy eggs, get a vaccine, and drive on highways knowing there are systems in place to protect our safety: the USDA checks our eggs for salmonella, the FDA checks vaccines for safety and effectiveness, the NHTSA makes sure highway turns are smooth and gentle for high speeds.

But what about when we run a Google search or look up a product on Amazon? What do we know about the safety of the algorithms behind these systems? While some jurisdictions are pursuing oversight, there is still no “FDA for algorithms,” if you will.

There is no “FDA for algorithms,” if you will.

To help show why this is so troubling, I recently conducted a literature review of 62 studies that expose how big tech’s algorithmic systems inflict a range of harms on the public, like predatory ads in a Google search or misleading health products recommended on Amazon. While government institutions have yet to deliver oversight, researchers and journalists have done amazing work to gather empirical evidence of algorithmic harms.

These “algorithm audits” will only become more important as algorithmic systems permeate more of our digital lives.

In this blog post I explain the four main categories of algorithmic harms that I found through the literature review (here is the full preprint, if you prefer):

• Discrimination, like when Amazon’s algorithmic system for screening résumés discriminated against women.
• Distortion, like how Amazon’s search algorithm promotes misinformative health products for vaccine-related search-queries.
• Exploitation, like Google’s ad system exploiting sensitive information about substance abuse for targeted advertising.
• Misjudgment, like when Facebook’s AI erroneously halted ad campaigns for small businesses.