How Syndrome’s Omnidroids were Biased in The Incredibles

A brief example of measuring gender bias in machines

In a scene from Brad Bird’s 2004 masterpiece, Mr. Incredible discovers that Syndrome has killed a wave of supers using the omnidroid machines.

The whole scene is full of expertly-crafted details: as Mrs. Incredible realizes she does not know Mr. Incredible’s location, Mr. Incredible finds that Syndrome’s database does not know Mrs. Incredible’s location. Also, the animators did an amazing job with Mr. incredible’s facial reactions:

Just like the humans that create them, machines can discriminate based on gender. One popular real-world example is an Amazon hiring algorithm that was scrapped in 2018 due to gender bias. Reuters reported:

Amazon’s system taught itself that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.”

Thinking about The Incredibles recently, I wondered if Syndrome’s machines also had some gender bias. I went through the scene and created a database of superheroes, recording their “Threat Rating,” their sex, and the number of rounds in which they faced an omnidroid. Here’s what the data looks like:

Do males have higher threat scores?

One simplistic test here is to see if the average “threat score” of males is equal to the average threat score of females. If we inspect it visually, there appears to be no significant difference.

And indeed, a two-sample t-test comparing the average threat shows there is no statistically significant difference between the average threat score of male and female supers.

Do the Omnidroids perform better on males?

Another way to look for bias is to compare how long it took the omnidroids to terminate the average male vs. the average female. A visual inspection suggests there may be a difference: it looks like every female was terminated within one round.

Indeed, a t-test reveals a p value of 0.03315, which passes the conventional criteria for statistical significance. Based on this dataset, it seems that omnidroids were systematically better at terminating females, even though males and females had similar average threat scores. Maybe Syndrome should have hired an auditor.

Here is the data and analysis code if you want to look for yourself!