I can’t stop thinking about Cathy O’Neil’s ‘Weapons of Math Destruction’

Algorithms influence everything around us, so questions about their fairness, reliability, and accuracy, and governance are worth understanding.

None of this was clear to me about five years ago while I was living in New York City. At the time, I wanted to understand what exactly algorithms do and how they shape the world we live in. I gave it an honest shot by listening to talks, reading research papers, and tuning in to conferences, but I just couldn’t cut through all the buzzwords and sales talk.

That changed when I read a book called Weapons of Math Destruction in 2016. Cathy O'Neil, a mathematician and data scientist, demystifies big data algorithms and explains how they are used virtually everywhere. These models, perennially praised by their creators for being the solution to mankind’s problems, are actually worsening social inequality, she writes.

WMD won the Euler Book Prize, in which O'Neil points to algorithms deployed in education, housing, advertising, insurance, and law enforcement, and explains the deleterious effect they have on communities of color as well as people from financially struggling backgrounds. I have a little library here in my apartment in Los Angeles and I always point to O'Neil's book as one of the sharpest, most straightforward, and easily comprehensible literary works on these powerful (and very often proprietary) mathematical models. This means they are hidden from public scrutiny, which obscures our need to understand and improve them.

Why WMD? — O'Neil published this book five years ago and in that time you could argue that people are starting to see the influence of these algorithms in their everyday lives. But transparency remains an issue despite advocacy by activists. In being so hidden, O’Neil writes that these algorithms encode biases without our permission, and help predatory firms exploit unsuspecting people.

Input may receive a portion of sales if you purchase a product through a link in this article. We only include products that have been independently selected by Input's editorial team.

O'Neil's background equips her with the acumen to spot these issues. After working on Wall Street, she left the world of finance to work in data science and support Occupy Wall Street. She writes:

“Most troubling, [algorithms] reinforce discrimination. If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his zip code), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a ‘toxic cocktail for democracy.’”

This worry about a “toxic cocktail for democracy” makes sense when you consider our current social and political landscape. Algorithms used for facial recognition include the ones used by the New Orleans police department, which lied about using it for years. Algorithms are behind the debilitating and mass misinformation on social media that eventually led to political uproar and violence — including QAnon-inspired Capitol riots in early January. Algorithms are being used to identify the ideal job candidate but often discriminate against people of color, immigrants, and women. Algorithms for predictive models are being used by private and public housing authorities, which are riddled with errors that target minority groups based on their gender, age, and race.

These are upsetting issues, no doubt. But I appreciate that O’Neil doesn’t end WMD on a pessimistic note. In fact, she offers readers (and hopefully policy makers) some ideas on how to build an algorithm-filled world with caution and compassion. “We have to explicitly embed better values into our algorithms,” she writes in WMD, “creating Big Data models that follow our ethical lead. Sometimes that will mean putting fairness ahead of profit.”

I love this book, so if you want to understand these programs and what it means to live in a time where our everyday lives are shaped by these algorithms with no public accountability, you can grab a copy of WMD for $17. Or you could get a used copy at Thrift Books. O'Neil probably won't mind.