Unsafe at Any Algorithm

In the autumn of 1965, a thirty-one-year-old lawyer named Ralph Nader published a book arguing that General Motors was selling cars it knew to be dangerous. Nader focused on the Chevrolet Corvair, whose rear suspension design made it prone to rolling over.

GM’s response was not to fix the car; it was to hire private detectives to dig up dirt on Nader. Investigators followed him, questioned his acquaintances about his sex life and his political views, and arranged for women to approach him in public and try to entrap him. When Nader reported the surveillance, GM’s president was summoned to testify before the United States Senate and had to apologize on national television. The resulting publicity sold more copies of Nader’s book than any advertising campaign could have, and Congress passed the National Traffic and Motor Vehicle Safety Act later that same year.

If you want to understand how industries respond to safety regulation, this story is good place to start. The playbook has not changed much in sixty years: deny the harm, attack the messenger, and insist that the market will handle everything. What history shows is that the market doesn’t handle it: regulation does.

Sweden mandated front seatbelts in new cars in 1959. The evidence for their effectiveness was already solid by then: Nils Bohlin, an engineer at Volvo, had developed the three-point belt the year before, and Volvo had made the patent freely available to every manufacturer in the world. This was not altruism—Volvo wanted to market its cars as safe—but the effect was the same.

The United States didn’t require seatbelts in new cars until 1968, and individual states did’t begin requiring drivers to wear them until the 1980s. (Australia’s Victoria became the first jurisdiction anywhere in the world to require seatbelt use in 1970—a decade before most American states got there.) Airbags, first demonstrated as viable technology in the 1950s, were not required in all new American passenger cars until 1998.

At every step, car makers argued that the requirement was premature, that consumers would choose safety features if they really wanted them, and that regulation would raise costs and kill innovation. At every step, the data showed substantial reductions in deaths after regulations changed. The argument that consumers would choose safety if they valued it was falsified by every study that examined it. When buyers compare cars, they mostly look at price, fuel economy, and styling. They do not systematically seek out crash-test ratings. This is not a character flaw; it is a predictable feature of how people make decisions under uncertainty about low-probability events.

Five years before Nader published his book, a pharmacologist at the United States Food and Drug Administration named Frances Kelsey was assigned to review an application for a new drug called thalidomide. The drug had already been approved in West Germany in 1957, where it was prescribed to pregnant women for morning sickness. By 1960, it was being sold in forty-six countries.

Kelsey was troubled by the application’s safety data. The drug affected peripheral nerves in adults, and she wanted to know more about how it crossed the placenta. The manufacturer pressured her repeatedly to approve it. She declined, asking for more data each time.

By 1961, German pediatrician Widukind Lenz and Australian obstetrician William McBride had independently linked thalidomide to severe birth defects. Children born to women who had taken the drug during early pregnancy were born without limbs, or with drastically shortened ones, as well as damage to their eyes, ears, and internal organs. 10,000 children or more were affected in countries where the drug had been approved, but thanks to Kelsey’s skepticism, the United States was spared.

The scandal prompted Congress to pass the Kefauver-Harris Amendment in 1962. Before that amendment, a pharmaceutical company needed to show only that a drug was not demonstrably harmful before selling it. After it, companies had to demonstrate that a drug actually worked, and required that patients give informed consent to experimental treatments.

It is worth thinking about that for a moment. Before 1962, you could sell a drug in the United States without proving it did anything. You just needed to avoid proving that it was immediately lethal. The thalidomide disaster changed that, but only because the disaster had been so catastrophic and so visible that the political cost of inaction became higher than the political cost of regulation.

As noted in previous articles in this series, there’s a pattern here. First comes denial: the evidence is contested, the studies are flawed, the sample sizes are too small. This phase can last for years or decades, especially when the industry funds its own research

Then comes the argument from uncertainty. Even if there is a problem, we do not know enough yet to regulate. More study is needed. Any regulation now would be premature and might target the wrong thing entirely.

Finally, there is the market argument. Consumers will demand safe products if they want them. Competition will drive manufacturers to provide safety. Regulation is unnecessary because market forces will take care of it. This argument fails empirically in case after case because consumers cannot evaluate risks they cannot observe. They cannot detect a placental crossing rate for a sedative. They cannot assess the probability that a suspension design will cause a rollover or compare the structural integrity of crumple zones. Markets aggregate preferences for things people can evaluate; they do not reliably handle latent hazards that require technical expertise and longitudinal data to detect.

Finally, after regulation is imposed, comes acceptance. The industry discovers that compliance is cheaper than predicted, that safety features are selling points, and that the regulation did not, in fact, destroy the sector. The American auto industry survived seatbelts. The pharmaceutical industry survived the Kefauver-Harris Amendment. If you ask them now, they will tell you that of course they support safety. Ask them about the next proposed regulation, though, and you will hear the same arguments they made about the last one.

When the tech industry tells you that privacy regulation will destroy innovation, or that algorithmic transparency requirements will make AI unworkable, or that holding platforms liable for content will end the internet, you are hearing a very old argument. It has been wrong before. The burden of proof runs in the other direction now: those who claim the market will handle it should be required to explain why this time is different from every other time.

see the whole series · email me

Hilts2003

Philip J. Hilts: Protecting America’s Health: The FDA, Business, and One Hundred Years of Regulation. University of North Carolina Press, 2003, 9780807854716.

Mashaw1990

Jerry L. Mashaw and David L. Harfst: The Struggle for Auto Safety. Harvard University Press, 1990, 9780674845305.

Nader1965

Ralph Nader: Unsafe at Any Speed: The Designed-In Dangers of the American Automobile. Grossman Publishers, 1965.