Alternatives to the dysfunctions described in this series of post exist.
Ranked-choice voting in Ireland, Australia, New Zealand, and more than a dozen US cities
has not produced chaos:
it has produced legislatures that more closely reflect what voters actually want.
Independent redistricting commissions have measurably reduced partisan gerrymandering
in Arizona, California, and Michigan,
where independent bodies now draw district lines rather than the legislators who benefit from them.
New York City’s public matching funds for small donations have shifted the incentive structure for candidates,
making it possible to run a competitive campaign on small contributions
rather than depending on a handful of major donors.
Broad constitutional reform through sustained public participation succeeded in Iceland
following the 2008 financial crisis.
Campaigns that engaged the active participation of roughly 3.5 percent of a population
have been sufficient to force political change in case after case.
Electoral organizing, legal challenges, constitutional campaigns,
and redistricting advocacy have all worked when that threshold of organized, sustained pressure was reached.
The rich and powerful will always resist;
while the specific tools differ,
sustained, organized pressure wins time after time [Young2024].
A Paradise Built in Hell
On the morning of December 6, 1917,
a French munitions ship collided with a Norwegian vessel in Halifax Harbour, Nova Scotia.
The resulting explosion killed nearly two thousand people and flattened the north end of the city.
It was the largest human-made explosion before the nuclear age.
Within hours,
survivors were pulling strangers from rubble,
improvising hospitals in churches and railway stations,
and sharing food with people they had never met.
The next day a blizzard arrived.
Residents of Truro, two hours away by train,
loaded relief supplies and medical teams before anyone had formally organized them.
People came from across eastern Canada and the northeastern United States,
not because anyone had issued orders,
but because other people needed help.
This is not the story most people expect.
The version of human nature embedded in popular culture
and reproduced in disaster media coverage
is that when things fall apart, so do people.
Civilization is a thin crust over barbarity:
scratch the surface and you get looting, assault, and the strong preying on the weak.
This story is wrong in almost every particular,
but it keeps being told because it serves purposes that have nothing to do with accuracy.
The sociologist E.L. Quarantelli spent decades studying disasters
and came to a conclusion that surprised many people:
panic and antisocial behavior are the exception, not the rule.
Communities typically show increases in prosocial behavior:
strangers help each other,
crime rates generally fall,
and people who were barely acquaintances briefly become something like a community.
Rebecca Solnit documented this pattern across a century of catastrophes.
Her case studies,
including the 1906 San Francisco earthquake,
the 1917 Halifax explosion,
the 1985 Mexico City earthquake,
the September 11 attacks in New York,
and Hurricane Katrina in New Orleans,
illustrate Quarantelli’s findings.
Disasters reveal a capacity for mutual aid
that is usually suppressed by the atomization of modern consumer society.
Hurricane Katrina in 2005 produced the most extensively documented divergence
between media narrative and documented reality in modern history.
In the days after the storm, major news organizations reported roving gangs in the Superdome,
mass rape,
and snipers firing at rescue helicopters.
Subsequent investigation found that the reported gang violence did not happen,
the murder rate in the city did not spike,
and most of the “looting” was people taking food and water to survive.
But these lies had consequences.
Hospitals delayed evacuating critically ill patients while waiting for military escorts.
Trucks carrying food and water were turned back from routes deemed dangerous when they weren’t.
A group of survivors trying to walk across the Crescent City Connection bridge to reach Gretna,
where they had been told buses were waiting,
were turned back at gunpoint by police who said they were keeping their community safe.
The fiction of social breakdown caused deaths that the storm itself had not.
Solnit has a name for what happened in New Orleans: elite panic.
Ordinary people in a disaster tend to behave with remarkable generosity and calm,
but authorities and elites tend to panic—not about the disaster, but about the public.
Since they believe that the social order that keeps them on top
is only held together by the threat of force,
a disaster that removes their ability to enforce their rules looks like the end of civilization.
The gap between what happens in disasters and what gets reported
is also explained by what counts as news.
Editors make decisions about what to show based on what will attract attention,
and dramatic conflict attracts more attention than organized mutual aid.
The result is systematic selection bias in disaster coverage.
If the media consistently describes human nature as more violent and more selfish than it actually is,
people are pre-conditioned to believe that cooperation is unlikely,
which makes them less likely to cooperate.
Just as advertising can manufacture demand,
biased reporting can manufacture mistrust,
and in doing so, hurt us all
[Quarantelli1998,Solnit2009,Tierney2006].
The Ozone Hole That Closed
In 1974,
two chemists at the University of California published a paper
predicting that chlorofluorocarbons (CFCs) would destroy the ozone layer in the stratosphere
that shields Earth from ultraviolet radiation.
CFCs were used as propellants in aerosol cans and refrigerants in air conditioners,
and while the paper’s authors didn’t yet have a hole to point to,
they had atmospheric chemistry on their side.
The chemical industry’s response was to fund counter-research,
hire lobbyists,
and describe the scientists as alarmists
whose work was too speculative to justify regulatory action.
This was the same playbook that the tobacco industry had been running for two decades,
and for a while it worked.
The Alliance for Responsible CFC Policy,
a trade group representing the manufacturers,
argued that the science was uncertain.
Industry representatives testified before Congress
that banning CFCs would cost hundreds of thousands of jobs
and devastate the American economy.
Du Pont,
which held a large share of the CFC market,
said in 1975 that it would stop making CFCs only if a worldwide scientific consensus emerged
and the appropriate regulatory bodies took action.
This was not a promise to act;
it was a description of conditions
the company presumably believed would never be met.
Eleven years later,
in 1985,
a team from the British Antarctic Survey
reported a massive and growing thinning of the ozone layer over Antarctica every southern spring.
Their data was so far outside expected ranges
that they initially assumed their instruments were broken.
NASA confirmed the finding using satellite data that,
embarrassingly,
had been sitting in archived files for years
after automated quality-control software had flagged the anomalous readings as errors
[Roan1989].
Industry resistance collapsed in just two years,
and the Montreal Protocol was signed in 1987.
The protocol’s design explains why it worked when so many other environmental agreements have not.
It set binding phase-out schedules for ozone-depleting substances,
with different timelines for developed and developing countries.
It established trade sanctions against non-signatories,
which meant that countries outside the agreement faced economic costs for staying out.
And it created the Multilateral Fund,
which transferred technology and money from wealthy countries to developing ones
to help them adopt CFC alternatives.
This last element is the one that gets least attention
and does the most work.
When the Montreal Protocol was negotiated,
China and India were skeptical.
Both were industrializing rapidly,
both had growing demand for refrigeration and air conditioning,
and both pointed out (reasonably enough)
that the damage to the ozone layer had been caused almost entirely by wealthy countries.
The demand that they now forgo the same technologies their economic competitors had used
looked like a way of keeping them poor.
The Multilateral Fund changed the calculation.
By 2023,
the fund had disbursed over $4 billion
to help developing countries transition away from ozone-depleting substances.
China became one of the largest recipients of technology transfer funding
and one of the most consistent compliers with phase-out schedules.
India followed a similar path.
Neither country did this because their leaders suddenly became environmentalists:
compliance became economically rational once the fund made alternatives affordable.
The protocol’s structure gave it leverage that most international agreements lack.
A country that refused to sign could not import controlled substances from signatory countries
and could not export products made with those substances to them.
By the early 1990s,
enough of the global economy was covered by the agreement
that staying outside it became genuinely costly.
Du Pont,
which had spent years arguing that alternatives to CFCs were technically impossible,
announced shortly after the protocol was signed
that it had developed workable substitutes
and would accelerate their commercialization.
What had been technically impossible became technically straightforward
once the regulatory framework made the old product unmarketable.
The substitutes developed to replace CFCs were hydrofluorocarbons—HFCs.
They did not destroy the ozone layer.
They did, however, turn out to be extremely potent greenhouse gases,
some of them thousands of times more warming per molecule than carbon dioxide.
In switching from one problem to another,
the world had traded an acute crisis for a contribution to a chronic one.
The Kigali Amendment to the Montreal Protocol,
adopted in Rwanda in 2016,
addressed this.
It added HFCs to the list of controlled substances
and set phase-down schedules for them as well.
Developed countries agreed to begin reductions by 2019;
most developing countries by 2024 or 2028,
with a small number of the hottest-climate countries,
including India and Pakistan,
given until 2032.
The amendment was negotiated under the same structure as the original protocol,
with the same Multilateral Fund available to support transitions.
Climate scientists estimated at the time
that full implementation of the Kigali Amendment would avoid
up to 0.4 degrees Celsius of warming by 2100.
This is not is a story about individual consumers making better choices.
Millions of people did not read scientific papers and switch to pump-action hairspray.
The mechanism was a binding international agreement
with differentiated obligations,
a technology transfer fund,
and trade sanctions against non-participants.
The lesson for climate change shouldn’t need to be spelled out,
yet it rarely appears in public discussions.
Renewable energy investment,
corporate sustainability pledges,
and carbon pricing mechanisms will all help,
but the decisive ingredient for the ozone layer was a binding agreement with teeth.
Similarly,
if we want to mitigate the cognitive pollution caused by social media,
country-by-country age verification isn’t going to make a difference
[Parson2003].
Land to the Tiller
In 1947,
the United States government did something
that its own politicians would have called socialism
if anyone else had done it.
Under American military occupation,
Japan’s agricultural land was seized from landlords
and sold to the tenant farmers
who had been working it,
at prices set well below market value,
paid in bonds that inflation promptly turned into confetti.
This was expropriation, and it worked.
The Cold War was the reason.
American planners in Tokyo feared that rural poverty and landlord domination
were exactly the conditions in which communist movements flourished.
They had watched what happened in China and did not want a repeat,
so they did what they would never have considered at home:
they redistributed productive assets
from the wealthy to the poor
on a massive scale
and called it democratization.
Between 1947 and 1950,
roughly thirty percent of Japan’s farmland
changed hands under the land reform program.
Landlords who had lived off tenant rents for generations suddenly held bonds
whose real value was eaten away month by month,
while the tenants who had always done the work owned the fields.
The landlord class as an economic force essentially ceased to exist.
What replaced it was a rural middle class of owner-farmers.
In the following decades,
those farmers’ children moved to the cities
and provided the workforce for Japan’s industrial expansion.
The land reform did not just change who owned the fields;
it restructured the society
that would industrialize in the 1950s and 1960s
[Dreze2013,Studwell2013].
South Korea and Taiwan followed the same template,
for the same reasons,
at almost exactly the same time.
In both places,
American advisors pushed land reform
as a counter to communist land redistribution programs
that were mobilizing peasant populations elsewhere in Asia.
In South Korea,
the Land Reform Act of 1950 capped landholdings
and required excess land to be sold to the state
for redistribution to tenant farmers.
In Taiwan,
the program between 1949 and 1953
transferred land from Taiwanese landlords to the tenant farmers who cultivated it.
The compensation paid to landlords in both countries
was structured in ways that made delay expensive:
bonds whose value eroded,
or equity in state enterprises
whose worth depended on economic policies the landlords no longer controlled.
The design was intentional.
Reform administrators understood
that the landlord class would use any instrument available
to reverse the transfer,
and they structured the compensation
to reduce the resources available for that reversal.
This was not incidental:
the land reforms created the conditions
for the subsequent industrial policies to succeed,
because the rural population had both the stability
and the incentive to participate in markets
rather than spending their energy surviving extraction.
Things went differently in Latin America.
Bolivia’s 1952 land reform and Guatemala’s 1952 program
under President Jacobo Árbenz
both attempted to redistribute agricultural land
in societies with high inequality.
Bolivia’s reform survived in partial form
but was repeatedly undermined by subsequent governments.
Guatemala’s program was ended in 1954
when the CIA backed a coup
that restored land expropriated from the United Fruit Company.
Chile’s reform effort under Salvador Allende
was reversed after the 1973 coup backed by the United States.
In each case,
the political conditions
that allowed redistribution to happen
were themselves unstable,
and the reform did not survive the removal of the government that carried it out.
The Japanese, Korean, and Taiwanese cases all share a feature that is easy to overlook:
the reforms were imposed from outside,
and so were insulated from the normal political power of the landlord class.
This raises an uncomfortable question
about whether the reforms could have happened through domestic democratic politics.
The state of Kerala, in southern India,
provides an answer.
Kerala’s land reform story begins with electoral politics
rather than military occupation.
The Communist Party of India won state elections in Kerala in 1957
on a platform that included land reform,
and despite being dismissed from power by the central government before completing its program,
it returned to power and passed the Kerala Land Reforms Act in 1969.
The legislation abolished tenancy arrangements
that had kept agricultural laborers in conditions of near-permanent dependency,
placed ceilings on landholdings,
and required excess land to be redistributed.
Landlords resisted,
courts were used to delay implementation,
and the process took years to work through,
but it worked.
The Kerala case is important because
it demonstrates that land reform can happen through democratic elections
in a country
where the landlords have full political rights
and access to courts and legal challenges.
Landlords resisted energetically,
but the political organization of tenant farmers and agricultural laborers
was strong enough and persistent enough
to sustain reform across multiple election cycles
and through sustained legal obstruction.
What makes Kerala remarkable is what happened afterward.
By the 1990s the state had achieved literacy rates, life expectancy, and infant mortality figures
that compared favorably not just to other Indian states
but to countries with far higher per-capita incomes.
Land reform broke the power of a class
that had used political dominance to block public investment in health and education;
once that class’s power was broken,
public services became possible.
The words “land reform” have also been used to describe something very different.
Stalin’s forced collectivization of Soviet agriculture between 1929 and 1933
drove peasants into collective farms at gunpoint,
killed or deported millions of people labeled “kulaks” for owning a cow or two,
and caused a famine
that killed somewhere between five and eight million people in Ukraine alone.
Agricultural output collapsed for years.
Mao’s collectivization in China followed the same blueprint with even worse results.
The Great Leap Forward of 1958 to 1962
forced peasants into communes,
requisitioned grain from villages even as harvests failed,
and caused a famine that killed an estimated thirty to forty-five million people.
These programs had nothing in common with the reforms described in this lesson.
Japan, Korea, Taiwan, and Kerala gave farmers ownership of the land they worked.
Stalin and Mao abolished private ownership entirely
and replaced it with state control enforced by violence,
combined with the systematic destruction of any incentive to grow food.
Critics who invoke collectivization to argue against democratic land reform
are comparing policies that created owner-farmers with policies that destroyed them
[Conquest1986,Walder2017].
The argument made against land reform in all of these cases
was that it would destroy productivity,
undermine investment incentives,
and leave everyone worse off.
Big tech makes the same arguments today
about proposals to democratize social media and break up virtual monopolies.
There is no reason to believe the outcomes would be different
[Studwell2013].
Conclusion
These essays have described how power is structured, how harm
is produced and obscured, who bears the costs, and how regulatory and
political contests have unfolded in other industries. This final
lesson asks what the historical record shows about how change actually
happens in documented cases rather than in theory. The answer is
consistent across domains and largely unwelcome to people who prefer
to change the world through individual choices or technical solutions:
change happens when organized groups apply sustained economic and
political pressure over time, and it rarely happens any other way. The
record also shows that nonviolent campaigns have historically been
more successful than violent ones, and that the reasons why are
structural and replicable.
The dominant popular narrative about social change centers on individuals:
Rosa Parks refused to give up her seat and the Civil Rights Movement was born.
This narrative is factually wrong and strategically disabling.
Rosa Parks was the secretary of the Montgomery chapter of the NAACP
and had recently attended the Highlander Folk School,
a training center for labor and civil rights organizers.
The Montgomery Bus Boycott that followed her arrest was organized by the Montgomery Improvement Association,
coordinated carpools across a city for over a year,
and was sustained by the labor of hundreds of people whose names are not remembered.
The choice of Parks as the plaintiff in the subsequent legal case was deliberate:
other potential plaintiffs had been rejected as less strategically suitable.
This is what organized political campaigns look like.
The reduction of that campaign to one person’s spontaneous act of courage
makes it both more inspiring and less useful as a model
[Beckerman2022].
The historical record of successful social change campaigns shows
consistent structural features that cut across very different political contexts.
Indian independence was achieved through a disciplined mass movement
that combined civil disobedience,
economic disruption,
legal challenge,
and international publicity over decades.
Polish Solidarity built an independent trade union into a national opposition movement
that eventually outlasted the communist state,
sustained through martial law and repression by organizational capacity and international support.
The South African anti-apartheid campaign combined internal mass action
with an international sanctions and divestment campaign
that imposed economic costs the apartheid government could not absorb indefinitely.
The British suffragette movement used tactics ranging from
petitioning and public speaking to window-smashing, arson, and hunger strikes,
and it succeeded only after the combination of sustained pressure
and the changed political calculus produced by women’s wartime labor
made continued denial of the franchise politically untenable.
These campaigns differ in tactics, duration, context, and outcome.
What they share is organizational discipline,
sustained commitment across setbacks,
and an understanding of where the economic and political pressure points lay
[Chenoweth2011,Lakey2018].
The most rigorous quantitative analysis of this question
is Erica Chenoweth and Maria Stephan’s study of 323 resistance campaigns between 1900 and 2006.
Their finding is that nonviolent campaigns succeeded roughly twice as often as violent ones,
and that the threshold for success was consistent:
campaigns that engaged the active participation of roughly 3.5 percent of the population did not fail.
The mechanism is not mysterious.
Nonviolent campaigns can recruit from a broader population,
including people who will not take up arms but will march, boycott, strike, or withdraw labor.
Broader participation creates broader legitimacy
and makes it harder for the state to frame repression as protecting order
rather than suppressing dissent.
The 3.5 percent figure is not a guarantee;
it describes a historical pattern.
But it is a more useful starting point than the assumption that
popular majorities produce change automatically.
Economic disruption is the mechanism that connects organized pressure to actual policy change.
Boycotts raise the cost of doing business with a target.
Strikes remove the labor on which production depends,
and divestment campaigns raise the cost of capital
for targeted firms or governments and create reputational pressure on institutional investors.
The Montgomery Bus Boycott worked because it destroyed the bus company’s revenue from its Black ridership.
The South African divestment campaign worked because
it raised the cost of the apartheid state’s international borrowing
and created political problems for governments whose pension funds held South African assets.
In each case the mechanism was economic:
the people in power faced a cost-benefit calculation that changed.
Moral suasion may have affected some individuals.
It did not change the structural calculation that drove policy.
Moral arguments have a poor track record as the primary lever of social change,
and this fact is frequently misunderstood.
It is not that moral arguments are irrelevant:
they build coalitions,
provide the normative framework that justifies what a movement is asking for,
and affect the willingness of potential participants to accept personal costs.
But moral arguments addressed to those in power,
without the economic or political pressure that makes their rejection costly,
consistently fail.
Slaveholders did not free enslaved people because they were persuaded that slavery was wrong.
The tobacco industry did not voluntarily stop marketing cigarettes to children
because public health advocates published articles about harm.
Corporate privacy practices do not change because researchers demonstrate the extent of surveillance.
What changes the behavior of those who benefit from a harmful arrangement
is when not changing becomes more costly than changing,
and that calculation is economic and political.
The word “political” is consistently used disparaginly in tech culture,
as if politics were something that happens elsewhere
and that a well-run technical organization can avoid.
Politics is the process of making collective decisions in the absence of agreement on goals.
Every decision about which features to build,
which users to prioritize,
and which harms to accept does this.
The refusal to engage with questions framed as political
does not remove politics from the process;
it delegates those decisions to whoever is willing to engage.
Refusing to vote is a political act.
Refusing to join a union is a political act.
Choosing to work on a product without asking who it will harm is a political act.
The only question is whether the political choices being made are made consciously
and with an understanding of their consequences
[Young2024,Chenoweth2011].
Engineers learn to reason about direct, traceable failures:
a faulty valve leads to a boiler explosion or a bug crashes a program.
This model frames harm as rare, dramatic, and attributable,
but the most serious damage caused by industry hasn’t actually worked this way.
Instead,
it has been diffuse, cumulative, slow to emerge,
and difficult to attribute to any single decision or actor.
When leaded gasoline lowered the IQs of an entire generation of children,
no single tank of fuel caused a measurable injury.
Similarly, no particular cigarette is responsible for any particular cancer death.
The harm is real and massive,
but it was distributed across millions of exposures,
tens of millions of people,
and decades,
so those responsible didn’t meet the legal requirement of direct and proximate cause.
This pattern is no longer confined to physical toxins.
Social media platforms optimized for engagement produce radicalization and depression as a byproduct.
The harm is diffuse:
no single recommendation causes a school shooting or an act of genocide.
The long, probabilistic causal chain makes it difficult to assign fault
and therefore difficult to regulate.
The term cognitive pollution is increasingly used to describe this.
As with other forms of pollution,
it is proving difficult to regulate,
and tech companies have every incentive to maintain that difficulty.
After all,
as long as harm cannot be attributed to them,
they can externalize its cost onto the people who absorb it.
The tobacco industry did not accidentally produce uncertainty about the link between smoking and cancer.
It funded research specifically intended to produce uncertainty,
identified scientists willing to dispute the consensus,
and maintained that effort for decades after the science was settled.
The same pattern appears in the history of leaded gasoline,
asbestos,
oxycontin,
and now social media and AI
[Oreskes2010,Michaels2008].
Civil and chemical engineers are now taught about pollution,
not because the profession had a crisis of conscience,
but because society decided over the course of many decades and through many court cases
to hold polluters liable for harm.
Noise pollution, and now light pollution, are retracing that history,
and I think that if we take mental health as seriously as physical health,
it’s inevitable that we will start to hold companies accountable for the mental suffering they cause.
If we frame harm as rare, dramatic, and attributable,
responsible engineering means avoiding the specific decision that produces attributable failures.
Under the pollution model,
on the other hand,
responsible engineers are accountable for long-term cumulative effects.
A few recent court judgments in the United States may show that this shift is finally happening,
but they will undoubtedly be contested,
and as the overturn of Roe v. Wade shows,
precedent isn’t enough of a guarantee.
Legislation that holds tech companies responsible for the damage their products do to users’ mental health
can come sooner,
will be far more robust,
and will have more impact than pious gestures like banning young people from using social media
[Perrow1999,Singer2023].
How Tobacco Was Tamed
In 1950, Hill and Doll published a landmark paper in the British Medical Journal.
They had interviewed hundreds of lung cancer patients and healthy controls
in London hospitals
and found that
people with lung cancer smoked cigarettes at dramatically higher rates
than people without it.
The conclusion was not ambiguous;
the tobacco industry’s response was a masterclass in how to make clear things seem murky.
Within four years, American cigarette manufacturers had formed
the Tobacco Industry Research Committee,
later renamed the Council for Tobacco Research.
They hired scientists, funded studies,
placed ad in newspapers,
and issued press releases arguing
that the evidence was inconclusive and that more research was needed.
Iinternal documents that became public proved that they knew these were lies,
and that tobacco company executives understood the health risks long before the public did.
Their goal wasn’t to disprove the science:
it was to create enough uncertainty
that politicians felt they could not act
and smokers felt they could not be sure.
This strategy was later used by leaded gasoline producers
against evidence linking lead to cognitive damage in children,
and by fossil fuel companies against climate change.
It follows a template:
Fund scientists willing to generate alternative hypotheses,
however implausible.
Insist that correlation is not causation.
Describe any call for regulation
as an attack on personal freedom or scientific integrity.
Create front organizations with neutral-sounding names
that can advocate on your behalf without obvious commercial interest.
Delay, delay, delay—because delay is profit.
If this sounds familiar,
it’s because
tech companies that argue they are “just a platform”
and not responsible for the content they amplify
are doing exactly the same things [Oreskes2010].
Forty Years to an Agreement
The United States Surgeon General issued his landmark report in 1964,
fourteen years after Doll and Hill’s paper.
The report stated clearly that smoking caused lung cancer,
and recommended action.
The tobacco industry spent the next thirty-four years fighting a rear-guard action.
What eventually forced a reckoning was a combination of factors:
litigation by individual plaintiffs and then by state governments,
pressure from public health advocates,
investigative journalism,
congressional hearings,
and the slow accumulation of economic costs
borne by state Medicaid programs
that finally gave states
both the motivation and the legal theory to sue.
The key legal move was a shift in how states argued their cases.
Rather than proving that smoking had harmed specific individuals,
which the industry could defeat by arguing about individual risk tolerance,
states sued to recover the cost of treating sick smokers.
Mississippi was first in 1994,
and by 1998 forty-six states had reached the Master Settlement Agreement
with the four largest cigarette manufacturers.
The companies agreed to pay $206 billion over twenty-five years,
restrict marketing to minors,
and disband the organizations they had used to manufacture doubt.
It was a victory,
but it took 40 years to go from a clear scientific finding to a partial legal resolution,
and even then the industry survived and moved into new markets.
Other countries responded slowly, but they did respond.
For example,
consider what happened in Australia.
The Labor government introduced legislation requiring plain packaging for all tobacco products:
no logos, no distinctive colors,
Just the brand name in a standardized font on a drab olive-brown background,
surrounded by large graphic health warnings.
The Tobacco Plain Packaging Act passed in 2011 and took effect in December 2012.
The industry’s response was immediate and coordinated.
British American Tobacco, Philip Morris, and Imperial Tobacco
challenged the law in Australia’s High Court,
arguing it amounted to an unlawful acquisition of their intellectual property.
The High Court rejected this unanimously in August 2012.
Then Philip Morris International filed a challenge under
an obscure investor protection treaty between Hong Kong and Australia,
arguing that plain packaging violated the treaty’s provisions
on the fair treatment of foreign investors.
That proceeding dragged on for years
before an arbitral tribunal dismissed it in 2015
on the grounds that Philip Morris had restructured its Australian operations
specifically to gain access to the treaty—a move so transparently opportunistic
that even the arbitrators were not impressed.
The ISDS Gambit
Philip Morris’s case against Australia
was brought under a mechanism called
investor-state dispute settlement, or ISDS.
ISDS clauses appear in many bilateral and multilateral trade agreements
and allow foreign investors to sue governments in private arbitration tribunals
when government actions damage their investments.
The intent was to protect foreign businesses from arbitrary expropriation
by governments in countries with weak rule of law.
In practice,
the effect has been to give corporations a veto mechanism over democratic regulation.
Philip Morris was not a Hong Kong company.
It restructured its corporate holdings in 2010
specifically to route its Australian business through a Hong Kong subsidiary.
The case is now a standard example in discussions of how ISDS can be weaponized.
Several countries have since renegotiated or withdrawn from treaties
with broad ISDS provisions.
The European Union’s reformed trade agreements
have moved toward investment courts with public judges
rather than private arbitration panels.
None of this happened quickly,
and none of it would have happened
without the Australian experience demonstrating the abuse clearly enough
that reformers had a concrete case to point to.
What eventually made the difference with tobacco was not better science but organized pressure
operating through multiple channels simultaneously.
Litigation created financial costs large enough to change corporate behavior.
Public health campaigns shifted popular attitudes,
which in turn changed what politicians felt they could support.
Investigative journalism revealed that the industry had known what it denied knowing,
which destroyed the credibility of its scientific spokespeople.
International organizations like the World Health Organization
created a Framework Convention on Tobacco Control
that gave national health ministries
political cover and legal tools
they had previously lacked.
None of these were sufficient on their own:
litigation alone produced settlements that left the industry intact,
public health campaigns alone had failed for decades against the industry’s advertising budgets,
and regulation alone was blocked by industry lobbying.
It was the combination that worked,
albeit slowly.
The question for anyone wanting to rein in tech companies is therefore
how long it takes
to build a combination of litigation, regulation, and public pressure
strong enough to impose costs on an industry
that is wealthy enough and politically connected enough to resist?
The tobacco case suggests the answer is measured in decades,
and that even then you get a settlement rather than a solution
[Brandt2007,Epstein2007].
Unsafe at Any Speed
In the autumn of 1965,
a thirty-one-year-old lawyer named Ralph Nader
published a book arguing that General Motors was selling cars it knew to be dangerous.
Nader focused on the Chevrolet Corvair,
whose rear suspension design made it prone to rolling over.
GM’s response was not to fix the car;
it was to hire private detectives to dig up dirt on Nader.
Investigators followed him,
questioned his acquaintances about his sex life and his political views,
and arranged for women to approach him in public and try to entrap him.
When Nader reported the surveillance,
GM’s president was summoned to testify before the United States Senate
and had to apologize on national television.
The resulting publicity sold more copies of Nader’s book than any advertising campaign could have,
and Congress passed the National Traffic and Motor Vehicle Safety Act later that same year [Nader1965].
If you want to understand how industries respond to safety regulation,
stories like these are a good place to start.
The playbook has not changed much in sixty years:
deny the harm,
attack the messenger,
and insist that the market will handle everything.
What history shows is that the market doesn’t handle it:
regulation does.
Sweden mandated front seatbelts in new cars in 1959.
The evidence for their effectiveness was already solid by then:
Nils Bohlin,
an engineer at Volvo,
had developed the three-point belt the year before,
and Volvo had made the patent freely available to every manufacturer in the world.
This was not altruism—Volvo wanted to market its cars as safe—but the effect was the same.
The United States didn’t require seatbelts in new cars until 1968,
and individual states did’t begin requiring drivers to wear them until the 1980s.
(Australia’s Victoria became the first jurisdiction anywhere in the world
to require seatbelt use in 1970—a decade before most American states got there.)
Airbags,
first demonstrated as viable technology in the 1950s,
were not required in all new American passenger cars until 1998.
At every step,
car makers argued that the requirement was premature,
that consumers would choose safety features if they really wanted them,
and that regulation would raise costs and kill innovation.
At every step,
the data showed substantial reductions in deaths after regulations changed.
The argument that consumers would choose safety if they valued it
was falsified by every study that examined it.
When buyers compare cars, they mostly look at price, fuel economy, and styling.
They do not systematically seek out crash-test ratings.
This is not a character flaw;
it is a predictable feature of how people make decisions under uncertainty
about low-probability events [Mashaw1990].
Five years before Nader published his book,
a pharmacologist at the United States Food and Drug Administration
named Frances Kelsey was assigned to review an application for a new drug
called thalidomide.
The drug had already been approved in West Germany in 1957,
where it was prescribed to pregnant women for morning sickness.
By 1960, it was being sold in forty-six countries.
Kelsey was troubled by the application’s safety data.
The drug affected peripheral nerves in adults,
and she wanted to know more about how it crossed the placenta.
The manufacturer pressured her repeatedly to approve it.
She declined, asking for more data each time.
By 1961,
German pediatrician Widukind Lenz and Australian obstetrician William McBride
had independently linked thalidomide to severe birth defects.
Children born to women who had taken the drug during early pregnancy were born without limbs,
or with drastically shortened ones,
as well as damage to their eyes, ears, and internal organs.
10,000 children or more were affected in countries where the drug had been approved,
but thanks to Kelsey’s skepticism,
the United States was spared.
The scandal prompted Congress to pass the Kefauver-Harris Amendment in 1962.
Before that amendment,
a pharmaceutical company needed to show only that a drug was not demonstrably harmful before selling it.
After it,
companies had to demonstrate that a drug actually worked,
and required that patients give informed consent to experimental treatments.
It is worth thinking about that for a moment.
Before 1962,
you could sell a drug in the United States without proving it did anything.
You just needed to avoid proving that it was immediately lethal.
The thalidomide disaster changed that,
but only because the disaster had been so catastrophic and so visible
that the political cost of inaction became higher than the political cost of regulation.
As noted several times,
there’s a pattern here.
First comes denial:
the evidence is contested,
the studies are flawed,
the sample sizes are too small.
This phase can last for years or decades,
especially when the industry funds its own research
Then comes the argument from uncertainty.
Even if there is a problem, we do not know enough yet to regulate.
More study is needed.
Any regulation now would be premature and might target the wrong thing entirely.
Next is the market argument.
Consumers will demand safe products if they want them.
Competition will drive manufacturers to provide safety.
Regulation is unnecessary because market forces will take care of it.
This argument fails empirically in case after case
because consumers cannot evaluate risks they cannot observe.
They cannot detect a placental crossing rate for a sedative.
They cannot assess the probability that a suspension design will cause a rollover
or compare the structural integrity of crumple zones.
Markets aggregate preferences for things people can evaluate;
they do not reliably handle latent hazards
that require technical expertise and longitudinal data to detect.
Finally, after regulation is imposed,
comes acceptance [Hilts2003,Savedoff2012].
The industry discovers that compliance is cheaper than predicted,
that safety features are selling points,
and that the regulation did not, in fact, destroy the sector.
The American auto industry survived seatbelts.
The pharmaceutical industry survived the Kefauver-Harris Amendment.
If you ask them now,
they will tell you that of course they support safety.
Ask them about the next proposed regulation,
though,
and you will hear the same arguments they made about the last one.
When the tech industry tells you that privacy regulation will destroy innovation,
or that algorithmic transparency requirements will make AI unworkable,
or that holding platforms liable for content will end the internet,
you are hearing a very old argument.
It has been wrong before.
The burden of proof runs in the other direction now:
those who claim the market will handle it
should be required to explain why this time is different from every other time.
How the Rivers Ran Again
In December 1952,
cold air trapped a layer of warm, smoky air close to the ground in London.
For four days,
a yellow-brown fog of coal soot blanketed the capital,
so thick that people could not see their own feet.
Buses stopped running because drivers could not see the road.
Cattle at the Smithfield show were killed before they could suffer further.
People died in their homes, in hospitals, and on the streets.
The British government’s initial response was to deny that the fog had killed anyone;
a spokesman suggested that the excess deaths were caused by influenza.
At least 4,000 people died in those four days,
and researchers later estimated the total at closer to 12,000
once the delayed effects on the elderly and the already-sick were counted.
The government finally acknowledged the connection in 1953,
under sustained pressure from Members of Parliament whose constituents had died.
The Clean Air Act followed in 1956,
restricting the burning of coal in domestic hearths
and requiring industrial smokestacks to be tall enough to disperse their emissions.
Air quality in London improved measurably within years.
The great smogs did not return.
The Thames had been in trouble for much longer.
By the mid-nineteenth century,
the river was an open sewer.
The summer of 1858 was so bad that Members of Parliament abandoned their riverside building
because the smell made work impossible.
Victorian engineers built a sewer system,
and things improved somewhat,
but a century later the Thames through London was still functionally dead.
Oxygen levels in the water were so low that fish could not survive;
a survey in the 1950s found none at all in a long stretch of the river.
What changed was not public disgust—Londoners had been disgusted by the Thames for two hundred years.
What changed was enforceable law.
The overhaul of sewage treatment in the 1960s,
driven by statutory requirements,
reduced the organic load entering the river.
Oxygen levels climbed,
and by the early 1970s,
fish were beginning to return to parts of the river
that had been lifeless within living memory.
In 1983,
a salmon was caught in the Thames for the first time since the 1820s.
That gap—one hundred and sixty years—tells you something about how long environmental damage persists
and how long it takes to undo.
On June 22, 1969,
the Cuyahoga River in Cleveland, Ohio, caught fire.
This sounds dramatic,
but it was also the thirteenth time the river had caught fire since 1868.
Oil, chemicals, and other industrial waste had been flowing into the Cuyahoga for decades,
and fires were not unusual.
What made 1969 different was a photograph.
Time magazine published images of the burning river,
and the story reached an audience that had never heard of it before.
Public outrage followed.
That,
combined with pressure from environmental advocates
who had been working for years without much political traction,
contributed directly to the passage of the US Clean Water Act in 1972
and the establishment of the Environmental Protection Agency.
The Cuyahoga itself is now a recreational river—people kayak on it.
This is not because Cleveland’s industries suddenly became virtuous;
it is because the Clean Water Act made pollution costly
in a way that the previous century of moral condemnation had not.
On November 1, 1986,
a fire broke out in a Sandoz chemical warehouse in Schweizerhalle,
near Basel, Switzerland.
Firefighters used water to fight the blaze,
and the runoff entered the Rhine,
carrying roughly thirty tonnes of pesticides, fungicides, and mercury compounds.
The chemical plume moved downstream through Germany and into the Netherlands,
killing eels and fish for hundreds of kilometers.
In some stretches the river smelled of insecticide.
The eel population, already stressed, was devastated.
Drinking water intakes along the river had to be shut down.
The catastrophe made the politicians of every country along the Rhine’s banks understand,
in a way that years of incrementally worsening data had not,
that the river’s problems were shared problems
and could only be solved by shared commitments.
The Rhine Action Programme,
signed by Germany, France, the Netherlands, Luxembourg, and Switzerland,
set binding targets for the reduction of pollutants.
By the early 1990s,
salmon had returned to the Rhine for the first time in decades.
The river is now among the most intensively monitored and regulated waterways
in the world [Nixon2011].
From the 1930s through the 1960s,
the Chisso chemical company discharged mercury-containing wastewater
into Minamata Bay in Kumamoto Prefecture, Japan.
The mercury accumulated in fish and shellfish,
which the local population ate as a dietary staple.
Cats,
which ate fish scraps,
suffered first and became an early warning that was ignored.
Beginning in the 1950s,
residents began experiencing severe neurological symptoms:
loss of coordination, numbness, vision and hearing damage, convulsions.
Children were born with profound disabilities.
Chisso denied responsibility for years,
and the Japanese government was slow to act.
Official recognition of the disease and its cause came only in 1968,
more than a decade after the symptoms first appeared.
By that point,
tens of thousands of people had been exposed,
and thousands were severely affected.
The legal battles over compensation continued for decades.
Japan’s response to Minamata and related industrial poisoning cases
produced some of the strictest environmental law in the world by the 1970s,
including the Basic Environment Law
and statutory rights that allowed victims to sue companies for health damage.
The country that had allowed Minamata to happen
became one of the first to enshrine victims’ right to a clean environment in statute.
The Great Smog, the Thames, the Cuyahoga, the Rhine, and Minamata
are not stories about environmental virtue.
No sudden wave of ecological consciousness swept through London in 1955 or Basel in 1987.
What changed in each case was the legal and economic cost of pollution.
Industries and municipalities
that had treated rivers and air as free dumps for a century
changed their behavior when they faced fines,
required upgrades,
and liability for damages.
The mechanisms differed:
criminal penalties in some jurisdictions,
civil liability in others,
international treaty obligations in others.
The result was the same.
When it became costly enough,
the behavior changed.
Markets do not price externalities without compulsion [Singer2023].
The reasons are simple;
what is complicated is the politics of making industries pay for costs
they have been externalizing for free.
Every major environmental regulation in the twentieth century
was fought by the industries it affected,
using arguments about economic harm that turned out to be exaggerated
and predictions of technological impossibility
that turned out to be wrong.
The same thing is happening now with attempts to regulate the harm caused by
social media, AI, and large tech platforms’ surveillance of everyday life.
Two structural interventions have produced results.
The European Union’s Digital Markets Act,
which took effect in 2024,
requires platforms designated as “gatekeepers”
(those with market capitalizations above €75 billion
or monthly user bases above 45 million in Europe)
to allow interoperability with competing services,
to refrain from self-preferencing their own products in search results,
and to allow users to uninstall pre-installed software.
Fines for non-compliance reach 20% of global revenue.
The act is the first regulatory framework
designed specifically around the leverage that platform dominance creates,
rather than around the consumer prices those platforms charge.
India’s Unified Payments Interface uses a different model:
intervene before dominance rather than after.
Instead of regulating private platforms that have already achieved lock-in,
India built public payment infrastructure
that any platform can connect to on equal terms.
Google Pay, PhonePe, and Paytm compete on the same rails;
none owns the customer relationship—that belongs to the user’s bank account.
No single platform can raise fees on the underlying infrastructure
because the infrastructure is public.
Brazil’s Pix system follows similar principles,
as do comparable approaches adopted by central banks in Ghana and Sri Lanka.
The question today is not whether enshittification can be stopped,
but why regulators in Canada, the US, and elsewhere choose not to stop it
[Shapiro1999,Sapp2026].
When the Diagnosis Is Wrong
The case studies above share a pattern:
industry denial, manufactured uncertainty,
and eventual regulation.
Another kind of regulatory failure is equally instructive:
one in which the pressure for regulation is genuine and well-intentioned
but aimed at the wrong target.
Moral panics about new media are old.
Dime novels corrupted working-class youth in the 1880s.
Comic books produced juvenile delinquents in the 1950s,
according to psychiatrist Fredric Wertham’s Seduction of the Innocent [Wertham1954].
His testimony brought comic books before a Senate subcommittee in 1954
and led directly to the Comics Code Authority,
a self-regulatory body that banned dark content so comprehensively
that it effectively gutted the medium for a generation.
Rock music followed,
then Dungeons and Dragons;
in each case the new medium was identified as uniquely dangerous to children,
its effects described as direct and irreversible,
and the evidence offered was a mixture of anecdote,
dubious laboratory studies,
and credentialed testimony.
Video games arrived in force in 1993,
when a Senate subcommittee held hearings focused on two games:
Mortal Kombat, which allowed players to rip an opponent’s spine out as a finishing move,
and Night Trap, which featured vampires draining blood from actresses in a B-movie setting.
The hearings generated excellent television
and led directly to the creation of the Entertainment Software Rating Board,
the voluntary age-based rating system still in use today.
The legislators’ alarm was understandable,
but the content that generates maximum outrage is chosen for its visceral impact,
not because it represents what most people actually play.
The average video game in 1993,
like the average video game today,
involved puzzles, sports, or platform navigation.
Then in April 1999,
two students at Columbine High School in Colorado
killed twelve classmates and a teacher before taking their own lives.
Both had played Doom.
Senators introduced legislation,
retailers pulled games from shelves,
and the attorney Jack Thompson spent years filing lawsuits
claiming games were murder simulators and that the industry bore direct responsibility for the shootings,
winning settlements before being permanently disbarred for misconduct.
Dave Grossman offered a more careful version of the alarm [Grossman1995,Grossman1999].
His claim rested on military training research:
fewer than a quarter of soldiers in World War II actually fired their weapons in combat,
so the Army redesigned its training using operant conditioning with human-silhouette targets
to raise that rate in Korea and Vietnam.
Violent video games applied the same conditioning to children
without consent or any ethical framework.
Unlike the public panic, his argument was internally coherent.
Unfortunately,
the scientific literature did not validate it.
The core methodological problem is that laboratory measures of aggression
bear little relationship to real-world violence.
Such measures typically involve things like
how loud a noise blast a participant gives an opponent,
or how much hot sauce they pour for someone who dislikes spicy food.
Studies that find effects measure immediate post-game behavior in artificial settings,
and the effect sizes are small.
Large longitudinal studies,
which are better positioned to detect real-world outcomes,
consistently find no meaningful relationship between video game consumption and violent behavior.
Meta-analyses that account for publication bias—the tendency of journals
to publish positive findings—find that the field systematically overestimated effects
because studies finding no relationship were less likely to be published
[Ferguson2015,Markey2017].
The simplest check on the strong version of the argument is cross-national.
Japan and South Korea are among the highest per-capita consumers of video games in the world,
and South Korea built a multi-billion-dollar professional esports industry.
Both countries have dramatically lower rates of violent crime than the United States.
The Netherlands and other Northern European countries show the same pattern.
Gun availability, income inequality, and the specific history of racially organized social violence
are better explanations for American rates of violence
than video game consumption,
but are also much harder for the public and public figures to face.
In 2011 the Supreme Court settled the legal question,
if not the empirical one.
California had enacted a law restricting the sale of violent video games to minors.
The Court struck it down in Brown v. Entertainment Merchants Association,
applying the same First Amendment analysis it would apply to books or films.
The majority noted that the research California presented
did not establish a causal link between violent games and harm to minors,
and that the burden of proof for restricting expression falls on those who seek restriction.
That burden had not been met.
By then the panic was already fading,
displaced by fresh anxieties about social media and smartphones.
What had not faded were the harms the panic had never addressed.
While legislators debated spine-ripping finishing moves,
the games industry had been building something that warranted far more scrutiny: loot boxes.
A loot box is a randomized reward purchased with real money;
you pay to receive an item of unknown value,
which may be common or rare.
Children’s games marketed this mechanic aggressively to young players.
Several European regulators eventually concluded that loot boxes constitute gambling.
Belgium banned them in 2018.
The United Kingdom’s Gambling Commission produced guidance treating certain loot box mechanics
as gambling products requiring the same protections applied to casinos.
The United States,
whose legislators had spent years fighting over Mortal Kombat,
moved slowly.
The lesson is not that moral panics are always wrong,
or that industries accused of harm should be left alone.
It is that effective regulation requires an accurate diagnosis of the actual harm,
not the harm that generates the most compelling congressional testimony.
The video game violence campaign failed because the evidence never supported the hypothesis;
the energy expended on it left the real harm—addictive design and gambling mechanics
in games marketed to children—largely unaddressed.
When a proposed regulation is described as protecting children,
the right question is not only whether children need protecting,
but from exactly what,
by what mechanism,
and whether the remedy addresses that mechanism.
We don’t have to wait for disaster to start the process.
The world dealt with the hole in the ozone layer before it cost lives,
and we could choose to act now on social media and AI.
In 1987, the Welsh government planted 700,000 trees near Newtown
to absorb carbon dioxide.
Most of those trees won’t reach maturity for another thirty years.
The people who planted them knew perfectly well they would never see the benefit.
Nobody forced them to do it.
They just thought it was the right thing to do.
Defending that intuition is harder than it looks.
We normally ground moral obligations in relationships:
you owe something to someone because you made a promise,
caused them harm,
or stand in some ongoing connection to them.
Future people don’t satisfy any of these conditions.
They can’t help us, sue us, or cast votes.
And yet we talk and (sometimes) act as if we owe them something.
The question is whether that behavior reflects a coherent moral position
or is just a vague sentiment we invoke when it’s convenient and ignore when it’s expensive.
The Case For
Three philosophical traditions give different answers, and they mostly agree.
Utilitarians count welfare:
if future people will exist,
their suffering and flourishing count in the moral calculation.
Since there could be many more future people than present ones,
their aggregate welfare could easily outweigh ours.
This is the premise behind William MacAskill’s argument
that the most important thing we can do is improve the long-run trajectory of civilization
rather than address present problems [Macaskill2023].
Rights theorists start from a different place but reach a similar conclusion.
If future people will have the same moral status as present people—which
seems hard to deny—then the fact that they don’t exist yet
is morally arbitrary in the same way that geographic distance is morally arbitrary.
We don’t think we owe less to someone suffering in a country we’ve never visited.
Why would temporal distance be different?
John Rawls asked his readers to imagine designing a society
without knowing what position they would occupy in it.
He extended this thought experiment across generations:
if you didn’t know which generation you’d be born into,
you would want rules that protected each generation
from being stripped of resources by its predecessors [Rawls1999].
The philosopher Edmund Burke made a version of this point two centuries ago,
describing society as a partnership between the dead, the living,
and those yet to be born [Burke1790].
These arguments are not identical, but they converge on the idea that
our obligations extend beyond the people we can see and hear.
The Problem of Non-Identity
The philosopher Derek Parfit spent years trying to figure out
whether any of this actually made sense.
His conclusion was unsettling [Parfit1987].
The problem is that the specific people who will exist in the future
depend on the specific choices we make today.
If we had enacted serious climate legislation in 1990,
the people born in 2050 would be different people from the ones who will actually be born.
So when we say we are harming future people by burning fossil fuels,
which people are we talking about?
The people who will actually exist owe their existence
to the chain of events that includes our decisions.
This doesn’t mean the future doesn’t matter.
It means that if we want to defend obligations to future generations,
we probably can’t do it by pointing to individual harm.
We need an argument that works at the level of conditions and possibilities
rather than identifiable victims.
The Legal Gap
The law has mostly not solved this problem,
which explains why our legal obligations to future people are so weak.
Courts require an identifiable party who has suffered a concrete injury.
Future generations fail this test on both counts.
The workarounds are real but imperfect.
Environmental impact assessments require governments to consider long-term consequences.
Some constitutions are designed to be difficult to amend,
protecting future generations from present majorities.
In 1993,
the Philippine Supreme Court allowed a group of children to sue the government
on behalf of future generations in Oposa v. Factoran,
a ruling that has influenced environmental law in other countries.
Wales and Hungary have created parliamentary commissioners
specifically tasked with representing future interests.
These are genuine innovations.
But the structural problem remains:
today’s legislators have no electoral incentive
to sacrifice for people who cannot vote.
The Case Against, and the Cynicism Problem
The philosophical arguments for intergenerational obligation are more coherent
than the arguments against.
The non-identity problem is the most serious challenge,
and it is a challenge to one particular way of framing the obligation,
not to the obligation itself.
The utilitarian, rights-based, and contractarian cases all survive it largely intact.
What is harder to survive is the behavioral evidence.
The gap between what we say about future generations and what we actually do is not subtle.
Greenhouse gas emissions are still rising;
fisheries are still being depleted beyond recovery,
and pension systems are underfunded while present spending continues.
Hans Jonas argued that our technological power has outrun our ethical frameworks
precisely because those frameworks were designed for a world
where our actions had local, reversible consequences [Jonas1984].
That may be true,
but it doesn’t explain why the frameworks have remained so convenient.
The uncomfortable hypothesis is that “what we owe the future” functions primarily as
a way of gesturing at seriousness without having to do anything.
Thirty years of climate pledges backed by thirty years of continued extraction
is fairly strong evidence that the stated preference is not the actual preference.
Every cost-benefit analysis used in actual policy applies a discount rate to future welfare,
which means a dollar of harm to someone in 2100 is worth almost nothing in present calculations.
This is not a fringe view;
it is embedded in standard regulatory practice.
There is also a pointed objection from the present:
roughly 700 million people alive right now lack clean water, food security, or basic medical care.
Diverting moral energy toward speculative future benefits
while ignoring identifiable present suffering is a luxury.
The language of obligations to future generations can even function
as cover for deferring justice for communities already bearing the costs of extraction and pollution,
who are disproportionately poor.
This is not an argument that the obligation doesn’t exist.
People regularly fail to act on obligations they acknowledge;
we don’t conclude that theft is acceptable because people steal.
But it is an argument that philosophical clarity about what we owe
is doing almost no work in the world,
and that the useful question may not be “do we have a duty to future generations?”
but rather “what would actually change our behavior?”
When the Future Is Used Against the Present
MacAskill’s longtermism is not merely an academic position.
It is the philosophical foundation of a cluster of movements and organizations
that have come to exert significant influence over technology policy and philanthropic giving.
Émile Torres, a philosopher who helped develop these ideas before becoming one of their sharpest critics,
named the cluster TESCREAL:
Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism, and Longtermism.
The ideologies in this cluster share the longtermist premise
that the scale of the future is so vast
that almost any present sacrifice can be justified if it plausibly improves long-run outcomes.
The utilitarian arithmetic is straightforward.
If trillions of people could live across star systems over millions of years,
then even heavily discounted future welfare vastly exceeds present welfare.
A donation that slightly reduces the probability of human extinction
is, on this accounting, worth far more than a donation that cures a child of malaria.
The future people win the calculation.
They always win, because there are so many more of them.
The practical conclusions drawn from this reasoning
were not to give to the most urgent present needs
but to fund AI safety research, pandemic preparedness for civilization-ending pathogens,
and other efforts aimed at preventing extinction or steering the long-run trajectory of civilization.
This is not irrational given the premise.
The problem is that the premise—
that we can make meaningful probability estimates about civilization-scale outcomes
centuries or millennia into the future—
is not grounded in anything.
It is speculation wearing the mathematical clothing of rigor.
The deeper problem is structural.
If the future is so important that it swamps present obligations,
then whoever controls the trajectory of the future
has more moral authority than any present institution or democratic process.
This is exactly the reasoning several tech billionaires have adopted.
Elon Musk has described his accumulation of capital, media platforms, and satellite networks
as necessary to prevent humanity from being locked into a bad long-run future.
OpenAI’s founding mission statement—to develop AI safely “for the benefit of all humanity”—
is a longtermist framing that positions the organization’s leadership
as trustees of civilization accountable to no present constituency.
The argument that today’s people must accept disruption, displacement, and concentrated power
because future people’s welfare is at stake
is being made by very wealthy people
about costs borne by others who have no voice in the decision
[Andreessen2023].
This is the same rhetorical structure as other forms of extraction:
the resources and labor of present people converted into benefits
claimed to accrue to a future whose composition and preferences cannot be known,
managed by self-appointed trustees who cannot be held accountable
because their beneficiaries do not yet exist.
The people who will supposedly benefit have no way to confirm or contest the claim.
The people bearing the costs can be told that their objections fail to grasp the scale of what is at stake.
The philosophical validity of longtermism as a position about what we owe the future
is entirely separable from its political use as a reason
why present inequality, present suffering, and present concentrations of power
should not be interrupted.
An argument that future generations’ interests justify present sacrifice
is most credible when the people making the argument are also bearing the sacrifice.
It is least credible when it is made by the already-powerful
to justify policies that harm people who were not consulted
and will not share in whatever future is being optimized for.
Isaiah Berlin spent much of his career analyzing how this kind of reasoning goes wrong
at the level of philosophy rather than just politics.
His book The Crooked Timber of Humanity took its title from Kant’s observation that
“out of the crooked timber of humanity, no straight thing was ever made” [Berlin1991].
In it,
Berlin argued that the goods people pursue are genuinely plural and genuinely irreconcilable.
Liberty and equality pull against each other.
Security and freedom pull against each other.
Community and individuality pull against each other.
These are not failures of imagination or coordination problems awaiting a technical fix.
They are permanent features of human life,
and no political arrangement will eliminate them.
The view Berlin was arguing against he called monism:
the belief that freedom,
utility,
the glory of the nation,
the arc of history,
expected future welfare,
or something else is the one supreme value,
and that apparent conflicts between goods are ultimately solvable by reference to it.
Monism is not just an abstract error.
It is, Berlin argued, the philosophical root of political tyranny,
not because monists are necessarily cruel
but because once you accept that one value is supreme,
you can justify any present harm.
If the supreme value is large enough—and
nothing is larger than the welfare of all future humanity—the calculation
can authorize anything done to anyone living now.
The people bearing the cost can always be told that their objections
reflect a failure to grasp the scale of what is at stake.
Liberal democracy, on Berlin’s account, is not a system for finding the right answer.
It is a system specifically designed for a world in which
there is no right answer—in which the goods people legitimately pursue
are inevitably in tension
and the best that can be achieved is a negotiated, revisable, accountable balance among them.
What TESCREAL bypasses is not merely democratic process as a procedural nicety.
It is the entire institutional architecture designed to prevent any single value,
and any single group of people claiming to serve that value,
from overriding the rest.
The Material Basis of Democratic Accountability
The philosophical argument that longtermism justifies concentrating power
has a concrete political parallel that requires no philosophy to understand.
Democratic governance rests on a bargain.
The governed have things that their governors need: labor, taxes, and consumer spending.
This dependency is the material basis of democratic leverage.
When value is generated by AI systems owned by a handful of corporations
already expert at tax optimization,
every mechanism of democratic governance weakens simultaneously.
The tax base shrinks,
consumer spending contracts,
and collective bargaining fades away.
The tendency for the return on capital to exceed economic growth accelerates
because AI severs the last link between capital accumulation and the need for human labor;
as that happens,
wealth becomes concentrated in fewer and fewer hands [Piketty2017].
The economic incentives point toward entities with the fewest democratic accountability mechanisms.
An authoritarian government that deploys AI to replace its workforce faces no electoral consequences
and gains a surveillance and control dividend on top of the economic efficiencies.
Governments with vast capital, centralized decision-making, and no electorate to answer to
are structurally better customers for this technology than democracies.
This is one concrete reason the political movement around AI acceleration
has shifted support toward figures who have demonstrated
no loyalty to democratic constraints [McGrann2026].
Dario Amodei, the CEO of Anthropic, said it directly:
“The balance of power of democracy is premised on the average person having leverage
through creating economic value.
If that’s not present, I think things become kind of scary.”
The CEO of one of the leading AI companies is describing the technology he is building
as a threat to the material basis of democratic governance.
His company has not endorsed a single piece of legislation to address it.
What We Can Defend
Years ago,
while hiking in the woods,
I spotted a set of bamboo windchimes hanging from a tree.
There were five weather-stained rods carved to different lengths,
each of which had initials carved on it and a pair of dates.
I can’t remember the years exactly—this was back in the days of chemical cameras,
so if I still have the photo I took it’s in a box in my basement—but
I remember they ran from the 1930s and 1940s to the 1980s.
The bar the chimes hung from had a hole carved in it for a sixth.
I still sometimes wonder why it was missing.
Had a squirrel chewed through the twine used to tie it on,
so that it fell into the undergrowth and was lost?
Had someone’s time not come yet?
Or had there been no one left who knew or cared to add it?
After all I’ve read,
I believe that caring about the future is a personal choice,
and that the most defensible version of our obligation to it is simple:
don’t foreclose options.
I think we should strive to give our grandchildren,
and their grandchildren,
as many choices as we had.
They should be able to enjoy the things that we can enjoy,
like safe drinking water, healthy coral reefs, and free elections.
I believe that we don’t own those things.
We are just their trustees,
and like all trustees,
we are accountable for what we hand on—even if the beneficiaries aren’t in the room.
These posts started as a series titled “Big Tech is Like…”.
An earlier post in this series included eight of those;
here are a few more.
Big Tech is Like a Company Town
E.P. Thompson’s account of the transition from putting-out to factory production
made a point that factories were initially not more efficient at producing cloth:
they were more effective at eliminating worker autonomy [Thompson1963].
Before industrialization,
workers controlled their own time and pace,
could share work across household members,
and could resist disadvantageous terms through slow work, variable quality, and informal coordination.
The factory closed off these responses:
workers arrived at fixed times,
worked at a supervised pace,
and until labor unions emerged,
had no practical means of collective withdrawal.
The shift to app-based platform work is doing the same thing today.
The app is not merely a more convenient interface.
It is supervisory infrastructure that monitors pace and completion rates
and eliminates the degrees of freedom that less tightly managed arrangement permitted.
Grab and Gojek motorcycle drivers in Jakarta,
who own their vehicles but are subject to algorithmically set rates they cannot negotiate
and risk deactivation for declining rides,
have organized repeated strikes—the precise form of collective resistance
available to workers who cannot withdraw their invested capital.
George Pullman built Pullman, Illinois in the 1880s alongside his railroad car factory:
a planned community where the company owned the workers’ housing, stores, church, library, and bank.
Workers were paid partly in scrip redeemable only at company stores.
When Pullman cut wages during the depression of 1893 while holding rents fixed,
workers could not cushion the blow by cutting other expenses,
because the company controlled those too.
They had no recourse and nothing left to lose.
The 1894 Pullman Strike paralyzed rail traffic across the country
and required federal troops to suppress,
which tells you something about what total control eventually produces.
A developer whose business depends entirely on the App Store,
a seller whose inventory and customer relationships live on Amazon,
or a creator whose audience exists only within a single platform
is in an equivalent position:
the terms can be changed at any time,
the cost of departure is prohibitive,
and the entity that provides the housing also adjudicates disputes about it.
What the company town reveals is something market economics tends to obscure:
the difference between an employer and an infrastructure provider is a matter of degree,
not kind.
An employer controls your income;
an infrastructure provider controls the conditions
under which you can earn, spend, and participate in social life.
Pullman did not merely employ his workers:
he owned the physical environment in which they lived,
which meant that the power relationship extended past the end of the workday
into every dimension of daily existence.
Pullman justified all of this in the language of paternalistic improvement [Lindsey1942].
The company town was presented as a planned community
designed to give workers superior housing, sanitation, and amenities
compared to what they could obtain elsewhere.
This was partly true:
Pullman, Illinois was better built than most industrial housing of the period.
But the justification also framed the absence of collective bargaining as a feature rather than a constraint.
Workers who were being improved did not need unions.
They needed to trust the judgment of the person improving them.
Conveniently,
this gave the appearance of converting a power arrangement into a benevolent one.
The Pullman case was not unusual.
Coal patches in Appalachia and the Canadian Maritimes operated on the same model:
company-owned housing,
company stores where scrip could be spent,
company control of access roads and community infrastructure.
In the post-Civil War South, plantation stores extended dependency relationships
across the formal end of slavery,
tying sharecroppers to specific land through debt
that was nearly impossible to escape
without leaving behind the credit, relationships, and community standing
that they depended on.
Mining camps across Latin America reproduced the same structure under different flags.
The company town is a recurring organizational form
because it solves a genuine problem for whoever controls it:
it makes exit nearly impossible while appearing to be voluntary.
Foxconn’s manufacturing campuses in Shenzhen house hundreds of thousands of workers in company dormitories,
sell goods through company stores, and provide company-managed transportation—
the Pullman model operating at a scale Pullman could not have imagined,
producing the consumer electronics sold by companies that will not acknowledge the supply chain.
The distinction between short-term dependency and structural lock-in
is what keeps workers and developers in arrangements that are getting worse.
Short-term dependency is a rational calculation:
you stay because the current terms are good enough and leaving has costs.
Structural lock-in is different.
The costs of leaving have been deliberately raised
so that the calculation does not change even as the terms worsen.
A platform can worsen its terms incrementally;
each incremental worsening is individually insufficient to justify the exit cost,
even if the cumulative total is large.
A more extreme form of this appears in the danwei,
the work unit system through which Mao-era China organized urban life.
The danwei was the unit through which citizens accessed employment,
housing, healthcare, food rations, and education for their children.
To be removed from a danwei was to lose access to all of these simultaneously.
The contemporary parallel is super-apps like WeChat
that integrate messaging, payments, healthcare bookings,
government service access, merchant discovery, and social identity in a single system.
A user whose account is suspended loses not just a communication tool
but the infrastructure through which they conduct their civic and commercial life.
The Pullman Company controlled the economic conditions of workers’ lives.
The super-app, at its most developed, aims to be a danwei [Stoller2019,Walder2017].
Big Tech is Like the Beauty Industry
The beauty industry does not merely sell products.
It works by making people believe they have flaws they were not previously considered flaws,
by creating status competition around those flaws,
by ensuring that the standards people are supposed to meet remain unstable,
and then sells temporary relief from the insecurity it has helped create.
The customer is never meant to arrive,
because that would end the business model.
The same logic governs much of the digital economy.
Platforms do not simply sell access to information, communication, or entertainment.
They produce social comparison at industrial scale,
amplify the feeling of falling behind or being left out,
and then sell ads, subscriptions, and self-optimization tools as the remedy.
Thorstein Veblen identified all of this,
even though he was writing in 1899 rather than in the age of Instagram.
The point of status goods is not that they are useful or pleasurable.
It is that they are visible evidence of social rank.
The phrase “keeping up with the Joneses”,
which entered American English in the early twentieth century,
names the same dynamic more plainly.
Consumption becomes competitive because social standing is comparative,
which means there is never “enough”.
This competition is profitable because
the goalposts move as soon as people approach them [Veblen1899].
The beauty industry is a particularly efficient machine
for converting status anxiety into revenue
because it treats the human body as a perpetually unfinished project.
Skin can always be smoother,
hair shinier,
teeth whiter,
age less visible,
and weight lower.
What matters is not whether any particular intervention works as advertised.
What matters is the manufactured belief that our peers are taking action,
so we must too.
This is why beauty advertising oscillates between aspiration and warning:
you want to be admired,
but you fear that you will also be judged.
Women have always borne the brunt of this manufactured insecurity.
The modern beauty industry developed alongside labor markets and marriage markets
in which women’s economic security and social standing were tied more closely to appearance
than men’s.
As legal barriers to women’s advancement weakened,
appearance standards became a more intense disciplinary mechanism,
not a less important one.
A market built around telling women that their bodies require continuous corrective spending
fits very comfortably inside a misogynist social order [Peiss2011].
South Korea has the highest cosmetic surgery rate per capita in the world,
produced by a feedback loop among K-pop image norms, social media filters,
cosmetic surgery clinics, and legally tolerated workplace appearance discrimination
that has exported its manufactured insecurity to dozens of countries along with its skincare products.
Big tech has given this machinery telemetry, automation, and scale.
The platform feed is a Jones machine.
It puts other people’s vacations, kitchens, bodies, weddings, and workout routines in front of us
not as occasional local gossip but as a continuous global stream.
The result is not just envy:
it is the normalization of comparison as a default mental state.
Am I fit enough, rich enough, busy enough, stylish enough, politically
informed enough, raising my children well enough, or aging well enough?
A system that can keep those questions humming in the background can monetize them indefinitely.
The comparison does not have to be truthful to be effective.
Beauty advertising has always relied on lighting, retouching, and selective casting.
Social platforms inherit all of that and add filters,
algorithmic selection,
and engagement optimization.
The most attention-grabbing images are not the most representative ones.
They are the ones most likely to intensify feeling s admiration, desire, resentment, or shame.
The platform does not need users to believe that everyone else is happier, younger, or richer than us.
It only needs users to feel,
for a few seconds at a time and many times a day,
that they are falling behind.
Influencer culture is itself manufactured.
The beauty industry has long depended on the blurring of intimacy and commerce.
Advice from a beautician and endorsements by actresses all worked because
they borrowed the authority of friendship or expertise while remaining commercial speech.
Influencers do the same thing with better metrics.
They present consumption as personality and sponsorship as authenticity.
This is also where misogyny returns in a more explicit form.
Women are subjected to the harshest forms of visual ranking online,
and platforms profit from that ranking
regardless of what they say in public about empowerment or self-expression.
Image-heavy systems reliably reward content that conforms to existing beauty norms
while presenting itself as spontaneous self-presentation.
The labor required to produce that appearance is substantial and usually hidden.
The result is an economy in which women are pushed to become both product and salesperson,
while the platform captures a share of every transaction.
The beauty industry also helps explain why the language of choice is inadequate.
No one is forced to buy a serum or post a filtered selfie,
but choices made in response to organized social pressure are not free
in any meaningful social sense.
A teenager deciding whether to participate in appearance-based competition on a platform
is making an individual decision
inside a system designed to make non-participation costly,
just like a programmer deciding to maintain a LinkedIn presence
full of visible hustle and polished enthusiasm.
The compulsion is social before it is economic,
and economic because it is social.
What the beauty analogy adds to our understanding of big tech
is not just that platforms sell ads—every industry does that.
It is that platforms are in the business of manufacturing dissatisfaction
and organizing it into recurring revenue.
Their entire business model is that you are not quite enough yet,
but perhaps one more purchase will fix that.
A business model built on that premise
has no interest in ever letting us feel finished [Packard2007,Wu2016].
Big Tech is Like the Sharecropping System
After the American Civil War,
former slaves and poor white farmers in the South farmed land they did not own
under a system called sharecropping.
Contracts that gave the landowner a percentage of the harvest,
required them to buy supplies on credit from the landowner’s store at prices the landowner set,
and prohibited them from selling to anyone other than the landowner.
The debt was structured so that a bad harvest
(or even a good one, depending on how the accounts were kept)
left the farmer owing more at year’s end than at the beginning.
The system was legal,
entered into “voluntarily” (by people with no other options),
and reproduced the economic relations of slavery without the formal institution.
Historians and economists have found systematic underweighting of harvests and overcharging of credit accounts.
The system was designed to perpetuate indebtedness, not resolve it.
To make a bad situation worse,
the merchant providing supplies was often the same person as the landowner;
if not,
they usually operated as a tied supplier.
The farmer could not buy supplies from a competing merchant because
the crop lien pledged the entire harvest to the furnishing merchant
as collateral for the advance.
“On credit” prices at the furnishing merchant’s store were set at a markup over cash prices
that could run from thirty to sixty percent.
By pledging next year’s crop to cover this year’s debt,
the farmer legally committed future labor before that labor was performed.
This is the structure of platform lock-in:
time and effort are committed to a platform
before the platform’s terms are known for the following year.
A sharecropper who improved the land was more trapped,
not less,
because the accumulated investment had nowhere else to go.
The same is true of a creator who has built a hundred thousand subscribers
on a platform that then changes its revenue-sharing terms.
Peonage was the name for debt arrangements that crossed from exploitative into criminal.
It refers to holding a worker in involuntary servitude through debt
while using threats or actual violence to prevent them from leaving.
Federal peonage statutes were enacted after Reconstruction,
and some prosecutions did occur in the early twentieth century
after investigative journalism and advocacy exposed conditions
in the turpentine camps and cotton plantations of the Deep South.
These prosecutions targeted individual employers;
they did not address the systemic accounting fraud or the market structure
that produced mass indebtedness [Daniel1972,Blackmon2008].
The platform equivalent of this operates through percentage fees,
arbitrary algorithmic changes,
and paid promotion requirements.
When YouTube changed its monetization criteria in 2018
to require one thousand subscribers and four thousand watch hours
before a channel could earn advertising revenue,
channels that had not yet crossed those thresholds were cut off from income they had been building toward.
When organic reach on Facebook declined sharply for business pages after 2012,
businesses that had invested in building Facebook audiences were told
they could pay for promotion to reach the audiences they had already acquired.
The audience is the crop;
the algorithm change is the landlord raising the rent after the harvest is in.
A related but distinct mechanism appears in gig economy work,
which parallels the putting-out system of early modern textile production.
Merchants in eighteenth and nineteenth century Britain supplied raw materials to rural householders,
who processed them at home using their own equipment.
The merchants then collected the finished goods at prices they set.
The household owned its tools;
the merchant owned the raw material, the finished product, and the customer relationship.
If demand fell,
the merchant stopped delivering material and the household had to absorb the income shock.
The merchant’s flexibility was the household’s precarity, by design [Devries2008].
Gig economy platforms reproduce this structure.
The delivery worker owns the car or bicycle.
The platform owns the customer relationship, the pricing mechanism, and access to the market.
A worker who attempts to find customers outside the platform risks deactivation,
which functions as accusations of theft or embezzlement did in the putting-out system:
because formal ownership of the customer relationship belongs to the platform,
any attempt to access that relationship independently constitutes
a violation of the terms of service that governs continued access to work.
Big Tech is Like a Fast Food Franchise
A McDonald’s franchisee invests several hundred thousand dollars building and equipping a restaurant,
recruits and trains staff,
manages daily operations,
and absorbs the risk of a bad location.
McDonald’s sets the menu,
the supplier list,
the pricing guidelines,
the design specifications,
the training requirements,
and the standards against which the franchise can be audited and revoked.
The franchisee’s capital is at risk;
McDonald’s corporate’s is not.
McDonald’s also, in most arrangements, owns the real estate and charges the franchisee rent.
The arrangement is voluntary in the sense that no one forces the franchisee to sign.
It is asymmetric in the sense that every significant decision rests with one party.
The franchisee agreed to all of this in advance.
McDonald’s corporate revenue derives primarily from real estate, not from food.
The company acquires the land and building,
then leases them to the franchisee
at rates that capture a substantial share of the location’s economic value
regardless of operating performance.
Franchise royalties are calculated as a percentage of gross sales,
not profit,
meaning the franchisor collects whether the franchisee makes money or not.
The hamburgers are the mechanism by which the real estate and royalty revenue are generated.
This structure is not incidental to the model;
it is the model.
The parallel to third-party sellers on Amazon,
app developers on iOS and Android,
and content creators on social media is structural:
the platform sets the rules,
extracts a percentage,
and can change the terms or terminate the relationship on short notice [Schlosser2001].
Platform fees work like franchise royalties:
they are a take on gross transaction value,
not on seller profit.
Apple’s standard rate is thirty percent of revenue from in-app purchases,
a figure that it set unilaterally
and has adjusted modestly only under regulatory pressure.
The developer has no alternative distribution channel for iOS users,
because Apple prohibits sideloading and third-party app stores on its platform.
This makes the fee unavoidable for any developer who wants access to iOS customers.
What’s more,
Apple and Google review apps before they are listed,
can remove them after listing,
and adjudicate appeals internally.
There is no neutral third party with authority to override a platform’s decision.
All this creates an investment trap for platform participants.
A seller who has accumulated reviews, rankings, and sales history on Amazon
cannot transfer that reputation to another marketplace;
a developer whose app has a rating history on the App Store
cannot move that history to Google Play.
This is not an accident:
it is the mechanism that keeps participants in the system
after the platform has extracted the value of introducing customers to them.
Antitrust law has historically been reluctant to treat voluntarily agreed contractual terms as coercive,
even when the practical alternatives to agreement are limited.
The prevailing view is that if the franchisee had an alternative and chose this arrangement,
the arrangement is presumptively legitimate.
This framing does not account for information asymmetry at the time of contracting
or the way switching costs increase over time.
And it does not account for the fact that the “voluntary” choice
is often a choice between accepting one set of platform terms
or not accessing a market at all [Stoller2019].
The UK Supreme Court ruled in 2021
that Uber drivers were workers entitled to minimum wage and holiday pay,
not independent contractors.
Thsi was the first major judicial rejection of the claim that
a platform’s “voluntary” contractual structure exempts it from employment law,
now being contested and replicated across Europe.
Big Tech is Like Scientology
In the 1950s, L. Ron Hubbard developed a practice called auditing.
In a standard session,
a trained Scientology auditor asks the subject a series of questions
while the subject holds the electrodes of an E-meter that measures galvanic skin response
(the same physiological signal used in polygraph tests).
The questions are designed to surface traumatic memories,
which Scientology calls engrams,
so they can be discharged through conscious recall.
The sessions are recorded, and the records are kept in what Scientology calls “preclear folders.”
They contain whatever the subject disclosed during auditing:
accounts of illegal activity,
sexual behavior,
family conflicts,
financial difficulties,
and statements about other people.
The Church of Scientology denies that folders are used punitively,
but former members have testified that these folders were used in disciplinary proceedings
and in litigation against critics and defectors.
The analogy to big tech is not subtle.
Every major social media platform is, at its core, an auditing system.
It collects behavioral data—what you look at, what you hesitate over, what you react to—and
that information is qualitatively different from what you share with a retailer.
People post about illness and grief and their political beliefs and sexual identity
because the platform presents itself as a community,
not a database.
The fact that it is both doesn’t mean the user is naïve;
it means the platform is designed to exploit the social context
that makes sharing this information feel appropriate.
Scientology’s critics have documented a practice the Church calls Fair Game,
under which people who leave the organization and speak critically about it
(known as a suppressive person)
can be “deprived of property or injured by any means by any Scientologist
without any discipline of the Scientologist.”
The Church claims this policy was cancelled in 1968,
but its critics have documented its continuation under different names.
The pattern has included litigation designed to exhaust defendants financially,
harassment campaigns targeting employers and family members,
and the use of auditing records in legal proceedings.
Tech companies have not employed anything comparable in severity (that we know of).
They have, however, used legal and institutional power to manage criticism
in ways that Scientologist would recognize.
Facebook commissioned audits of third-party researchers who published findings the company disputed.
Google funded academic research in ways that created conflicts of interest
for academics who might otherwise study the company critically.
Uber deployed a team it internally called COIN
(for Competitive Intelligence)
to gather information on regulators, journalists, and competitors.
The distinction between these practices and Fair Game
seem pretty slim to the researchers, journalists, and regulators on the receiving end.
Scientology’s governing doctrine holds that the organization’s critics are necessarily criminals.
If someone attacks Scientology,
Hubbard’s writings state,
one need only look at their past to find the crimes they are hiding.
The logic is airtight because it is circular:
criticism itself is taken as evidence of wrongdoing.
This is a specific and pathological version of a general tendency.
When researchers publish findings critical of Facebook’s recommendation algorithms,
Facebook’s communications team responds not only with factual rebuttals
but with questions about the researchers’ methodology, funding sources, and motivations.
When journalists publish stories based on leaked documents,
companies issue statements about documents being “taken out of context”
and about reporters’ prior relationships with the company.
In 1993,
the Church of Scientology achieved recognition from the US Internal Revenue Service
as a tax-exempt religious organization,
ending 25 years of litigation.
Its strategy included filing thousands of personal lawsuits against IRS employees,
hiring private investigators to gather personal information on IRS staff,
and conducting what the IRS’s own documents describe as a covert intelligence operation against the agency.
Google, Meta, and Amazon have not run intelligence operations against their regulators
(that we know of).
They have collectively spent over $100 million per year on lobbying in the United States alone,
employed virtually every major lobbying firm in Washington,
and placed former executives in regulatory positions
in a sustained campaign to shape the rules governing them.
Scientology is structured so that participation becomes progressively more expensive.
New members begin with free or low-cost introductory materials.
Progression up the “Bridge to Total Freedom” requires increasingly expensive courses and auditing sessions.
Former members have documented spending hundreds of thousands of dollars over years of participation.
The social world of Scientology reinforces continued involvement:
friends, family, and community ties are largely internal to the organization,
which means that leaving means losing them.
The structural lock-in that platforms engineer follows the same logic.
A photographer who has spent years building an audience on Instagram
is not free to leave without abandoning what they have built.
A developer who has built a business on the iOS App Store faces the same kind of switching cost.
The Church of Scientology has survived decades of hostile press,
regulatory action across multiple continents,
and prominent defections.
It has done so by treating litigation as a cost of doing business,
and by providing genuine community to members.
The question now is whether the mechanisms that have gradually constrained Scientology
will operate at the scale of companies whose products are used by billions of people
who have no obvious alternative [Wright2013].
Big Tech is Like a Ransom Business
The Canvas learning management system was hacked a couple of days ago,
so this seems like a good time to point out that extortion,
if it’s professional enough,
is indistinguishable from any other fee-for-service arrangement.
The victim pays for the return of something that was theirs,
the captor provides a guarantee of safety,
intermediaries take a cut,
and everyone has an interest in the transaction completing cleanly.
In 1994,
when the FARC guerrilla organization in Colombia was near the height of its power,
kidnapping was a line item in its budget.
The organization maintained specialized units for identifying targets,
executing abductions,
holding captives in jungle camps,
and conducting negotiations.
Insurance companies led by Lloyd’s of London
responded by creating kidnap-and-ransom (K&R) policies for multinational corporations,
and specialist firms like Control Risks Group built a business on negotiating with kidnappers.
By the late 1990s,
an abduction in Colombia, Venezuela, or the Philippines was like buying a house:
the kidnapper demanded a high figure,
the negotiator offered a low one,
and after weeks or months of back-and-forth they agreed on something in the middle
and settled up in cash.
Both sides had an interest making this running smoothly;
in particular,
kidnappers who killed hostages damaged their own reputations with future potential clients.
Researchers studying the “industry” found that
K&R specialists worked hard to prevent ransom inflation:
they trained negotiators to push back,
kept payment records confidential,
and advised clients not to advertise their coverage,
because a public policy was an advertisement for kidnapping your staff [Shortland2019].
The rise of ransomware attacks over the last decade has followed the same path.
The 2017 WannaCry attack encrypted hundreds of thousands of computers across 150 countries in a single weekend,
demanding Bitcoin payments in exchange for decryption keys;
the attack was later blamed on North Korean state actors.
Four years later, the DarkSide ransomware group (probably based in Russia)
shut down the Colonial Pipeline in the United States and demanded approximately $4.4 million in Bitcoin.
The company paid within hours.
Modern ransomware groups operate on an affiliate model:
the core developers write the malware and maintain the payment infrastructure,
while affiliates handle the actual intrusions.
On the other side of the table,
cybersecurity firms handle the details just like Control Risks Group did,
and cyber insurance policies now cover ransom payments,
which means that insurance companies are wrestling with the same concerns
about moral hazard and ransom inflation
that Lloyd’s was worrying about in the 1990s.
When Colonial Pipeline paid DarkSide,
they almost certainly broke US Treasury rules prohibiting payments to sanctioned entities.
Governments have been consistently inconsistent in their positions on this:
they urge companies not to pay while acknowledging privately that there is no realistic alternative.
This is the same ambivalence that surrounded K&R payments in the 1980s,
when Western governments officially discouraged negotiating with kidnappers
while intelligence services routinely assisted with exactly that [Dudley2022].
Bueno de Mesquita and Smith’s The Dictator’s Handbook
was one of the inspirations for this series of posts.
Their model of how people get and hold power isn’t as cynical as it first appears,
and it helped me make sense of
some of the company politics I’ve endured (and been part of) over the years.
Reading outward from that lucky find led to the post below.
We’re All Family Here
In November 2022,
after laying off about a third of its original workforce,
Elon Musk sent an email to the remaining Twitter employees
asking them to click a button to confirm that they were committed to working “hardcore”
for the company’s next phase.
Those who did not click by the deadline would be treated as having resigned.
A few months earlier,
Twitter’s former leadership had described it as a family.
The family metaphor is endemic to tech.
Amazon has “Day 1 culture”,
while Google offered free meals, nap pods, and climbing walls,
and expected employees to treat the campus as home.
The metaphor does real work:
it extracts commitment,
discourages outside offers,
and makes employees (particularly younger ones)
feel that the relationship is something other than a transaction.
What it does not do is change what the relationship actually is.
Families (at least, those outside organized crime) do not terminate members for underperformance.
They do not eliminate positions when margins tighten,
or ask you to sign a noncompete agreement before letting you in.
In most of the United States, and in varying degrees elsewhere,
employment is at-will:
either party can end it, at any time, for any reason not specifically prohibited by law.
This means that the “family” exists at the employer’s pleasure.
The political scientist Harold Lasswell defined politics in 1936
as the study of “who gets what, when, how.”
His definition contains no implication that the getting is fair,
no assumption that what gets distributed is material,
and no requirement that the process be democratic.
It is simply a description of how groups make binding decisions
about the allocation of things people want.
Politics is what happens when a group of people
who do not fully agree on goals or values
nonetheless need to act together.
The alternative to politics is not harmony—it is coercion.
Isaiah Berlin argued that this disagreement is not a temporary condition
awaiting the right institutional fix [Berlin1991].
The goods people pursue—liberty and equality, security and freedom, efficiency and fairness—are
genuinely plural and irreconcilable:
you cannot maximize all of them simultaneously,
and no rearrangement will make the conflict disappear.
Liberal democracy is not a system for finding the right answer.
It is a system for managing the permanent tension between right answers that contradict each other.
A group that appears to have no politics is usually one
in which someone has already won so decisively
that further fighting seems pointless.
Both definitions apply to workplaces.
An organization contains people
who disagree about what matters, what to build, who to hire, where to cut, and who should lead.
Those disagreements do not disappear because the employee handbook calls everyone a family.
They get resolved through decisions that favor some people’s views and interests over others.
That process is workplace politics.
Bueno de Mesquita and Smith developed a framework called selectorate theory
to explain why leaders behave the way they do [BuenodeMesquita2011].
The core observation is simple:
leaders of countries, companies, and volunteer organizations need enough support to stay in power.
They get that support by distributing benefits
to a minimum necessary winning coalition.
The winning coalition is not the whole organization:
it is the subset of people whose support the leader actually requires.
In an autocracy, this might be the military brass, a security service, and a handful of oligarchs.
In a publicly traded company, it is the board, major institutional shareholders,
and a small number of indispensable senior executives.
Everyone else—the people who are told they are family,
and that the company’s success is their success—is interchangeable.
They are what the theory calls the selectorate:
large enough to give the winning coalition options if any member defects,
but not powerful enough to claim a significant share of private benefits.
This is why perks, mission language, and family rhetoric are so common
in organizations that also behave ruthlessly when conditions change.
The perks are cheap ways to signal belonging to people
who are not actually in the winning coalition.
The rhetoric costs nothing
but extracts real commitment.
When the company faces a genuine crisis, the winning coalition keeps their jobs.
The family discovers it was not, in fact, the family.
This is where a common misreading needs correction.
The people who use family rhetoric are not,
for the most part,
cynical manipulators who despise their employees.
Nor are they altruists who genuinely believe the metaphor
and are simply wrong about how the world works.
Most are somewhere in between:
people who have genuine beliefs about what the organization should do,
who also benefit when those beliefs prevail.
Jeffrey Pfeffer, in decades of research on organizations,
found that political skills
like the ability to build coalitions,
read organizational dynamics,
and time moves correctly
are stronger predictors of career advancement than technical competence [Pfeffer1992].
The engineer who wants to rewrite the legacy codebase genuinely believes it needs rewriting,
and also gets promoted if the project goes ahead under her leadership.
The VP who champions a reorganization genuinely thinks it will improve outcomes,
and also ends up at the top of the new structure.
Interests and beliefs are not opposites.
People pursue what they think is right,
and what they think is right is shaped by their position in the organization.
A sales leader who believes the product team should prioritize enterprise features is not lying.
She is telling the truth as experienced from where she sits.
She also reaps the benefits if her plan is adopted.
The family metaphor is not unique to American tech companies.
Japan’s postwar lifetime employment system
produced what selectorate theory would describe as
a very large nominal selectorate of permanent employees
with a winning coalition drawn from senior management and major shareholders.
When the asset bubble collapsed in 1990 and corporations needed to cut costs,
the permanent employment guarantee at the heart of the “family” bargain was broken
through expansion of temporary and contract employment
that covered the same work under worse conditions.
The family language remained;
the security it implied did not.
The phrase “we don’t do politics here” appears regularly in tech companies.
It usually means one of two things.
The first is a genuine belief that technical decisions should be made on technical merit,
that interpersonal dynamics should not determine outcomes,
and that coalition-building is a form of corruption.
This belief is reasonable,
but almost entirely wrong about how decisions actually get made.
The second meaning is that
when the people in the winning coalition say “we don’t do politics here”,
what they usually mean is that
they have already gotten what they want from the current structure,
so there is no need for them to engage in visible political activity.
People who are well-served by existing arrangements
can afford to describe those arrangements as natural
and political contestation as illegitimate.
Basecamp, the project management software company,
made headlines in 2021 when its founders banned “societal and political discussions”
on internal company channels.
They framed this as keeping the workplace professional and focused.
Roughly a third of the company’s employees resigned within days,
recognizing that the decision to ban discussion of politics was itself a political decision,
made unilaterally by the winning coalition.
Understanding that your workplace is a political environment
is not the same as deciding to become a political operator.
It does not require manipulation or coalition-building for its own sake.
What it does require is honesty about what is actually happening when decisions get made.
Someone who believes their technical approach is correct and advocates for it strongly,
who seeks allies among colleagues with aligned interests,
and who times their proposal for when decision-makers are receptive
is not doing something shameful.
They are participating in the ordinary process
by which organizations make decisions in the absence of shared goals.
The person who refuses to do any of this and then wonders why their ideas never get adopted
is not taking the moral high road.
They are making a practical error while feeling virtuous about it.
Your organization is political.
The question is not whether to participate in its politics.
The question is whether to participate consciously and honestly or not
[Crick2000,Runciman2014].
The Hidden Ledger
The political decisions that shape organizations do not operate in a vacuum.
They interact with social categories that were themselves created through prior political decisions—
categories that shape who finds it easy to enter a winning coalition
and who remains part of the interchangeable selectorate.
In 1950,
South Africa’s apartheid government created the Race Classification Board
to assign legal racial categories to people whose status was ambiguous.
The examiners used what they called the pencil test:
if a pencil inserted into someone’s hair stayed in place without falling,
the person might be classified as “Coloured” rather than “White.”
They listened to accents,
examined fingernails for pigmentation,
and interviewed neighbors.
Families were split:
siblings were classified into different categories
because they had inherited different combinations of features,
giving them different legal rights and permitted occupations,
and requiring them to live in different neighborhoods.
The pencil test is an extreme example of
something that operates wherever racial classification exists:
a social and political determination dressed as a natural fact.
Race is not a biological category.
This is not a political opinion—it is the settled position of geneticists,
who have found more variation within conventionally defined racial groups than between them.
Race has been defined and redefined by specific people for specific political reasons.
The US Census Bureau’s changing list of racial categories is a useful illustration.
In 1930,
Mexicans were for the first time classified as a separate racial category rather than white.
A decade later they were reclassified as white again
following diplomatic pressure from the Mexican government.
The category “Hispanic” does not appear in census data before 1970;
it was created by the Nixon administration to aggregate Spanish-speaking populations for federal programs.
The sociologists Michael Omi and Howard Winant called this process
racial formation.
Irish immigrants in the 1840s appeared in popular cartoons as racially distinct from Anglo-Saxons,
while Italian immigrants in the early twentieth century were subject to legal discrimination
partly justified on racial grounds.
Each of these groups eventually became “white” through political processes
that included them while excluding Black Americans [Oluo2018,Omi2015].
Understanding how racial categories are built, and by whom, is necessary for understanding
how algorithmic systems operate in societies organized around race and caste.
A system cannot be race-neutral or caste-neutral if it is trained on data generated
by institutions that were not.
A hiring model trained on historical promotion decisions learns to prefer candidates
who resemble the people who were previously promoted,
in organizations that excluded people by race and class.
A recidivism prediction tool trained on arrest data learns patterns from policing decisions
that were themselves racially disparate [Kendi2016,Wilkerson2020].
Algorithmic hiring systems,
insurance scoring models,
and predictive policing tools share two structural features
that determine their impact:
they collect data without meaningful consent,
and the subjects cannot see or challenge the decisions made about them.
Credit bureaus are the most mature example of this model,
and the clearest preview of where the others are heading.
The three major credit bureaus collect financial data about hundreds of millions of people
without their active participation or consent.
They do not collect data from the people whose files they maintain.
They collect it from creditors like banks, credit card issuers, and debt collectors,
who report account status and payment history.
The person whose data is being collected is not notified when a new entry appears in their file,
has no opportunity to contest it before it is recorded,
and may not know it exists until they apply for credit and are denied.
The bureau’s relationship is with the furnisher and the purchaser of the data,
not with the subject.
Credit data determines whether people can rent housing or get a job.
The companies profit whether the data they hold is accurate or not.
A person whose file contains an error bears the full cost of that error while the bureau bears none.
In the US, the Fair Credit Reporting Act gives consumers some rights to dispute errors,
but the dispute process is designed and operated by the bureaus themselves,
and has repeatedly been found to be inadequate.
Accuracy and fairness are not the same thing, and the credit bureau model conflates them.
A credit score can be perfectly accurate as a summary of past borrowing behavior
and still encode the effects of decades of discriminatory lending.
Neighborhoods that were redlined in the twentieth century
still show lower average credit scores today:
not because their residents are less creditworthy,
but because they were systematically excluded from the wealth-building mechanisms that credit history reflects.
A score that accurately summarizes a history of exclusion still perpetuates exclusion.
In the 2017 Equifax breach,
the personal financial information of approximately 147 million people was exposed,
including Social Security numbers, birth dates, and home addresses.
Equifax’s response was to offer credit monitoring services—sold by Equifax.
The settlement reached in 2019 provided most affected consumers with a few dollars apiece,
after the claims fund was overwhelmed by the number of applicants.
No one at Equifax faced criminal charges.
The parallel to large technology platforms is not a stretch.
Google, Meta, and others collect data about billions of people without meaningful consent
and sell access to that data to advertisers,
whose relationship with the platform is the commercially important one.
There is no effective dispute process,
and when these systems produce discriminatory outcomes,
the platform does not bear a cost proportional to the harm.
The credit bureau model took decades to produce Equifax.
The tech industry produced the same structure at global scale in about fifteen years,
and called it connecting the world [Chouldechova2017,Oneil2016,Pasquale2015].
China’s Social Credit System bars individuals with low scores
from purchasing plane and train tickets,
enrolling their children in private schools,
and accessing certain financial services.
This is the endpoint of the credit bureau model,
where a score derived from past behavior determines present civic participation.
Why Don’t You Just…
In 2013, the United Kingdom launched Universal Credit,
a welfare reform ostensibly intended to simplify the benefits system
by merging six separate payments into one.
The new system was designed to be applied for online,
but many claimants had no reliable internet access.
Those who did often lacked fixed addresses,
which the system required before it would register them.
Without registration they couldn’t receive payments;
without payments they couldn’t maintain an address,
and without an address they couldn’t register.
Government officials and advice workers,
when presented with this loop,
would sometimes suggest that claimants “just go to the library” to use a computer.
They did not know—or had not thought through the fact—that
some libraries require a membership card to use their computers,
that membership cards require proof of address,
and that the Universal Credit application times out and loses your work
if you do not complete it in a single session.
The word “just” was doing an enormous amount of lifting.
“Just” does that in a lot of conversations.
When someone with relative power is told about a problem they have not personally faced,
a common response is to suggest an individual solution:
“Why don’t you just move to a better neighborhood,
just report the harassment to HR,
just open a bank account,
just apply for a scholarship.”
The suggestion is not usually made in bad faith.
It is made because the person offering it
has, at some point in their life,
moved, or reported, or opened, or applied,
and found the process manageable.
What they cannot see is what made it manageable for them
and what makes it unmanageable for someone else.
Peggy McIntosh described this as an invisible knapsack:
a set of advantages so routine to those who carry them
that they are not experienced as advantages at all [McIntosh1989].
You do not notice that your accent marks you as non-threatening,
that your name gets you callbacks,
that your neighborhood has a library,
or that the official you need to speak to treats you as a legitimate claimant
rather than a probable fraud.
The person offering the “just” solution
is usually describing what they would do,
which is not the same thing as what the other person can do.
The most obvious thing “just” conceals is cascading prerequisites.
Many systems assume that the person using them already has a set of prior resources in place.
The Universal Credit example is one illustration:
online-only access built on a cascade of prior requirements,
each of which depends on the one before it.
A second thing “just” hides is the cost of the transaction.
Taking a day off work to stand in line costs money that people without savings cannot spare.
Challenging a decision by a government agency or a platform company
requires knowing how to challenge it,
having the literacy and the time to fill in forms,
and being willing to risk the relationship with the institution you depend on.
The last point matters more than it might seem:
if the institution denying you a benefit is the same institution
you are hoping will pay your rent next month,
asserting your rights has a price.
A third thing the word hides is cognitive load.
Sendhil Mullainathan and Eldar Shafir spent years studying what poverty does to decision-making,
and their conclusion was not what most people expect [Mullainathan2013].
Poverty makes people worse at decisions in the same way
that keeping someone awake for twenty-four hours makes them worse at decisions:
it depletes a finite resource.
Managing an unpredictable income,
keeping track of which bills are overdue,
or calculating whether buying the cheaper item in bulk saves more than it costs to store
consumes cognitive bandwidth
that is then not available for navigating bureaucratic systems.
The person telling someone to “just” do something
is typically not paying this tax.
The pattern is particularly vicious on tech platforms.
Companies routinely advise users to “just report” harassment,
“just use a VPN” to avoid surveillance,
or “just switch platforms” when a service degrades.
Each suggestion assumes that the user has time, technical literacy, and social capital
that many do not have,
and none of them fix the underlying system.
They transfer the cost of a structural failure onto
the person least able to bear it [Eubanks2018].
Who Gets to Decide
In August 2016,
the European Commission ordered Apple to repay thirteen billion euros in back taxes to Ireland.
The commission had investigated Apple’s tax arrangements in Ireland
and concluded that they amounted to illegal state aid:
Ireland had given Apple a selective advantage unavailable to other companies,
allowing the company to pay an effective tax rate of 0.005% on European profits
of sixteen billion euros in 2014.
The Irish government was ordered to collect the money,
and promptly announced it would appeal the ruling.
Ireland did not want the thirteen billion euros;
its government argued in court,
alongside Apple,
that the commission had made an error.
This only baffled the cynical:
Ireland’s unusually favorable tax treatment of multinationals was a deliberate policy
to attract foreign companies.
The mechanism Apple used was a variant of a structure known as the Double Irish,
which Irish tax law had made available through a combination of
specific provisions and deliberate regulatory tolerance.
Ireland taxes companies that are managed and controlled from Ireland.
The United States taxes companies incorporated in the United States.
Apple’s Irish subsidiaries
were incorporated in Ireland but had their management and control located outside Ireland,
so Ireland did not tax them.
They were also not incorporated in the United States, so the US did not tax them either.
At its peak, this arrangement sheltered tens of billions of dollars annually.
The intellectual property component made the structure self-sustaining.
Apple’s valuable patents were licensed from a subsidiary in a low-tax jurisdiction.
European sales flowed through Apple Sales International,
which paid royalties back to the IP-holding entity.
The royalties reduced taxable profit in the high-tax jurisdictions where the sales occurred.
Transfer pricing
(the setting of prices for transactions between subsidiaries of the same company)
is supposed to follow the arm’s length principle:
the price should be what unrelated parties would charge each other.
In practice, there is no market price for a license to Apple’s entire product ecosystem,
so whatever number Apple’s accountants put in the contract became the price.
Tax authorities in each country then had to prove the price was wrong,
which requires the kind of information that companies are not generally eager to provide.
Apple was not alone, and Ireland was not the only country involved.
The Dutch Sandwich added a Netherlands entity to the structure,
exploiting a Dutch tax provision that exempted certain royalty payments from withholding tax.
Royalties would flow from Ireland to the Netherlands,
then onward to the zero-tax jurisdiction,
each step reducing the amount captured by any jurisdiction that might want to tax it.
Luxembourg served a similar function for Amazon;
by putting its European headquarters there,
the company could book sales across the continent while keeping profits in a jurisdiction
that had negotiated unusually favorable rulings with dozens of major multinationals.
Google’s arrangement was so elaborate to have its own nickname:
the Double Irish with a Dutch Sandwich [Shaxson2011].
The OECD’s BEPS project,
launched in 2013,
produced a fifteen-point action plan to close these loopholes.
Countries that adopted its recommendations
were supposed to require substance in the jurisdictions where companies claimed tax residence,
limit the deductibility of interest payments used to shift profits,
and require country-by-country reporting to make the overall structure visible.
Ireland amended its tax law in 2015 to prevent new companies from using the Double Irish structure,
but gave companies already using it a five-year phase-out period.
The 2021 agreement among 136 countries on a global minimum corporate tax rate of 15%,
negotiated under OECD auspices,
went further than any previous multilateral effort.
It established the principle that no matter where a multinational’s profits were booked,
at least 15% would be paid somewhere.
The agreement was called historic,
which was accurate,
and transformative,
which was not,
since implementation depended on each country passing domestic legislation.
Gabriel Zucman’s estimates suggest that
roughly 40% of multinational profits are shifted to tax havens annually,
costing governments approximately $200 billion per year in corporate tax revenue.
This is money that does not fund schools,
infrastructure,
or healthcare in the countries where the economic activity actually occurred.
The companies that benefit from this system defend it as legal, which it is.
They also describe it as responsible tax planning,
which is a creative use of the word responsible
[Zucman2015,Schneier2023].
The money that does not reach public treasuries does not simply disappear.
Some of it returns to public life through a different channel,
one controlled by the same people who structured the avoidance.
In June 2010,
Bill Gates and Warren Buffett announced the Giving Pledge:
a commitment by the world’s wealthiest people to give away the majority of their fortunes.
By 2023, more than 230 billionaires from thirty countries had signed,
representing combined pledges of over a trillion dollars.
The press coverage was broadly admiring,
and mostly did not ask who would decide where the money went.
The combination of great private wealth and public purpose is not new.
Andrew Carnegie,
who accumulated his fortune partly by suppressing wages and,
in 1892,
hiring Pinkerton detectives to break a strike at his Homestead plant that left ten workers dead,
spent his final decades funding libraries, universities, and concert halls.
Cities that received Carnegie libraries did so on his terms:
local governments had to provide the land and commit to maintaining the buildings in perpetuity.
The offer was generous; it also bypassed democratic mechanisms.
The more consequential precedent was the Flexner Report of 1910,
commissioned by the Carnegie Foundation for the Advancement of Teaching.
It recommended consolidating medical education around elite university-based institutions
and closing the smaller proprietary schools it judged substandard.
These changes raised training standards.
They also closed most of the schools that had trained Black physicians
or admitted women in significant numbers.
Private money had shaped outcomes that no democratic process had approved [Reich2018].
Today’s private foundations differ from Carnegie’s personal giving in one critical respect:
the American tax code.
Since the Revenue Act of 1917,
charitable contributions have been deductible from taxable income.
When a billionaire transfers appreciated assets into a private foundation,
they avoid capital gains taxes on those assets and receive an income tax deduction immediately.
The assets remain under the effective control of the foundation’s board,
which in practice consists of the donor and their chosen representatives.
Private foundations are required to distribute at least five percent of assets annually,
but the other ninety-five percent continues to grow tax-free under the donor’s direction.
What looks like giving money away is, legally and financially,
converting taxable personal wealth into a permanently controlled institutional endowment
while capturing the tax benefit upfront.
Donor-advised funds extend this further:
the donor takes an immediate deduction
but retains the ability to direct distributions to any eligible recipient for decades.
In 2022, donor-advised funds held over $230 billion in assets.
In the 2000s,
the Gates Foundation, working alongside the Walton Family Foundation and the Eli and Edythe Broad Foundation,
channeled hundreds of millions of dollars into reshaping American K-12 education.
Their preferred agenda included charter schools, standardized testing, merit pay for teachers,
and the Common Core State Standards.
For a period,
these became the dominant policy agenda of American school reform.
The decision-makers at these foundations were not elected
and could not be removed by the parents, students, or teachers whose schools were being reorganized.
In 2017,
Gates Foundation CEO Sue Desmond-Hellmann published an open letter acknowledging
that the foundation’s strategy had not produced the results it intended.
She described it as a learning experience.
Most of the teachers whose work and lives had been thrown into turmoil used stronger language.
This kind of philanthrocapitalism extends beyond education.
For many years the Gates Foundation was the second-largest funder of the World Health Organization,
behind only the United States government.
During the COVID-19 pandemic,
this made it an influential voice in debates about vaccine distribution,
intellectual property waivers for low-income countries,
and global health system priorities.
The fact that this influence was exercised with what donors believe are good intentions
does not change the fact that they lack the democratic legitimacy of public institutions.
Who gets what, and why, is not only a question about the distribution of economic resources.
It is also a question about who is allowed to shape
the rules of that distribution [Giridharadas2018].
How can you condense the millions of words that have been written about privacy
into a blog post?
The answer is that you can’t;
all you can do is point at a few landmarks,
like a tour guide trying to show people Toronto in an afternoon.
What Privacy Is Not
Privacy is not a timeless natural right.
As Sarah Igo describes in her history of privacy in America,
what is and isn’t private has expanded in some directions and contracted in others over the past century
with no consistent underlying principle.
Sexual behavior between consenting adults was legally public well into the twentieth century,
in the sense that it was criminally regulated.
For much of the same time,
medical records were routinely shared between physicians, employers, and insurers without patient consent.
The expansion of legal privacy protection into both of these areas
was the result of specific case-by-case struggles [Igo2020].
Similarly,
financial transactions that were once private are now reported to governments
under anti-money-laundering and tax-compliance measures,
and communications that were once protected by the practical difficulty of interception
are routinely monitored at scale [Snowden2020].
These shifts also do not reflect a coherent theory of what should be private.
They reflect the outcomes of contests between specific interests,
and the interests of states and large corporations have generally prevailed over
the interests of individuals.
The concept of a private individual self—a zone belonging to a person rather than to a household,
a clan, or a community—is also historically recent and unevenly distributed [Jenkins2024].
In pre-modern Europe, the unit of privacy was the household, not the person,
and the head of household had authority over what happened inside it.
Women and servants had no separate private sphere.
In Ottoman law, the harim (from the Arabic haram, meaning forbidden or protected)
designated the domestic sphere of the household as legally inviolable:
not the “harem” of European fantasy, but a legal protection preventing
state or community intrusion into the home without cause.
This was a meaningful protection, but it applied to the household as a unit,
not to the individuals inside it.
Japan’s traditional distinction between uchi (inside) and soto (outside)
organized public and private behavior very differently from Western liberal individualism.
The interior of the household was intensely private from outside scrutiny,
but the expectation of individuals’ interior privacy—thoughts, feelings, and desires
kept from family members—was much weaker.
The communal interior of the family, not the individual self, was the protected unit.
Indigenous knowledge systems in many parts of the world
operate through collective custodianship of sacred, medicinal, or ceremonial knowledge.
This is a form of collective privacy:
certain knowledge belongs to particular communities and is not available to outsiders.
Colonial administrators and anthropologists treated this as obscurantism,
and “freedom of information” in the colonial context often meant forced extraction:
ceremonies documented, sacred objects removed, and genealogies recorded for administrative purposes.
Privacy as extraction is the mirror image of privacy as protection.
Tiffany Jenkins’ history recounts how the boundary between public and private
has been drawn and redrawn by courts, legislators, and social movements,
always in someone’s interests [Jenkins2024].
The modern expansion of state intervention into domestic life
with child protection laws, domestic violence legislation, and regulation of reproductive rights
was often demanded by reformers and feminists against the privacy claims of male heads of household
who wanted the state to stay out.
The private sphere has not been progressively liberated from state interference;
it has been a contested terrain in which some forms of privacy have been won and others have been imposed.
The Psychology of Private Space
The sociologist Erving Goffman argued in 1959 that
everyday social interaction is a kind of performance [Goffman1959].
People present a “front stage” version of themselves when they know they are being observed:
maintaining composure,
managing impressions,
and performing the role expected of them in a given context.
The “back stage” is where the performance can relax:
the kitchen before the dinner party,
or the break room after the meeting.
People aren’t less authentic on one stage or the other;
the back stage is just where contradictions can be worked out
and where the next performance can be prepared.
This is not dishonesty:
everyone does it because social life requires it.
But it means that privacy is not merely about concealment;
it is about having space in which not every thought and action is subject to evaluation and judgment.
A society without back stages is not one in which people become more authentic.
It is one in which performance becomes continuous and exhausting,
and in which mistakes cannot be made without permanent record.
To see how harmful this is,
look at the lives of child stars.
Its practical consequences are measurable.
Surveillance changes behavior even when nothing being done is wrong.
After Edward Snowden’s revelations in 2013
made the scale of the NSA’s surveillance clear [Snowden2020],
researchers documented a measurable drop in Google searches
for terms associated with terrorism, drugs, or other sensitive topics—not
because fewer people were curious,
but because curiosity became something that required calculation.
This is the chilling effect:
the awareness of being watched changes what people are willing to risk.
Anna Funder’s oral histories of Stasi survivors document
what sustained surveillance does over time [Funder2011].
The East German Ministry for State Security employed roughly 90,000 full-time officers
and had a network of approximately 180,000 informal informants
in a country of 16 million people,
a ratio of state surveillance capacity to population that has never been matched.
Many of those informants were family members, recruited without one another’s knowledge.
The files they produced recorded personal relationships,
sexual behavior,
political opinions expressed in private,
and the contents of letters that were opened and resealed.
The goal was not to prevent crime,
but to map the population thoroughly enough
to identify, isolate, and destroy anyone who might become a source of organized opposition.
What Funder’s interviews reveal is not only the accumulation of files,
but the internalization of the watcher.
Stasi survivors describe a permanent alteration:
the habit of measuring every word before speaking it,
the inability to stop even decades after the state that required it had dissolved.
The surveillance state does not only operate while it exists;
it rewires the people it surveils.
The developmental case for private space is equally compelling.
Adolescents need to make mistakes without those mistakes being permanent.
They need to try on political positions, personal identities, and beliefs,
and then discard them without a record of the discarding.
This isn’t culturally specific:
all societies that have organized the transition to adulthood
have had designated spaces and periods in which the normal rules of visibility
and accountability did not apply.
Initiation rites that occur outside community view,
age-grade houses where elders could not enter,
and periods of deliberate ambiguity before social roles were fixed
were not privacy in the Western legal sense,
but they served the same developmental function.
The harm of involuntary exposure is correspondingly real.
The charivari was a public shaming ritual practiced across medieval and early modern Europe.
It deployed community visibility as a disciplinary tool against people who violated social norms.
The same mechanism operated in public “struggle sessions” during the Cultural Revolution in China,
in the public confession practices of various religious traditions,
and in the contemporary practice of doxxing.
These vary in severity,
but they share the same logic:
exposure can be a weapon,
so the ability to protect against exposure is not a luxury
but a precondition for social participation.
The Freedom to Become
Mary Gray’s study of rural LGBTQ+ youth in the United States
documents what happens when private space for identity exploration
is not available in physical form [Gray2009].
Young people in small towns or religious communities
where queer identity is dangerous or invisible
used the early internet to try out identities and learn that they were not alone.
The internet was not yet the optimized engagement machine it became;
it was a relatively unmonitored space in which pseudonymity was normal
and nobody was selling your search terms to your parents’ employer.
This was not specifically American.
The same pattern has been documented in Malaysia, Uganda, Indonesia, Turkey,
and other countries where same-sex relationships are criminalized or heavily stigmatized.
Online anonymity functions as a prosthetic private space
for people who have been denied physical private space.
It does not solve the underlying problem,
but it allows identity development to occur at all.
For most of history, “Anonymous” was a woman.”
— Virginia Woolf
Pseudonymity has a longer history than the internet.
George Eliot was the pen name of Mary Ann Evans,
who wrote under a male name partly to be taken seriously
in a literary culture that dismissed women novelists,
but also because a male persona gave her the freedom to inhabit subject positions
her own social location would not have authorized.
In the Soviet Union,
hand-typed, passed-by-hand underground literature called samizdat
was passed from person to person.
Writing and circulating it was criminalized;
it could only exist because private space existed,
even under a surveillance state,
in the gap between what the state knew and what it could actually monitor.
Alexander Solzhenitsyn’s The Gulag Archipelago was first circulated as samizdat
before being smuggled out and published abroad.
The private space of one person’s typewriter was a form of speech protection
that the absence of legal protection could not entirely eliminate.
The Hijra communities of South Asia—a third-gender identity category
that existed for centuries before British colonization—survived
partly because the British administrative apparatus never successfully catalogued them.
The Criminal Tribes Act of 1871 attempted to classify and control gender and sexual nonconformity,
but the practices, initiations, and knowledge that defined Hijra identity
were not legible to census-takers and administrators.
Collective privacy as survival is not the same as individual privacy as a right,
but they share the property that control over what is known about you
is also control over what can be done to you.
The double bind is consistent across these examples.
The people with the greatest need for private space to develop and maintain identity
are usually those whose identity is most criminalized, stigmatized, or surveilled.
Simone Browne documents how oppression of Black people in the United States
has always depended on making Black bodies, households, and movements visible to white authority,
from the pass system of the antebellum South
to stop-and-frisk policing and facial recognition [Browne2015].
The practical privacy available to Black Americans has always been
substantially less than that available to white Americans,
not because of individual choices but because of systems built to make them more visible to the state.
The “nothing to hide” argument—the
claim that only those with something to hide need privacy—only
makes sense from the position of someone whose identity, relationships, and beliefs are not criminalized.
It is not a neutral observation about human behavior;
it is the argument of someone who has never needed the protection they are asking others to give up.
The Moral Arguments
The first American legal argument for a right to privacy
was made by Samuel Warren and Louis Brandeis
in an 1890 law review article.
They argued that the individual had a “right to be let alone”,
i.e., a right to protection not just from physical intrusion but from unwanted publicity.
Their immediate concern was sensationalist newspaper journalism.
Both men were wealthy, socially prominent, and worried about gossip columns.
Their argument was real and important,
but its social location is worth noting:
the first articulated right to privacy in American law emerged from
the needs of the prominent to be protected from exposure,
not from the needs of the powerless to be protected from the state.
This origin has never fully left the concept.
Privacy arguments protect domestic violence abusers alongside their victims.
They protect financial fraud alongside confidential medical records,
and corporate malfeasance alongside personal correspondence.
Privacy as a legal and political value does not come pre-sorted by who it helps.
The feminist critique made this plain.
The slogan “the personal is political,”
associated with the women’s liberation movement of the 1960s and 1970s,
was a direct challenge to the liberal privacy argument.
Women’s subordination was legally protected in the private sphere:
for example,
marital rape was not a crime in most US states until the 1970s and 1980s
because marriage was treated as a private relationship outside legal scrutiny.
Domestic violence of other forms was (and in practice still is)
routinely treated as a private family matter.
The claim that the state should not interfere in private relationships was,
in practice,
the claim that women had no recourse against husbands and fathers.
Jenkins argues that the twentieth-century expansion of state intervention into domestic life
was not an erosion of privacy but a redrawing of its boundaries [Jenkins2024].
Feminists and child advocates demanded that the state enter spaces
it had previously left to male authority.
In doing so, they were not anti-privacy;
they were arguing that the women and children in those spaces
deserved privacy from the men who controlled them.
The same concept, applied by different actors, produced opposite conclusions.
The communitarian critique of privacy takes a different shape.
Communities can and do provide forms of accountability that states cannot,
and privacy can be an obstacle to that accountability.
But this argument has a history of being deployed most aggressively against communities
that were not the ones making it.
In British India,
child marriage prohibitions and widow protection laws were introduced
in a framework that described Hindu family privacy as barbaric and in need of civilizing intervention.
The language of accountability to higher norms was used to justify intrusion
into the domestic practices of colonized people by administrators
who had no sustained interest in the well-being of the women and children they claimed to protect.
American privacy worked the same way domestically [Igo2020].
Poor families and immigrant families in the early twentieth century
were subjected to social worker surveillance and home inspection regimes
that middle-class families were not.
The expansion of welfare services came with
a corresponding expansion of state visibility into the households that received them.
Privacy, like other goods, was distributed in proportion to social power.
Identity and Legibility
In 1879, a young police clerk in Paris named Alphonse Bertillon
proposed a solution to a problem that had plagued law enforcement for decades:
how do you know if the person in front of you is who they say they are?
Before photographs were cheap to reproduce and before fingerprint databases existed,
professional criminals could simply give a false name and walk free.
Bertillon’s answer was anthropometry:
measure the skull, the length of the forearm,
and other bodily dimensions that, taken together,
were statistically unlikely to be identical in any two people.
The system spread across Europe, the United States, and colonies in Asia and Africa,
and was the first large-scale attempt to use the body as a database.
Bertillonage had a fundamental weakness:
measurements had to be taken correctly by trained operators.
By 1900 fingerprinting was replacing it almost everywhere
because fingerprints were more reliable and required less skill to record.
But the desire of states and institutions to pin individuals permanently to a record
did not die with the calipers.
The history of identity management is partly the history of states
trying to solve what James C. Scott calls the legibility problem [Scott1998].
A state cannot tax, draft, or police people it cannot identify.
Medieval English peasants might know themselves as “John the Miller’s son from the village by the ford”,
but that description doesn’t survive a move to a city or a change of occupation.
Surnames became standardized in Europe partly because governments needed them.
The same logic produced house numbers in Paris and Vienna,
censuses across the colonial world,
and passports that began as occasional travel documents issued by monarchs
and became, by the twentieth century, a requirement for crossing most international borders.
The problem is that the state’s desire for legibility doesn’t have a built-in limit.
The state that registers births so it can provide schooling
can use that register to conscript soldiers.
The government that issues identity documents to allow people to vote
can use those documents to deport people.
Once a population register exists,
every subsequent administration can use it for whatever purpose it finds useful.
Colonial governments exploited this systematically.
The British introduced population registers, caste certificates,
and tribal designations across India, Africa, and Southeast Asia
that served double purposes:
administration, taxation, and census on one hand,
and identifying potential troublemakers and restricting movement on the other.
South Africa’s pass laws,
introduced gradually from the eighteenth century and formalized under apartheid after 1948,
required Black South Africans to carry a reference book at all times.
The book recorded their employer,
their designated “homeland,”
and their permission to be in urban areas.
Police could stop anyone and demand the book;
failure to produce it meant arrest.
The pass system was one of the most sophisticated identity management infrastructure projects in history,
and it was designed entirely to restrict freedom of movement
and force labor into mines and factories at wages set by the government.
Keith Breckenridge’s history of South African biometrics
traces a direct line from the pass system
to the world’s first large-scale biometric population register,
introduced in South Africa in 1986.
Fingerprints were added to the passbook
because fingerprints are harder to falsify than signatures
and do not require literacy.
A system designed to prevent forgery of internal passports became,
almost automatically,
one of the most comprehensive biometric databases in the world at the time [Breckenridge2016].
This creates a dilemma that does not have a clean solution.
Democratic participation depends on being able to identify voters.
Electoral systems need to verify that voters are eligible residents
and prevent people from voting twice.
This requires some form of voter registration,
and voter registration requires identity documentation.
In the United States,
the history of voter registration is entangled with the history of voter suppression:
poll taxes, literacy tests, grandfather clauses,
and, since the 1990s, photo ID requirements that are nominally neutral
but fall disproportionately on communities
less likely to hold a driver’s license:
the poor, the elderly, and people of color.
The registration systems needed to enable political participation
have been weaponized to prevent it.
But the dilemma goes deeper than that.
Communities that have historically been targeted by state identity systems
have rational reasons to distrust those systems.
If the government has used population registers to intern Japanese-Americans
or to deport undocumented immigrants in mass raids,
people are right to be suspicious of any new identity system.
Even if today’s government only intends to use it for good,
it cannot bind the behavior of future governments.
This dilemma is now one of the biggest challenges facing democracy.
The communities most harmed by identity management in the past
have the best reasons to distrust registration systems,
but that distrust keeps them from participating in the political processes
that would let them constrain those systems.
Fear of being rounded up is itself a tool of disenfranchisement.
India’s Aadhaar system, launched in 2009, has enrolled over 1.3 billion people
in a biometric identity database based on fingerprints and iris scans
linked to a twelve-digit number.
Its designers argued that a universal, biometric identifier would eliminate
fraud and exclusion in government benefit programs
by making identity verification objective.
The argument was partly correct:
Aadhaar has reduced certain kinds of fraud
and brought some previously excluded people into the formal economy.
It has also created new forms of exclusion.
People who cannot authenticate are cut off from food rations and pension payments,
and the database itself represents a concentration of sensitive biometric information
that, if breached, cannot be reset:
you can change your password, but you cannot get new fingerprints [Khera2019].
The European Union’s General Data Protection Regulation (GDPR),
which came into force in 2018,
treats biometric data as a special category requiring explicit consent
and restricts its collection and processing.
The regulation is imperfect and inconsistently enforced,
but it exists because privacy advocates organized across member states
over roughly two decades to demand it.
It is the most significant constraint on commercial biometric data collection in the world.
The tech industry’s response has been intrusive pop-ups
designed to make users blame governments for safeguarding their privacy
rater than companies for trying to collect information for resale.
What abusive identity management has been constrained,
those constraints have been the result of organized political activity.
South Africa’s pass system was not abolished because its administrators became enlightened;
it was abolished because the resistance movement grew strong enough to make it unsustainable.
The constraints on Aadhaar came from litigants and activists who brought cases,
and the Voting Rights Act of 1965 in the United States came from marchers,
organizers,
and a political coalition that made the status quo more costly than change.
The center always wants more information.
The question is always whether the people most likely to be harmed by that information
have enough power to say no [Cole2002,Torpey2000].
Paying for the Privilege
The surveillance economy that social media and AI have created
goes far beyond anything the Stasi dreamed of.
The irony is that the Stasi had to coerce their informants;
we pay subscription fees for the privilege of being surveilled.
Corporate data collection operates under a framework built largely around consent
obtained through contracts that no one reads
and that cannot meaningfully be refused by people who want to participate in modern life.
When a platform argues that its collection of behavioral data is voluntary
and therefore not a privacy violation,
it is making a legal argument, not a factual one.
The legal distinction between state surveillance and corporate surveillance
also ignores how governments now use corporate data collections
to conduct surveillance they could not legally conduct directly.
Facebook’s recommendation algorithm promoted content dehumanizing the Rohingya people in Myanmar,
a fact the company’s own researchers documented.
The UN’s 2018 fact-finding mission identified Facebook as
a “contributing factor” to the genocide:
corporate surveillance infrastructure was weaponized for ethnic cleansing
at not cost to the platform or its owners.
When Privacy Protects Power
This asymmetry is most visible in the global system of financial secrecy.
Nicholas Shaxson’s investigation of offshore banking documents
how a network of tax havens
in the Cayman Islands, the British Virgin Islands, Switzerland, Luxembourg, Ireland, and Singapore
was constructed over the twentieth century
through legislation specifically designed
to allow wealth to evade the oversight of the countries where it was generated [Shaxson2011].
Switzerland’s banking secrecy law of 1934
was enacted partly to protect the assets of Jewish depositors from the Nazi government;
within years it was also protecting the assets that wealthy Europeans
were hiding from their own governments’,
and then the money, art, and treasures that Nazis had looted from their victims.
Gabriel Zucman estimates that approximately $7.6 trillion in private wealth is held in offshore accounts,
costing governments roughly $200 billion annually in lost corporate tax revenue
and further amounts in lost personal income tax [Zucman2015].
This is money not available for schools, infrastructure, or healthcare
in the countries where the underlying economic activity occurred.
The offshore system is legal because wealthy individuals and corporations have
the resources and political influence to make it legal.
The Panama Papers and Pandora Papers, published in 2016 and 2021,
revealed how this system operated across the world:
The Marcos family in the Philippines used shell companies in Switzerland and the Cayman Islands
to hold an estimated $5–10 billion looted from the Philippine treasury
during Ferdinand Marcos’s dictatorship.
Mobutu Sese Seko of Zaire accumulated an estimated $5 billion in Swiss and other offshore accounts
while the country’s public services collapsed.
Augusto Pinochet of Chile,
who tortured thousands in the name of fiscal discipline,
was revealed after his arrest in London to hold $27 million in hidden offshore accounts
that his family had denied existed.
Corporate secrecy operates the same way at a smaller scale.
Non-disclosure agreements are legitimate legal instruments
when they protect trade secrets or the terms of commercial negotiation.
They become instruments of harm when used to prevent victims of sexual abuse and harassment from speaking.
Harvey Weinstein’s non-disclosure agreements required victims’ silence as a condition of financial settlement.
Both Weinstein and Jeffrey Epstein continued to harm additional people for years
because the financial privacy of the settlement was used to suppress testimony
that would have enabled legal action.
The tobacco industry’s internal documents provide a decades-long parallel.
From the 1950s onward, tobacco company researchers produced evidence
that cigarettes caused cancer and that nicotine was addictive.
Those documents were treated as confidential corporate information—
trade secrets, attorney-client privileged communications—
while the companies’ public-facing research denied what their private research showed.
The internal memos were finally produced in 1998 litigation.
The deaths attributable to that gap between public claim and private knowledge are not a small number.
State secrecy operates the same logic at the largest scale.
Classification systems and executive privilege have been used
to prevent accountability for torture—the Abu Ghraib photographs were eventually published,
but the legal memos authorizing enhanced interrogation were withheld for years.
The Saudi government invoked sovereignty and confidentiality to slow accountability
for the murder of journalist Jamal Khashoggi.
Every authoritarian government that has committed atrocities
has used state secrecy as its first line of defense,
while using forced transparency—public trials, published confessions, exposed private lives—
as a weapon against opponents.
Privacy and exposure have always been deployed together,
each in the direction of power.
Transparency and Its Limits
The apparent solution—require transparency from the powerful—is correct in theory
but immensely complex in practice.
The difficulty is not an argument against transparency,
but does require advocates to be specific about who, what, and how.
Rob Jenkins and Anne Marie Goetz’s analysis of India’s Right to Information movement
documents what organized civil society can achieve [Jenkins1999].
The RTI Act, passed in 2005, gave Indian citizens the legal right to request government documents
and required agencies to respond within thirty days.
It emerged from a sustained campaign by village-level activists in Rajasthan
who were trying to verify whether public works funds were actually being spent
on the roads and wells they were supposed to fund.
The principle was simple:
government is funded by citizens and accountable to them,
and accountability requires information.
The law was imperfect and inconsistently enforced,
but still led to real change.
South Africa’s Truth and Reconciliation Commission,
which operated from 1996 to 1998,
made a different but related choice.
Perpetrators of human rights violations under apartheid
were offered amnesty in exchange for full public disclosure of what they had done.
The logic inverted the usual logic of settlement:
rather than silence as the condition of resolution,
truth-telling was the condition.
Perpetrators had to give up their privacy in order to receive amnesty;
victims could give testimony publicly,
which served both their dignity and the historical record.
The European Union’s General Data Protection Regulation,
which came into force in 2018,
includes a “right to be forgotten”:
individuals can request that search engines remove links to information about them.
This is a genuine protection for private individuals
who have been stalked, defamed, or whose past mistakes
have been permanently attached to their names by internet search.
It is also a tool that has been used by politicians, business executives, and convicted criminals
to suppress accurate reporting about their public conduct.
The same legal instrument protects both the person whose private medical condition
was published without consent and the politician who wants voters to forget a corruption conviction.
Privacy law cannot easily distinguish between them.
What holds across these cases is the principle:
privacy for persons, transparency for institutions.
Private individuals acting in their private capacity have strong claims to be left alone.
Public officials exercising public power,
institutions affecting public life,
and wealthy individuals whose private financial arrangements affect public resources
all have weaker claims.
This principle is never self-enforcing.
The direction of institutional pressure is always toward the legibility of persons
and away from the legibility of institutions.
Maintaining the opposite—protecting the private lives of individuals
while demanding transparency from power—
requires organized political effort to sustain [Scott1998].
The platforms and governments that describe privacy as “dead”
or as an obstacle to innovation are not making a neutral observation about changing social norms.
They are making an argument that happens to be extremely convenient for their interests.
The people arguing against that description are usually not doing so
on behalf of people who want to post embarrassing photos without consequence.
They are doing it on behalf of the people for whom privacy is the difference between safety and harm.
What abusive privacy and identity regimes have been constrained,
those constraints have come from organized political activity,
not from enlightenment at the top [Igo2020].
My original idea for these posts was to write a series titled “Big Tech Is Like…”
The sections below were fun to write;
I hope they’re also fun to read.
Big Tech is Like a Drug Cartel
[Wainwright2017] is one of my favorite books.
In order to show how the free market really works,
he went and studied it in its pure, unconstrained form:
the cocaine cartels.
It was a fun and insightful read,
and ever since I first encountered it
I’ve wanted him to go back and do a similar book about big tech companies.
After all,
they too sell artificially addictive products,
treat the legal system as a mere expense,
and are run by sociopathic narcissists.
Wainwright’s thesis is that every business faces a common set of problems:
how to maintain product quality,
how to protect market territory,
how to enforce agreements with suppliers,
how to recruit and manage staff,
and how to handle disputes with competitors.
Legal businesses deal with these issues through a combination of contract law,
trademark and patent protection,
employment law,
and (if all else fails) litigation.
Drug cartels solve the same problems by different means,
and their solutions reveal some interesting things about how business actually works.
Take brand protection, for example.
A consumer who buys a product with a known brand name
has some assurance of consistent quality because
the brand owner has reputational incentives to maintain standards
and can take legal action against counterfeit goods.
A cocaine cartel can’t use trademark law.
It can,
however,
use violence against competitors who sell adulterated product under the same name
or who operate in territory the cartel has claimed.
The aim—maintaining exclusivity and quality signals—is the same,
it’s just the mechanism that differs.
(And yes, the word “just” is doing a lot of work in that sentence.)
However,
violence is like litigation:
it’s expensive,
it’s noisy,
and you might lose in the end,
so you have a lot of reasons to try to negotiate a settlement,
even if it’s a bad one.
Worker relations are also similar.
Employment law has evolved over the last hundred and fifty years
to reduce mistreatment of workers through minimum wages,
safety requirements,
and prohibitions on arbitrary dismissal.
People who work in the drug trade have no such protections.
However,
the brutal labor relations practices of the cartels are not a consequence of the personalities involved.
They are instead the result of removing the legal protections that exist
precisely to prevent brutality and exploitation in legal labor markets.
As a final example,
consider vertical integration.
A business that controls its entire supply chain
is less vulnerable to supplier failure,
price gouging,
and quality problems.
Legal businesses achieve this through acquisition and contract,
but face antitrust limits on how far they can consolidate.
(At least, they used to:
antitrust enforcement has been steadily weakened since the early 1980s.)
Cartels can’t sue suppliers who fail to deliver,
so they have a powerful incentive to own suppliers outright.
The cartels that have achieved the most durable market positions
have typically integrated vertically as far as they can,
unconstrained by antitrust law.
Tech companies use the term “disruption”
to describe entering and reorganizing an existing market.
From the perspective of the incumbent businesses in that market,
disruption means that the pricing norms,
labor deals,
and the competitive equilibria they had established
are attacked by a new entrant willing to operate at a loss
or outside the regulatory frameworks that the incumbents are bound by.
Drug cartels are, in this sense, serial disruptors:
they subvert the legal system to undercut incumbents when they enter markets,
often at great cost to local communities.
If this sounds like the way Uber, AirBnB, and others have operated,
that’s not an accident:
the connection between market concentration and the capacity to externalize harm
is a structural feature of markets,
not something specific to drug cartels.
A highly concentrated firm in any industry has reduced competitive pressure,
which means it can absorb the reputational cost of poor labor or environmental practices
without losing market share.
It can also use its political leverage to shift the cost of its harms onto workers through suppressed wages,
onto communities through pollution and infrastructure strain,
and onto governments through underfunded services and tax avoidance.
Cartels exhibit this pattern in its most extreme form because they face the weakest institutional constraints,
but the underlying logic is no different from what Google, Meta, and others have been doing for years.
The safeguards of legal markets that people take for granted in first-world democracies
were invented specifically to prevent the damage done by unregulated markets.
Minimum wage laws,
environmental standards,
workplace safety rules,
antitrust regulations,
and consumer protection statutes are society’s equivalent of immune reactions.
Each was lobbied against by the industries they constrained
on the grounds that regulation would be impossible, ineffective, immoral, expensive,
or harmful to national security.
As we belatedly start to think seriously about regulating social media and AI,
I think it’s worth keeping the cocaine cartels in mind
[Skarbek2014].
Big Tech is Like a Long Firm Fraud
Another book that I really enjoyed was [Davies2022],
which is about how legendary frauds reveal the workings of the world.
while he doesn’t discuss big tech,
the parallels are inescapable.
Understanding the classic patterns of fraud laid out in Davies’ book
makes it possible to identify them in the software industry,
which is not yet regulated well enough to make them prosecutable.
Fraud is more common than prosecutions suggest,
and its most dangerous forms are not easily recognized as fraud
even by the people committing them.
Many frauds begin as genuine optimism,
turn into rationalization,
and end in deliberate concealment.
By the time the fraud is clear from the outside,
its perpetrators may have so thoroughly internalized their own narrative
that they are genuinely shocked by the prosecution.
There is often no clear moment when they knew knew they were crossing a line.
The semi-technical definition of fraud is
“a deliberate misrepresentation of material fact
that causes someone to act to their detriment and the deceiver’s benefit”.
It is easily confused with incompetence, negligence, and bad luck,
but it is none of those things.
A startup that believed its technology would work and turned out to be wrong is a business failure.
A startup that knew its technology did not work and told investors and customers that it did is fraud.
The difference is difficult to establish because
(a) people can deceive themselves or misremember events
and (b) they can also just lie.
Three kinds of fraud are particularly relevant to tech companies.
A Ponzi scheme appears to generate returns
by paying early investors with money from later investors,
rather than from genuine investment gains.
Each successful payment increases the scheme’s credibility and attracts additional investors.
The schemer does not need to maintain a lie in the face of contradictory evidence:
the evidence available to most participants is the evidence of payment,
which is real.
However,
the scheme can only continue as long as new investment exceeds required payments.
When new investment slows and required payments exceed available funds,
the scheme collapses.
The timing of the collapse is usually determined by market conditions outside the schemer’s control.
This structure means that a Ponzi scheme operator may not know when the collapse will come,
which provides an incentive for them to fool themselves as well as their investors.
After all,
the scheme worked yesterday—maybe it will work again tomorrow.
A long firm fraud operates on a longer timescale.
The fraudster starts by building a legitimate-seeming business,
establishing a track record of reliable transactions and prompt payments.
Once they have a reputation for reliability,
they use that to place large orders on credit,
receive the goods,
sell them quickly at a discount for cash,
and disappears before the creditors can collect.
The victim’s mistake is not in trusting the operator—the early trust is actually warranted.
The mistake is failing to notice that the scale and urgency of later transactions
is inconsistent with normal business patterns.
Control fraud operates inside a legitimate company
rather than through a fictitious one.
An executive who controls a company can use that control to route contracts to related parties,
manipulate reported earnings to maximize personal compensation,
extract value through structured transactions that are difficult for outside observers to interpret,
and maintain an appearance of legitimacy for years.
William Black’s The Best Way to Rob a Bank Is to Own One
described the role control fraud played in the savings and loan crisis of the 1980s,
and how it was made possible by Reagan-era deregulation that removed oversight mechanisms,
and by accounting standards that gave executives wide latitude in reporting.
The auditors who certified the books
were paid by the companies they audited
and had professional and commercial incentives to take management’s claims at face value
[Black2005].
Finally,
accounting fraud works because
accounting requires judgment at every stage where fraud can be inserted.
Revenue can be recognized early or late;
assets can be valued on optimistic or conservative assumptions,
and liabilities can be disclosed prominently or buried in footnotes.
Each of these choices meets professional standards and is individually defensible.
Where the fraud comes in is consistently choosing the most aggressive position in every ambiguous case.
Auditors frequently fail to catch accounting fraud
because of the volume of material they need to review,
and because their adversaries anticipate the specific tests that will be used to detect it.
Most importantly,
auditing fails because big accounting firms depend on their clients for their income,
which gives them a powerful reason to not find any problems
[Palazzo2025,Perrow1999,Schneier2023].
Wirecard, a German payments company certified by top-tier auditors,
fabricated approximately €1.9 billion in assets for years
while Germany’s financial regulator actively shorted companies that raised alarms about it.
This control fraud was enabled by exactly the same auditor capture and regulatory failure
that the savings and loan crisis demonstrated decades earlier.
Big Tech is Like the Enclosure Movement
Email, RSS, the open hyperlink, and the early web were commons:
shared infrastructure anyone could build on.
Social platforms converted that commons into walled gardens,
moving the audience inside and charging rent for access to it.
In doing this,
big tech companies were following a centuries-old playbook [Bollier2014].
The idea that an individual can own a piece of the earth’s surface
is younger than most people realize,
specific to certain legal traditions,
and was imposed on much of the world by force.
In most places throughout most of history,
land was managed collectively via overlapping use rights rather than exclusive ownership.
Understanding how private property in land was created,
and what was destroyed in the process,
lets us ask clearer questions about property rights generally:
not “is this property?”
but “who decided it was property, when, why, and who was dispossessed in the process?”
The best-known example of communal land being privatized
(to English speakers, anyway)
is the enclosure movement,
which peaked in the second half of the 18th century.
Before enclosure, common land was governed by overlapping, community-enforced use rights:
the right to graze a specific number of animals,
to cut timber for fuel,
or to fish particular stretches of a river.
Elinor Ostrom’s research on commons governance documented
how these systems were managed sustainably for centuries without either private ownership or state control
through locally developed rules, monitoring, and graduated sanctions [Ostrom2015].
The commons worked;
the argument that they were inherently prone to overuse
was made by enclosure’s beneficiaries and repeated long after it had been empirically refuted.
Parliament passed hundreds of private Enclosure Acts
between 1750 and 1850,
each one extinguishing common rights over a specific area
and converting that land into private property.
The process was not neutral arbitration between competing claims:
the landowners who stood to benefit were the same class that controlled Parliament.
The commoners whose rights were extinguished had no equivalent political representation;
when they received any compensation at all, it was consistently inadequate.
The result was a large-scale transformation of semi-independent smallholders into wage laborers.
The English enclosure model was not confined to England.
It was carried outward through colonialism as one of the primary instruments of dispossession.
In Ireland, India, sub-Saharan Africa, and the Americas,
colonial law systematically refused to recognize collective or customary land tenure.
Land that was not individually titled under a system legible to European courts
was declared waste, Crown land, or legally available for settlement.
This did to colonized peoples what the Enclosure Acts had done to English commoners:
extinguish practices that had sustained communities for generations,
convert resource into commodities that could be extracted or sold,
and make the people dependent on their new overlords [Linebaugh2014].
Brazil’s grilagem—the fraud of converting indigenous and public Amazon land
into private titled property using falsified documents—has been digitized in recent decades,
with forged records fed into government registries to launder claims at a scale
that replicates the logic of the Enclosure Acts across a continent [Linklater2015].
The differential legalization of pleasure follows the same logic as disruption.
Alcohol kills hundreds of thousands of people a year in wealthy countries and is sold in supermarkets.
Cannabis has a lower harm profile than alcohol by most measures
and was, for most of the twentieth century, a criminal offense
whose consequences fell almost entirely on Black and Latino communities in the US
and immigrant communities elsewhere.
Portugal’s 2001 decision to decriminalize personal possession of all drugs—
paired with expanded investment in treatment and harm reduction—
produced sharp falls in drug-related HIV transmission and overdose deaths
without increasing use rates relative to comparable European countries.
The pattern of what gets legalized, when, and for whom
tracks political economy, racial hierarchy, and commercial interest
far more reliably than it tracks harm [Hari2015,Nutt2012].
Big Tech is Like Professional Wrestling
The French philosopher Roland Barthes wrote about wrestling in 1957.
He wasn’t writing about sport;
he was analyzing a form of entertainment that reveals something important about how spectacle works,
and his argument was that professional wrestling is pleasurable not despite being scripted
but because it is scripted.
The audience is not there to watch a fair athletic contest.
It is there to watch the theatrical enactment of moral archetypes:
the villain who cheats openly,
the hero who suffers before triumphing,
and the referee who is briefly fooled before justice prevails.
The industry term for this collective fiction is kayfabe.
Wrestlers maintain kayfabe in public appearances,
in interviews,
and on social media,
even though every adult in the audience knows that wrestling is a performance.
This isn’t deception:
it’s a mutual agreement to treat the fiction as real
to make the experience more emotionally satisfying.
When a wrestler breaks kayfabe—acknowledges the performance from inside it—it violates a social contract,
not a factual claim [Barthes1972].
Big tech runs on a remarkably similar agreement.
Consider the long rivalry between Apple and Google.
Apple has spent decades positioning itself as the champion of user privacy,
in explicit contrast to Google’s surveillance-funded advertising model.
Tim Cook has delivered speeches describing privacy as a “fundamental human right.”
Apple introduced App Tracking Transparency,
which requires apps to ask permission before tracking users across other platforms.
This is the hero narrative.
Google, cast as the villain, harvests behavioral data at industrial scale.
What the kayfabe obscures is that Apple earns approximately $20 billion a year
from a deal that makes Google the default search engine on every iPhone sold.
The two rivals are financially interdependent,
and each needs the other to play its designated role.
Apple can charge premium prices for privacy-branded hardware,
and Google gets default distribution to Apple’s affluent user base
in exchange for revenue that Apple would otherwise have to replace.
Neither company has any incentive for the match to end.
In wrestling, the heel is the designated villain:
the character whose job is to be so obviously wrong
that the audience rallies behind whoever fights them.
Tech companies take turns playing this role.
Mark Zuckerberg testifies before the US Senate,
unable to answer basic questions about his own platform,
and for a season or two Meta is the villain everyone agrees on.
Then a new scandal breaks,
the spotlight shifts,
and a new villain takes the stage.
The rotation is not accidental.
Sustained outrage at one company might produce structural reform.
Outrage distributed across all of them produces blog posts like this.
The referee, in this reading, is the regulator.
Referees in professional wrestling apply the rules as written,
but the rules are designed to produce a compelling show rather than a genuinely fair contest.
The EU’s record fine against Google of €4.34 billion for Android antitrust violations in 2018
amounted to roughly two weeks of Google’s annual profit at the time.
Google appealed,
had the fine reduced slightly,
and continued the practices it had been fined for.
Barthes concluded that the real function of wrestling is to transform suffering into spectacle,
and spectacle into something that feels like justice.
The audience leaves satisfied because the villain was punished.
It does not matter that the match was scripted,
because the emotional experience of moral resolution is genuine.
This is what congressional hearings about social media accomplish for most people who watch them.
A senator reads a damning internal email,
a tech CEO says something evasive,
the clip circulates for a couple of days,
and people feel that accountability happened.
The executives return to their offices,
the company eventually pays a settlement with no admission of wrongdoing,
and the market structure that produced the problem continues unchanged.
Kayfabe is not lying.
It is a shared agreement to experience something as if it were real,
because that is more satisfying than experiencing it as what it actually is.
The question to ask about tech’s performance of competition,
concern for users,
and openness to accountability is not whether individual participants believe what they say.
Some of them probably do.
The question is whether the performance is producing structural change
or whether it is performing structural change for an audience
that finds the performance sufficient [Frankfurt2005,Mazer1998].
Big Tech is Like the Penny Press
The business model in which content is provided free to audiences
while advertisers pay to reach them
did not begin with social media.
It predates the web and television alike,
beginning with newspapers in the 1830s
and refined through a century of radio and broadcast television.
The technology changed but the logic did not:
the audience is not the customer but the product,
and the content is the mechanism for assembling and sorting that audience.
The penny press emerged in New York in the 1830s
when publishers like Benjamin Day realized that
a newspaper priced at one cent could not sustain itself through subscription revenue
but could attract large enough readership to sell advertising at rates that more than covered costs.
The New York Sun and its imitators were not selling information to readers;
they were selling readers to advertisers.
Sensational crime reporting, gossip, and human-interest stories
were not editorial lapses:
they built the audience that made the advertising worth buying.
Every subsequent development in advertiser-funded media follows from this original structure.
The competition between William Randolph Hearst’s New York Journal
and Joseph Pulitzer’s New York World in the 1890s
shows what this model produces when two well-funded publishers compete intensely for circulation.
Neither set out to destabilize democratic discourse;
they just wanted to sell newspapers.
However,
the market provided a clear signal:
crime, scandal, nationalist outrage, and stories written to produce emotional responses drove circulation,
and each escalation by one paper forced a matching response from the other.
Journalism that was inaccurate and inflammatory was not the result of a conspiracy.
It was the predictable output of two rational businesses
responding to the same incentive structure [Campbell2001,Nasaw2000].
The propaganda model [Herman1988] describes how
this system shapes content without requiring deliberate censorship.
Advertisers do not need to call editors and demand favorable coverage;
they simply withdraw revenue from outlets that publish content they dislike,
and editors learn what is acceptable without being told.
Interactive media doesn’t change any of this logic,
but it makes it more precise.
Behavioral data collected through social media, search engines, and app ecosystems
allows advertisers to target individuals rather than demographic categories.
The recommendation algorithm knows far more precisely
what content will hold each specific user’s attention for the next few minutes
than the newspaper editor deciding what to put on the front page.
Engagement maximization through algorithmic amplification and social harm are not separable
because the content that maximizes engagement is not randomly distributed.
Research consistently shows that outrage, fear, and social comparison
generate stronger and more persistent engagement signals
than accurate information, considered analysis,
or content that simply satisfies a question and lets the user close the tab.
Big tech companies that depend on advertising revenue
will therefore always amplify inflammatory distortion over mere facts [Wu2016].
Big Tech is Like Multi-Level Marketing
Jay Van Andel and Rich DeVos founded Amway in 1959
on the premise that anyone with enough ambition and the right social network
could build a business by selling cleaning products to friends and neighbors.
The products were real;
the business opportunity was considerably more complicated.
Amway is the founding institution of multi-level marketing (MLM),
an industry that by the 2020s had enrolled an estimated 120 million people worldwide.
The business model compensates them not just for selling products
but for recruiting others who will also sell products,
and for collecting a percentage of everything their recruits sell,
in a chain extending downward through a downline.
The mathematics of this structure are simple,
but tend not to appear in recruitment materials.
If each participant recruits three others,
and each of those recruits three more,
then a chain seven levels deep involves over 2,000 people
all of whom must sell product to sustain the commission structure above them.
The US Federal Trade Commission found in a 2011 analysis
that in one major MLM company,
fewer than 1% of participants earned a net profit after expenses.
The other 99% subsidized them [FitzPatrick2020].
When Uber launched, it told drivers they were entrepreneurs:
captains of their own ships,
free from the indignities of employment,
with no boss, no fixed hours, and all the flexibility they wanted.
What the pitch omitted was that Uber would set the price,
determine which rides were offered to which drivers,
deactivate accounts without meaningful appeal,
and systematically reduce driver earnings as market penetration increased.
Amazon Marketplace allows third-party sellers to list products,
reach Amazon’s enormous customer base,
and pay Amazon a commission on every sale.
It also allows Amazon to observe exactly which products are selling well
and then launch competing Amazon Basics versions,
using the sales data it collected from the sellers it hosts.
Marketplace recruits more sellers by showcasing successful ones,
in the same way that MLM recruitment materials feature the rare success story
while omitting the statistical reality for the average participant.
Social media scales this model even further.
Facebook’s users generate the content that makes Facebook worth visiting.
They also generate the social ties that make Facebook difficult to leave.
And they pay in attention and behavioral data
for the privilege of generating that content on Facebook’s infrastructure
under Facebook’s terms of service.
The users are the product.
This is not a metaphor:
it is a statement of the actual business model that appears plainly in investor materials.
Scientology’s auditing sessions collect whatever subjects disclosed:
accounts of illegal activity, sexual behavior, family conflicts, and statements about other people.
Former members have testified that these folders were used in disciplinary proceedings
and in litigation against critics and defectors.
Every major social media platform is, at its core, a similar system.
It collects behavioral data—what you look at, what you hesitate over, what you react to—and
that information is qualitatively different from what you share with a retailer.
People post about illness and grief and their political beliefs and sexual identity
because the platform presents itself as a community, not a database.
The fact that it is both doesn’t mean the user is naïve;
it means the platform is designed to exploit the social context
that makes sharing this information feel appropriate.
The biggest difference between MLM and social media
is that MLMs are occasionally subject to regulatory action.
The platform economy has largely avoided that outcome
by being larger, more diffuse, and more politically connected.
The most honest description of both is that they sell hope:
the hope that this time, for this person, the math will work out [Srnicek2016].
Big Tech is Like the Yakuza
In the days immediately after the March 2011 Tōhoku earthquake and tsunami,
investigators from the Asahi Shimbun documented how organized crime groups
supplied food, water, and emergency goods to affected communities
faster than official relief channels could mobilize.
This was not unusual.
The yakuza—Japan’s organized crime syndicates—have a long history of disaster relief,
partly because it generates goodwill,
partly because they maintain logistics networks and community ties
that allow them to operate quickly when formal institutions cannot,
and partly because disaster zones are also business opportunities.
Tech platforms now perform functions that governments once either provided directly
or regulated others to provide,
including identity verification, payment processing, and dispute resolution.
When a seller on eBay disputes a transaction,
eBay adjudicates the claim.
When a developer’s app is removed from the App Store,
Apple’s internal review process is the only available appeal.
When Facebook removes content in a country with regulated speech,
it is making regulatory decisions
in a jurisdiction where it has not been granted regulatory authority.
This is what makes tech’s political relationships so interesting.
Governments are simultaneously threatened by tech’s accumulation of quasi-governmental power
and dependent on tech’s infrastructure to operate.
The US government runs significant portions of its cloud operations on Amazon Web Services.
The Indian government used WhatsApp—owned by Meta—for public health communications.
The relationship is symbiotic in the same way that governments’ relationships with contractors always have been:
the state needs services the contractor provides,
the contractor needs the regulatory tolerance the state can provide,
and neither has a strong interest in severing the arrangement.
The yakuza model also illuminates how platforms handle competition.
Organized crime syndicates do not generally compete through price, but through territory.
Territorial disputes are settled through negotiation, credible threats, and occasional violence.
The enforcement mechanisms in tech may be different (so far),
but the territorial logic is similar.
Google defaults to Google Maps,
Apple’s App Store prohibits payment systems that compete with Apple Pay,
and Amazon uses its control of search ranking to disadvantage sellers
who also list products on competing platforms.
These practices are not illegal in most jurisdictions;
they are exercises of territorial power
by entities whose market position makes them difficult to challenge through normal competitive means.
The medieval Church provided civil infrastructure to large parts of Europe:
monasteries preserved manuscripts, the Church maintained hospitals and schools,
and Church record-keeping tracked births, deaths, marriages, and property transfers.
What the Church got back was the tithe—
an obligation levied on agricultural production,
enforced through ecclesiastical courts,
and owed regardless of the individual’s relationship with the Church.
Every household paid approximately one-tenth of its productive output.
Today, the App Store charges a thirty percent commission on every transaction made through it:
a tithe on productive activity conducted within the platform’s jurisdiction,
owed by anyone whose business depends on access to that platform’s infrastructure.
Excommunication—removal from the Church’s services—meant you could not conduct legal business.
Deplatforming works the same way:
a creator removed from a major platform loses access to customers, tools, and markets
with no due process and no external authority to appeal to [Southern1990].
So what do organized crime organizations provide in exchange for what they extract?
The yakuza have historically managed significant portions of Japan’s construction and entertainment industries
through a combination of legitimate business ownership and informal control over labor supply.
The arrangement is not purely extractive:
it provides predictability, dispute resolution, and protection from other organized crime groups.
Platforms offer analogous services.
Amazon Marketplace gives small sellers access to customers they could not otherwise reach.
App Store review provides users a degree of protection from malware.
Facebook Groups provide community infrastructure that many organizations genuinely depend on.
The question that needs to be asked of both yakuza-connected industries and platform-dependent businesses
is not whether the services have value,
but whether the entity providing them has made itself structurally necessary
specifically to extract rents that a competitive market would not sustain.
The yakuza (officially designated boryokudan in Japanese anti-crime legislation) are declining.
Japan’s anti-organized crime laws, passed in 1992 and strengthened since,
have made it progressively harder for syndicate members to interface with legitimate business.
Banks will not open accounts for known members,
real estate will not be rented to them,
and golf courses are required to turn them away.
Registered yakuza membership fell from roughly 180,000 in the 1960s to under 20,000 by the early 2020s.
None of this happened because the yakuza became less useful.
It happened because a sustained political decision was made
to make the cost of association with them prohibitive for legitimate businesses.
The tech industry’s political connections are currently a source of strength;
this history suggests they can be made a source of vulnerability [Adelstein2023,Kaplan2012].
Big Tech is Like the Radio Payola System
Supermarkets charge manufacturers slotting fees for shelf space,
paying premium rates for eye-level placement and end-cap displays.
A product placed at eye level is there because its manufacturer paid for the position,
not because it is the best option in its category,
but consumers have no way to know this.
In the 1950s and 1960s,
record labels paid disc jockeys to play their records,
presenting purchased airplay as their independent opinion.
Congressional hearings in 1959 exposed the practice,
the Federal Communications Commission made payola illegal,
and Alan Freed,
one of the most prominent DJs of the era,
was prosecuted and his career ended [Segrave1994].
However,
the hearings focused on individual wrongdoers rather than
on the structure that made the wrongdoing rational.
The practice therefore continued more or less uninterrupted
through intermediaries that technically complied with disclosure requirements
while achieving the same result.
This is an example of regulatory capture:
the agency charged with overseeing an industry becomes more responsive to the industry’s interests
than to the public interest it was created to protect [Stigler1971].
A related problem arises
when a platform operates as both a marketplace administrator and a competitor within that marketplace.
Google places its own vertical search products at the top of search results
while ranking competing services lower,
and Apple controls the ranking system on the App Store
that determines how visible its own applications and competing third-party applications are.
The consumer’s position is identical to the radio listener in the payola era:
prominent placement is a false signal of quality.
In 2016, a Chinese student named Wei Zexi died after following Baidu’s top-ranked search result
for a cancer treatment to a hospital that had paid for that placement,
exposing the search engine’s practice of selling results presented as editorial recommendations
to users who had no way to tell the difference.
Securities law prohibits an analogous conflict known as front-running.
A broker who executes customer orders is prohibited from trading against clients
using client order information
because taking advantage of insider information takes value from the customer.
The standards that financial regulators apply have not been extended to platform markets,
even though the identical conflict of interest produces the identical harm.
What would actually work is structural separation:
a search engine that derived no revenue from placement
would have no incentive to blur the line between paid and organic results.
Short of this,
accurate disclosure would require prominence and clarity that platforms have no incentive to provide,
since their business model depends on users not knowing that
the game is rigged [Doctorow2022,Stoller2019,Khan2017].
On Easter Sunday 1929,
a group of women walked down Fifth Avenue in New York City smoking cigarettes.
They had been hired by Edward Bernays,
a publicist working for the American Tobacco Company,
to light up in public and treat their cigarettes as what Bernays called “torches of freedom.”
Women smoking in public was a social taboo;
framing the act as feminist defiance was designed to dissolve that taboo
and open the female market to tobacco sales.
It worked.
Bernays was Sigmund Freud’s nephew,
and he had taken his uncle’s ideas about unconscious desire and applied them to commerce.
His insight was that you do not need to argue with people about whether they want something.
You can create the conditions under which they will want it
[Bernays2024].
The previous psychology post explained how
people make decisions differently than the rational-actor model predicts.
This post makes a related point:
before you can ask how people choose,
you need to ask where their preferences come from.
Standard economics treats preferences as given:
people arrive at markets with wants, and markets serve them.
Thorstein Veblen was among the first to argue that this is not how consumption actually works.
People don’t want things in the abstract;
they want things based on their social position,
relative to what the people around them have and display.
The desire for a particular pair of shoes cannot be separated from
the social meaning of those shoes in a specific time and place and to a particular peer group.
John Kenneth Galbraith pushed the argument further in The Affluent Society.
He called it the “dependence effect”:
the wants that production satisfies are themselves created by the process of production.
Advertising does not serve existing desires:
it manufactures new ones and attaches them to products.
The economy doesn’t exist to satisfy people’s needs—it exists to perpetuate itself
[Galbraith1998].
In South Korea,
cosmetic surgery has grown into a multi-billion-dollar industry
drawing patients from across East and Southeast Asia.
The procedures most in demand—eyelid surgery, jaw reduction, and particular forms of rhinoplasty—track
the standards of appearance disseminated through Korean entertainment products,
which are themselves produced by a commercial industry with strong incentives to generate aspiration.
Digital platforms have scaled this dynamic.
Algorithmic recommendation systems don’t just show you content that matches what you already want.
They build a behavioral model from pauses, clicks, shares, and watch time,
then serve content designed to maximize engagement.
Your preferences at the end of an evening’s scrolling are partly an artifact
of what the algorithm chose to show you.
Recommendation systems are also preference-construction systems.
This is distinct from the nudging described in the previous psychology post.
A nudge changes how a choice is presented without altering the options.
What we’re describing here is shaping what you want before you arrive at a choice.
In India, skin-lightening products generate annual revenues in the billions of dollars,
sustained by advertising that associates lighter skin with success, desirability, and social mobility.
The market for these products—many of which contain harmful ingredients—doesn’t reflect a natural preference.
It reflects decades of advertising, colonial inheritance, and film industry imagery.
Markets can satisfy preferences efficiently.
What they can’t do
(or rather, what the people who profit from them won’t do)
is tell you whether your preferences satisfying are worth having,
who created them,
or why [Packard2007,Veblen1899,Wu2016].
Fads and Bubbles
In February 1637, a single bulb of the Semper Augustus tulip sold in Amsterdam
for the price of a house on a canal.
If you had bought one of those bulbs in November 1636 and sold it in January 1637,
you would have tripled your money.
If you held it until March, you were ruined.
Tulip mania is the most entertaining entry in the catalog of financial bubbles
because it involves flowers.
But the mechanics are identical to what happened in 1720 with South Sea Company shares,
in the 1840s with British railway stocks,
in the 1980s with Japanese real estate,
in 1999 with dot-com IPOs,
in 2006 with Florida condos,
and in 2021 with NFTs.
Japan’s stock and real estate bubble peaked in December 1989—
when the Nikkei index hit 38,915 and Tokyo land was theoretically worth more than all US real estate—
and was followed by thirty years of stagnation
that every mainstream forecast predicted would end within a few years,
which is what “this time it’s different” sounds like from the inside.
The first thing to understand about a bubble is that it is not entirely irrational
while it is happening.
Tulips were genuinely scarce and genuinely fashionable.
Railway technology was genuinely transformative,
and the internet did change commerce.
Every bubble grows from a real phenomenon,
which is precisely what makes it so effective at separating people from their money.
The second thing is the greater fool theory.
You do not have to believe that a tulip bulb is worth a house.
You only have to believe that someone else will pay you more for it next month
than you paid for it today.
This is rational behavior when prices are rising,
but disastrous the moment they stop.
[Mackay1841] documented this pattern in 1841,
which remains readable and depressing because every pattern he identified is still with us.
Isaac Newton, one of the smartest people who ever lived,
lost the equivalent of several million dollars in the South Sea Bubble of 1720.
He reportedly said afterward that he could calculate the motions of heavenly bodies,
but not the madness of men.
He had already sold his shares at a profit,
watched the price continue rising,
bought back in near the peak,
and then watched it collapse.
Robert Shiller, who won a Nobel Prize partly for studying this problem,
argues that markets are driven less by rational calculation than by contagious stories [Shiller2015].
He calls this narrative economics:
when a compelling story spreads through a population,
it changes behavior,
and changed behavior changes prices,
which seems to confirm the story,
so it spreads further.
This feedback loop is difficult to interrupt because
the people caught up in it are not obviously doing anything wrong.
They are listening to their neighbors,
watching prices,
reading the news,
and making bets based on the best available information.
The information just happens to be mostly about what other people are betting.
The British Railway Mania of the 1840s combined technological innovation with genuine investment needs
to produce something close to collective insanity.
Between 1844 and 1846, Parliament approved over 400 new railway projects.
Engineers, lawyers, and promoters collected fees
regardless of whether the lines were ever built.
Landowners extracted fortunes from right-of-way negotiations.
Ordinary investors poured their savings into railways that had no surveyed routes,
no locomotives,
and sometimes no identifiable board of directors.
When the bubble collapsed,
it wiped out a significant fraction of the British middle class,
destroyed several major banks,
and left behind a network of railways that,
once consolidated by the surviving companies,
actually worked.
That last part matters.
Bubbles are destructive for investors,
but sometimes productive for infrastructure.
The dot-com crash of 2000 that wiped out trillions of dollars in market value
left behind fiber-optic cable in the ground,
server farms,
and the engineering knowledge to run large-scale internet services cheaply.
Most of the companies that failed deserved to fail:
their business models assumed that selling pet food online at a loss,
while paying for national television advertising,
would somehow eventually produce a profit.
The crash was the market performing its nominal function of
reallocating capital away from bad ideas.
The problem was that the reallocation happened after
several hundred thousand people lost their jobs and several million lost their retirement savings.
The housing bubble that collapsed in 2008 followed a different structure
because it was built on debt rather than equity [Lewis2010].
Banks lent money to people who could not repay it,
packaged those loans into securities that were sold to pension funds and insurance companies,
bought insurance against those securities from companies that could not pay out,
and collected fees at every stage of the transaction.
The economists who designed the risk models assumed,
based on historical data,
that housing prices never declined nationally at the same time.
They were right until they were catastrophically wrong.
Crucially,
the losses did not fall on the banks that had made the bad loans—the US government covered those.
The losses fell on homeowners who lost their houses and
workers who lost their jobs when the broader economy contracted.
John Kenneth Galbraith noted in his history of the 1929 crash
that the capacity of human beings to ignore evidence
that contradicts a profitable belief is essentially unlimited [Galbraith1954].
The 2008 crisis added the observation that when enough money is at stake,
the strongest defenders of free markets suddenly become socialists in all but name.
Another feature of speculative bubbles is that
they tend to be ahistorical.
The dot-com bubble was the first internet bubble;
there were no previous internet bubbles to learn from.
The mortgage securities market of the 2000s was large enough
that its collapse was qualitatively different from anything that had happened before.
The phrase “this time it’s different” has become shorthand
for the moment just before a collapse,
but it is genuinely true in a narrow sense:
the “why” of a particular collapse is usually novel enough that
people who want to believe can point to real differences from previous disasters [Kindleberger2005].
Which brings us to AI.
Large language models are genuinely impressive.
The companies building them have produced tools that fundamentally change how people write and program.
But the valuations of AI companies have,
since roughly 2023,
been running well ahead of their revenue,
their profit margins,
and any plausible estimate of the addressable market.
The story driving those valuations is that AI will soon be able to do everything,
which is a story that has been told before about nuclear power,
and about previous generations of expert systems and neural networks.
This doesn’t mean that AI companies will collapse to nothing,
any more than railways did.
The real question is who is going capture the value
and who is going to absorb the cost
when reality catches up with the hype.
Historically, the engineers and founders who cashed out early do well.
The pension funds, retail investors, and laid-off workers who arrived later in the story do not,
while the people who collected fees at every stage of the transaction do best of all.
The Myth of Meritocracy
The word “meritocracy” was coined by the British sociologist Michael Young
in a satirical novel in 1958.
Young’s dystopia describes a society that has perfected the measurement of individual talent
and ruthlessly sorts people by it,
producing a ruling class convinced of its own deservingness and contemptuous of those beneath it.
The word was adopted by politicians and executives as a term of praise,
and Young spent the rest of his life pointing out the confusion [Young1958].
How elite selection actually works in practice is very different
from how institutions describe it.
Studies of admissions to elite universities,
hiring at professional services firms,
and promotion within corporations consistently
find that formal credentials matter less than social legibility:
the ability to display the cultural capital
like tastes, manners, and cultural references
that signals membership in the right networks.
Lauren Rivera’s research on hiring at elite firms
found that interviewers routinely described candidates as “polished” or “rough around the edges”
in ways that correlated with class background rather than competence.
The stated meritocratic criteria were real,
but they operated within a prior filter that most candidates never saw [Rivera2015,Karabel2006].
Audit studies make the gap between stated and actual criteria measurable.
Researchers send identical resumes to employers,
varying only the name at the top,
and record callback rates.
The results are consistent across countries and decades:
resumes with names read as white receive significantly more callbacks
than identical resumes with names read as Black or Latino.
The same pattern appears when resumes signal class background or gender
or when disability is disclosed.
Believing in meritocracy does not make people fairer.
Research by Emilio Castilla and Stephen Benard found that
organizations that explicitly adopt merit-based pay principles show larger gender pay gaps
than those that do not.
The mechanism appears to be moral licensing:
people who believe the system is already fair feel less need to monitor their own judgments for bias.
The belief in meritocracy also predicts harsher moral judgments of people who fail.
After all,
if outcomes are deserved,
then failure is evidence of some personal deficiency.
This framing conveniently shifts responsibility from structures to individuals
and makes collective responses to poverty or unemployment
harder to sustain politically [Sandel2020,Wilkinson2011].
The meritocratic ideology takes specific forms in the tech industry.
The puzzle interview,
popularized by Microsoft in the 1990s and still widespread,
selects for people who have prepared for puzzle interviews,
which correlates with having time, networks, and educational backgrounds
that make such preparation possible.
“Culture fit” as a hiring criterion is even less bounded:
it is assessed subjectively and consistently reproduces the demographic composition of existing teams.
Neither criterion is without any signal about likely job performance,
but neither is calibrated against actual job performance data,
because most organizations never close that loop.
India’s IIT and IIM entrance examinations were designed as objective filters,
but a coaching industry whose best programs charge fees equivalent to several years of average household income
ensures that upper-caste and wealthy families dominate admission lists
despite formally caste-neutral scoring—the prior filter that formal meritocratic criteria operate within,
invisible to anyone who sees only the exam results.
The Representation Feedback Loop
In 1967,
at the height of the civil rights movement,
Nichelle Nichols played Lieutenant Uhura on Star Trek.
She decided to quit after the first season,
but Dr. Martin Luther King Jr. persuaded her to stay.
He told her she was not playing a role but representing a future.
Whoopi Goldberg,
having grown up without ever seeing a Black woman on television who was not playing a servant,
ran through her house shouting when she first saw Nichols on screen.
Mae Jemison,
the first Black American woman to travel to space,
has said that watching Uhura was formative for her sense of what was possible.
These stories and millions of others show that representation is not just symbolic:
it changed what futures became thinkable.
Research has shown that men outnumber women roughly three to one in speaking roles in family-rated films,
and that the gap is even larger among characters in science and technology careers.
If every doctor, lawyer, and CEO you ever see looks a particular way,
then looking like them becomes part of your picture of what belonging in those roles requires.
This is not a matter of being naïve or impressionable;
it is just how our minds work.
Breaking the cycle requires intervening somewhere in it,
which is precisely where resistance concentrates.
The NFL’s Rooney Rule,
introduced in 2003,
required teams to interview at least one minority candidate for head coaching positions.
The argument against it was that it was condescending—that qualified candidates should be hired on merit,
not to satisfy a quota.
The argument for it was that the existing process was already not meritocratic:
the informal networks through which coaching positions were filled
were built from relationships among people who looked the same,
so identical qualifications led to different outcomes depending on whose network you were in.
The Rooney Rule did not guarantee hiring,
but even that minimal intervention increased
the number of minority head coaches appointed in the years following its introduction.
Algorithmic recommendation systems reproduce the same dynamic at scale.
YouTube’s algorithm surfaces content similar to what previously earned engagement,
which means creators from underrepresented groups face higher barriers to discovery
than their more visible counterparts.
The algorithm is not making a judgment about quality:
it is applying a preference filter built from historical patterns,
and those patterns reflect who had access to platforms and production tools before the algorithm existed.
This framing is what makes the cycle self-perpetuating.
If you believe that whoever is currently represented in any field
is there because they were the most qualified,
then any change in the composition looks like a lowering of standards rather than a correction of a filter.
The response to evidence of the filter is to explain that the filter must be working correctly,
because look at the results.
The loop closes there unless something breaks it open.
Years ago,
when I first started writing for a programming magazine called Doctor Dobb’s Journal,
I decided to do a piece on the object-oriented features that were being added to MATLAB.
I didn’t know much about them,
so I called The MathWorks,
told the publicity rep who I was,
and asked if they could find me a co-author.
A couple of days later I got a call back from a woman who said she had volunteered to help with the article.
I explained again who I was and what I wanted;
she said she’d be happy to provide me with information and examples.
I then said,
“Thanks, but I’d rather have someone technical.”
After a slight pause, she said,
“Well, I have a master’s degree in Computer Science, and I implemented some of the new features.”
The rest of the conversation was short and uncomfortable.
The next day, I had an email from a different guy saying that he’d be working with me.
I was embarrassed at having put my foot in my mouth;
these days, I’m more embarrassed by how long it took me to wonder how she felt.
Why Discrimination Persists
Becker’s 1957 work on discrimination in labor markets generated an optimistic prediction [Becker1971].
Firms that discriminate pay a cost because
they forgo productive workers that non-discriminating competitors will hire instead.
Over time,
competitive pressure should therefore eliminate discrimination.
History has thoroughly falsified this prediction.
Discrimination has not been competed away in any labor market where economists have looked carefully.
The gap between Becker’s prediction and the observed reality reveals that
rational economics misidentifies what discrimination is and how it works.
Economists distinguish taste-based discrimination,
in which employers simply prefer not to hire members of certain groups regardless of productivity,
from statistical discrimination,
in which employers use group membership as a proxy for characteristics they cannot directly observe.
An employer who uses race or gender as a proxy for something like reliability
is drawing inferences from population-level patterns
that are themselves often the product of prior discrimination,
which creates a vicious circle.
The psychological research on implicit bias
complicates the picture further [Banaji2013].
Studies using the Implicit Association Test and its successors show that most people,
including those who explicitly reject prejudice,
harbor automatic associations between social groups and evaluative attributes.
These associations operate below the conscious level,
and lead to well-intentioned people evaluating resumes differently
depending on the apparent race or gender of the applicant.
Audit studies provide the cleanest evidence.
Researchers send pairs of fictitious resumes to real job postings,
holding everything constant except for names that signal racial identity.
The results are consistent across many countries and many decades:
resumes with names coded as white receive significantly more callbacks
than identical resumes with names coded as Black.
The same pattern appears when resumes signal class background, gender, or disability.
But understanding why discrimination persists also requires recognizing that some workers benefit from it.
Workers who belong to favored groups face less competition for jobs, promotions, and wages
when members of excluded groups are kept out.
This gives them an incentive to back political groups that will maintain discriminatory arrangements,
even if they object adamantly to being accused of discrimination [McGhee2021].
Such groups merely need people to believe that not actively being bad
is the same as being good.
India’s caste system and South Africa’s apartheid
both persisted long after any plausible economic case for them had dissolved.
They persisted because they served social and psychological functions
that economic analysis does not capture:
in particular,
they guaranteed those near the bottom of otherwise low-status hierarchies that
they were not at the very bottom.
If you can convince the lowest white man he’s better than the best colored man,
he won’t notice you’re picking his pocket.
Hell, give him somebody to look down on, and he’ll empty his pockets for you.
– Lyndon B. Johnson
The history of how discrimination gets built into professional structures
is worth understanding before applying it to computing.
The Flexner Report of 1910 recommended consolidating medical education
around a small number of university-based schools modeled on Johns Hopkins.
What is less often noted is what this consolidation closed:
the eclectic and homeopathic schools that had trained a significant proportion of women physicians,
and the historically Black medical colleges that were the primary pathway into medicine for Black practitioners.
The reorganization of medicine around a single credentialing model
was also a reorganization of medicine around a single demographic profile.
The American Medical Association’s subsequent decades of opposition
to women’s participation in medical societies
and to Black physicians’ hospital admitting privileges
extended and consolidated what the Flexner Report had begun.
The legal profession followed the same pattern.
Bar associations across Britain, the United States, and continental Europe
formally prohibited women from practice until courts or legislatures intervened.
In the UK, women were not admitted to the legal profession until 1919.
In the US, the Supreme Court upheld Illinois’s exclusion of women from the bar in 1872,
with a concurrence that stated women’s domestic roles were divinely ordained.
The formal barriers were eventually removed.
The informal barriers persisted:
partnerships that did not hire women,
chambers that did not take women as pupils,
and professional associations that did not see women as natural members.
These required decades more of pressure to dislodge,
and have not been fully dislodged.
The undercounting of women’s scientific contributions is documented across many countries and disciplines.
The pattern includes attribution of women’s work to male collaborators or supervisors,
systematic exclusion from authorship norms that credited the laboratory head
rather than the person who performed the experiments,
and deliberate suppression documented in specific cases
such as Rosalind Franklin’s role in the determination of DNA’s structure.
Studies of citation patterns, grant success rates, and peer review outcomes
continue to find systematic disadvantages for women researchers in many scientific fields,
controlling for measurable differences in output.
Mary Beard’s analysis of women’s exclusion from public speech
draws the pattern to its full historical length [Beard2017].
From the moment in the Odyssey where Telemachus tells Penelope to go back inside
and let men deal with public affairs,
through Roman rhetoric’s treatment of women’s public speech as inherently disgraceful,
through centuries of legal and customary prohibition on women speaking in court and in public assemblies,
the mechanisms of exclusion are remarkably consistent.
They include treating women’s speech as categorically out of place in authoritative contexts,
dismissing or ridiculing women who do speak in those contexts rather than engaging with what they say,
and redefining the terms of authority in response to women’s gains
so that what women achieve is always reclassified as less important than what they have not yet achieved.
In contemporary computing,
the primary mechanism of exclusion operates under the label of culture fit.
Hiring decisions made on the basis of cultural fit are rarely experienced as discriminatory
by those who make them.
The judgment is framed as a question of whether a candidate seems like one of us,
whether they would enjoy working with the existing team,
whether they share the team’s values and habits.
But when the existing team is demographically homogeneous,
hiring for cultural fit reproduces that homogeneity.
The criterion functions as a demographic filter while providing its users with a clean conscience.
The automation of hiring decisions does not solve this problem.
It replicates it at scale.
Algorithmic hiring tools trained on historical hiring data
learn to identify candidates who resemble those previously hired and promoted.
If the historical data reflects discriminatory patterns,
the algorithm encodes those patterns into a system that is harder to scrutinize and challenge
than a human decision-maker.
Amazon’s internal recruiting tool, abandoned in 2018,
penalized resumes that included the word “women’s”
and downgraded graduates of all-women’s colleges,
because it had learned from a decade of prior hiring decisions that were heavily male.
The standard response to demographic imbalance in tech is to point to the pipeline:
not enough women or underrepresented minorities in computer science programs,
and the problem will correct itself when universities produce more qualified candidates.
This explanation is flatly contradicted by the evidence.
Women’s participation in computing was substantially higher in the 1980s than it is today.
The decline in women’s representation occurred
while women’s representation in law, medicine, and other professions was increasing.
The pipeline analogy locates the problem in the wrong place.
Demographic problems that originate in hiring practices, retention conditions, and professional culture
cannot be solved by changing who enters the pipeline.
They require changing the conditions that determine who stays in,
who is promoted,
and who leaves.
The Moral Psychology of Building Harmful Things
In September and October 2021,
the Wall Street Journal published a series of articles
based on tens of thousands of internal Facebook documents provided by Frances Haugen.
Among other things,
the documents showed that Facebook’s own researchers had found that
Instagram worsened body image issues and increased suicidal ideation in teenage girls,
and that the company had known this for years while publicly denying that its platforms caused harm.
The documents also showed that Facebook’s algorithms systematically promoted outrage and divisive content
because it generated more engagement,
and that internal teams had identified this as a serious problem without being able to change it.
None of this is surprising in retrospect.
What is surprising is how ordinary the behavior looks when you read the documents:
working groups, slide decks, and recommendations sent to committees.
The harm was not produced by villains.
It was produced by an organization operating in ways that feel completely familiar.
The psychologist Stanley Milgram ran his obedience experiments at Yale in 1961 and 1962,
partly in response to the trial of Adolf Eichmann,
who had organized the logistics of the Holocaust
and whose defense was that he had simply followed orders [Milgram1974].
In the most famous version of Milgram’s experiments,
subjects were told they were participating in a study of learning
and were instructed by an authority figure to administer electric shocks to another person
whenever that person gave a wrong answer.
The other person was a confederate; the shocks were fake; the screams were recorded.
Sixty-five percent of subjects administered what they had been told was the maximum voltage —
labeled “Danger: Severe Shock” — because a person in a lab coat told them to.
Milgram’s explanation was that people in hierarchical situations shift into an agentic state:
they stop regulating their own behavior against their personal moral standards
and start executing instructions from whoever they perceive as legitimate authority,
locating responsibility upward rather than in themselves.
The psychologist Albert Bandura identified eight mechanisms
by which people disengage their moral standards
to engage in or tolerate harmful behavior without feeling responsible for it.
Moral disengagement does not require cruelty or indifference.
It operates through cognitive reframing available to people who consider themselves good.
Moral justification frames the harm as serving a higher purpose,
while euphemistic labeling describes content moderation failures as “trust and safety challenges.”
Displacement of responsibility points to someone else:
the advertisers demanded this, the users clicked on it.
Diffusion of responsibility distributes it across so many people
that no single one feels it:
the engineers wrote the algorithm, the product managers set the metrics, the executives approved the strategy.
Treating teenage users primarily as engagement statistics
accomplishes much of the same work as dehumanization [Bandura1999,Palazzo2025].
The sociologist Diane Vaughan’s analysis of the Challenger disaster identified a related mechanism
she called normalization of deviance:
the gradual process by which organizations come to accept risk thresholds
that would initially have been unacceptable,
through repeated exposure to near-misses that did not immediately produce catastrophe.
NASA engineers had known for years that O-rings on solid rocket boosters failed in cold temperatures.
Each launch that succeeded despite the problem
was treated as evidence that the problem was manageable.
The night before the Challenger launch,
engineers who understood the risk recommended against launching in cold weather.
Their recommendation was overruled by managers
who reframed the decision as requiring proof of danger rather than proof of safety.
No one decided to kill the seven astronauts who died the next day.
NASA had simply accumulated a set of practices that made the outcome possible [Vaughan1996].
The phrase “we just build hammers” is the most common form of moral disengagement in technology work.
It frames engineers as neutral suppliers of capability,
placing all responsibility for consequences on whoever chooses to use the tools.
This framing collapses when you look at what the tools actually do.
A recommendation algorithm that is trained to maximize engagement
and deployed on a platform used by teenagers
is not neutral the way that a hammer is.
It is a system specifically designed to capture attention,
running on knowledge about how human psychology responds to certain kinds of content,
at a scale that no individual user can resist or comprehend.
The programmers who built it made choices about what to optimize,
and their bosses approved those choices.
None of this means that people inside organizations that cause harm are helpless.
It means that the conditions that produce harmful outcomes are predictable,
that they operate through ordinary human psychology rather than through exceptional malice,
and that understanding them is prerequisite to changing them.
When the metrics that determine whether a career succeeds
are the same metrics that produce harm,
most people will find ways to make peace with the harm [Ehmke2025].