Big Tech is Like Scientology

In the 1950s, L. Ron Hubbard developed a practice called auditing. In a standard session, a trained Scientology auditor asks the subject a series of questions while the subject holds the electrodes of an E-meter that measures galvanic skin response (the same physiological signal used in polygraph tests). The questions are designed to surface traumatic memories, which Scientology calls engrams, so they can be discharged through conscious recall.

The sessions are recorded, and the records are kept in what Scientology calls “preclear folders.” They contain whatever the subject disclosed during auditing: accounts of illegal activity, sexual behavior, family conflicts, financial difficulties, and statements about other people. The Church of Scientology denies that folders are used punitively, but former members have testified that these folders were used in disciplinary proceedings and in litigation against critics and defectors.

The analogy to big tech is not subtle. Every major social media platform is, at its core, an auditing system. It collects behavioral data—what you look at, what you hesitate over, what you react to—and that information is qualitatively different from what you share with a retailer. People post about illness and grief and their political beliefs and sexual identity because the platform presents itself as a community, not a database. The fact that it is both doesn’t mean the user is naïve; it means the platform is designed to exploit the social context that makes sharing this information feel appropriate.

Scientology’s critics have documented a practice the Church calls “Fair Game”, under which people who leave the organization and speak critically about it (known as a “suppressive person”) can be “deprived of property or injured by any means by any Scientologist without any discipline of the Scientologist.” The Church claims this policy was cancelled in 1968, but its critics have documented its continuation under different names. The pattern has included litigation designed to exhaust defendants financially, harassment campaigns targeting employers and family members, and the use of auditing records in legal proceedings.

Tech companies have not employed anything comparable in severity (that we know of). They have, however, used legal and institutional power to manage criticism in ways that Scientologist would recognize. Facebook commissioned audits of third-party researchers who published findings the company disputed. Google funded academic research in ways that created conflicts of interest for academics who might otherwise study the company critically. Uber deployed a team it internally called COIN (for Competitive Intelligence) to gather information on regulators, journalists, and competitors. The distinction between these practices and Fair Game seem pretty slim to the researchers, journalists, and regulators on the receiving end.

Scientology’s governing doctrine holds that the organization’s critics are necessarily criminals. If someone attacks Scientology, Hubbard’s writings state, one need only look at their past to find the crimes they are hiding. The logic is airtight because it is circular: criticism itself is taken as evidence of wrongdoing.

This is a specific and pathological version of a general tendency. When researchers publish findings critical of Facebook’s recommendation algorithms, Facebook’s communications team responds not only with factual rebuttals but with questions about the researchers’ methodology, funding sources, and motivations. When journalists publish stories based on leaked documents, companies issue statements about documents being “taken out of context” and about reporters’ prior relationships with the company.

Scientology is structured so that participation becomes progressively more expensive. New members begin with free or low-cost introductory materials. Progression up the “Bridge to Total Freedom” requires increasingly expensive courses and auditing sessions. Former members have documented spending hundreds of thousands of dollars over years of participation. The social world of Scientology reinforces continued involvement: friends, family, and community ties are largely internal to the organization, which means that leaving means losing them.

The structural lock-in that platforms engineer follows the same logic. A photographer who has spent years building an audience on Instagram is not free to leave without abandoning what they have built. A developer who has built a business on the iOS App Store faces the same kind of switching cost.

The Church of Scientology has survived decades of hostile press, regulatory action across multiple continents, and prominent defections. It has done so by treating litigation as a cost of doing business, and by providing genuine community to members. The question now is whether the mechanisms that have gradually constrained Scientology will operate at the scale of companies whose products are used by billions of people who have no obvious alternative.

see the whole series · email me

Wright2013

Lawrence Wright: Going Clear: Scientology, Hollywood, and the Prison of Belief. Knopf, 2013, 9780307700667.