The Chronicle Review

How to Tame an Internet Troll

Geoffrey Moss for The Chronicle Review

September 28, 2015

In James Thurber’s 1942 short story "The Catbird Seat," the boisterous Ulgine Barrows shatters the peaceful diligence of Erwin Martin, head of the filing department at his firm. Barrows, as special adviser to the president of the firm, brays at Martin, using slang he barely understands. Apparently a Dodgers fan, she mimics Red Barber’s ball-field patter, shouting, "Are you tearing up the pea patch? Are you scraping around the bottom of the pickle barrel? Are you sitting in the catbird seat?" Martin eventually hatches a plan to turn Barrows’s ebullience against her, using words alone. He tells her he’s plotting to kill the president of the firm — suspecting their boss will find the very idea of homicidal impulses in a veteran employee outlandish if Barrows relates it to him. Generations of high schoolers have delighted in her unraveling, the climax of a revenge plot most beleaguered office workers can merely dream of.

The fantasy of using words alone to right a perceived injustice — or trigger a meltdown — has renewed relevance today. On Reddit, 4chan, Yik Yak, and Tumblr, whole Internet subcultures are devoted to mocking and shocking. Enraging an exploitable target is many Internet trolls’ raison d’être. In staider settings, moderators aspire to some digital equivalent of the Marquess of Queensberry rules, carefully policing the commenters for relevance and civility. Ranking and rating is simultaneously a personal passion and big business. Five-star tributes to books on Amazon express appreciation for hard work and refine the company’s algorithms, adding a few pennies to Jeff Bezos’ fortune.

If we could easily escape commenting culture, we might accept its pathologies as one more cost of free speech. But we cannot, and its ubiquity demands a more active response. Trolls and commenters can harm their victims, crossing the line between speech and conduct. Just ask Kathy Sierra, whose promising career as a technologist came to an abrupt end after she faced waves of online abuse. When anonymous death threats proliferated in 2007, Sierra withdrew almost entirely from public venues. Gender and race matter too: As my colleague Danielle Keats Citron’s Hate Crimes in Cyberspace recounts, women and people of color are routinely targeted by hatred online. Such biases are also evident in student evaluations, an old and familiar form of comment culture. But those reviews still have the potential to make or break faculty appointments and promotions.

Boosters of Internet culture tend to characterize the growing power of the "crowd in the cloud" as simultaneously meritocratic and egalitarian. They Whiggishly embrace the rise of a networked commentariat. From Yelp to Uber to TripAdvisor, fortunes are made via crowdsourced contributions that supposedly reflect the "wisdom of crowds." But any massive platform also hosts some cleverly manipulated commentary planted by competitors, troll factories, or miscreants. The sweet business strategy of hosting free, user-generated content is always accompanied by bitter, oft-stealthy battles for reputational capital.

Life online is seen as a chance to "wind up" the anxious, dupe the powerful, trash the earnest.

Two just-published books offer important insights into these problems of Internet culture — and its potential for redemption. In Reading the Comments (MIT Press), Joseph M. Reagle Jr., an assistant professor of communications at Northeastern, offers a rollicking-yet-thoughtful tour through comment threads across the web — from book reviews to Facebook spats and from commercial contexts to intimate spaces of self-expression. Amply spiced with jokes and comics, and anchored with just enough theory to structure the discussion, Reagle’s book should be read by anyone with an interest in "the bottom half of the web."

In This Is Why We Can’t Have Nice Things (MIT Press), Whitney Phillips, a lecturer in the department of communications at Humboldt State, observes a narrower slice of the Internet: the trolls who mock, harass, or menace online. But she is after larger academic game: a generalizable method for studying protean networks online. Trolls’ anonymity may seem to make them an impossible object of analysis — who really knows if BigJerk34, lulzhead3, and MOONWALKinyourFACE are distinct persons, bots, or "sock puppets" of one master troll? Phillips takes those dilemmas seriously, opening lines of communication to trolls, but also intentionally focusing on the cultural effects (rather than psychological roots) of trollish behavior. She embeds her description and evaluation of trolls’ disruptive antics in a larger cultural critique.

A subculture renowned for ungrammatical speech and unserious arguments may seem like an unpromising topic. But the Encyclopedia Dramatica, Anonymous, and their many memetic mutations are not isolated petri dishes of alienated malcontents — rather, they reflect larger trends. When local news adopts "if it bleeds, it leads" as a ratings-driven mantra, how can it complain about getting trolled into covering "jenkem," a fake teen drug craze contrived by trolls? Online and offline cultures are now hybrids, mutually influencing one another.

Moreover, as the powerful find clever ways to manipulate algorithmic assessments of popularity and authority, commercial hierarchies are re-emerging online, as concentrated as ever. For many book publishers, Amazon is the critical intermediary between author and audience. Amazon paints its reviewing service as empowering a community of readers to help its managers decide which books to boost and which to consign to oblivion. But the judgment of the crowd can be plenty arbitrary.

Of course, Amazon’s laissez-faire approach is not the only mode of commenting. Sites create diverse conditions for audience interaction. Just as rival electoral rules (such as "first past the post" versus "proportional representation") can lead to different political systems, so too can commenting rules lead to vastly different cultures. Complete anonymity can foster the out-of-control, zany atmosphere of boisterous message boards. But it also encourages disgusting, degrading, or illegal material. Comments tied to real names tend to be more responsible, but also blander. And there are myriad in-between rules, ranging from persistent pseudonyms to barely verified registrations.

A coldly nihilistic politics is a comfy ideological home for self-pleased solipsism.

Comments at the bottom of the web have the power to subvert or overwhelm the ostensible "main event," overthrowing old entitlements to unmerited publicity. For many of my friends, the only entertainment yet another Tom Friedman or David Brooks column offers is the myriad, mocking parodies it provokes. Quotidian absurdities at Fox News also provide ample fodder for the meme machine. For example, when the anchor Megyn Kelly dismissed the complaints of pepper-sprayed protesters by saying, "It’s like a derivative of actual pepper. It’s a food product, essentially," meme-meisters went to town. "Water boarding? It’s a shower, essentially"; "Skin cancer? It’s a tan, essentially"; and dozens of other variations emerged, all emblazoned as captions of Kelly extending her peerless command of analogical reasoning.

So are comments and memes the new "power of the powerless," the 21st-century equivalent of Havel’s greengrocer defying the regime? Both Phillips and Reagle show the ability of comments to humble the hubristic with well-deserved shamings. But the nasty, sexist, and racist aspects of trolling are rarely far behind the Swiftian sting of digital protest. For example, Know Your Meme documented at least one sexist meme about Megyn Kelly in the wake of her online comeuppance; surely there are more.

Phillips rigorously defines the many dimensions of trolling. She roots her account in online activities that are motivated by what trolls call "lulz" — "unsympathetic, ambiguous laughter … similar to Schadenfreude." While she discusses diverse phenomena in the book — the fake teen drug craze mentioned above; a successful effort to provoke Oprah Winfrey to say the phrase "9,000 penises" on the air; "copy­pasta" of nonsense or demeaning text in comments or message boards — by the end of the book one sees the family resemblance of trolls’ varied methods.

The troll engages "exploitable" targets via "detached, lulzy play" and humor "predicated on mental dexterity and emotional distance." There’s no hope of sustained engagement with issues or communicative consensus. Rather, life online is seen as a chance to "wind up" the anxious, dupe the powerful, trash the earnest, alienate the already marginal. There doesn’t need to be a larger aim. As one particularly apt quote from a veritable troll bible, Encyclopedia Dramatica, puts it: "Lulz is engaged in by Internet users who have witnessed one major economic/environmental/political disaster too many, and who thus view a state of voluntary, gleeful sociopathy over the world’s current apocalyptic state as superior to being continually emo."

Of course, as with all trolling, we’re probably not meant to take that statement too seriously: Trolls are apt to deny the need for any justification of their behavior. But a coldly nihilistic politics is a comfy ideological home for self-pleased solipsism.

Had Phillips stopped her analysis there, it would already be a substantial contribution to media studies: a methodologically meaty treatment of an increasingly popular mode of communication online. But she pushes further, rooting the psychology of trollery in a broader study of the culture in which it emerged: the early years of the George W. Bush presidency, when, as she puts it, isolated, "proto-­trollish energies" coalesced into a force on the Internet.

"This place and this time matters," she argues, because "Americans were unmoored — and were encouraged by the mainstream media and the Bush administration to remain unmoored — from history, from war, from the suffering of others, from the suffering of fellow citizens." As "the president made jokes about looking for weapons of mass destruction under his Oval Office desk," and Donald Rumsfeld’s cryptic aphorisms substituted for serious analysis of military strategy, trolls flourished. As media froth crowded out substance, opportunists hijacked comment sections with scatological jokes and ad hominem attacks. The comment field alone didn’t foreordain the pathologies of trolling. Unserious and reckless media figures became role models for those mocking them. Like the angry child in the PSA asked by his dad how he learned to use drugs, trolls could easily say to an oft-accusatory mass media: "You, all right! I learned it by watching you!"

Those developments should be a cautionary tale for anyone who foresees a utopian future for education in crowdsourced content and commentary. Imagine, for instance, a world where all student and professor evaluations were put online, forever, and subject to endless summary and recalculation — a true 360-degree review.

That may be demanded in the future by employers with an unslakeable thirst for ever bigger data about potential hires — just look at the emerging arms race of endorsements on Linked­In. And we already know some of the downsides of this reputational economy. Professors assign less work to get higher evaluations. Students are scared away from ambitious topics or controversial interjections in class.

Without careful structuring, a world of universal ranking degenerates into the bland leading the blind, laundered through online systems that purport only to report on the will of the people.

The TV series Community (set in a hapless postsecondary institution) gave a zanily compressed parody of that future in an episode featuring a made-up app called MeowMeowBeenz. Students, professors, and administrators could simply aim their smartphones at one another to rate friends or enemies with 1 to 5 "meowmeowbeenz." (Cat faces replaced stars on the app — and might be a good stand-in for U.S. News & World Report’s peer-reputation scores in its notorious rankings.) But only a few realized at the outset the key rule of the app: Anyone with a 5 rating had exponentially more power to rate others. A ruling class of 5’s quickly emerged, while plebes knocked one another down to preserve their own relative status.

Was the episode made as a cautionary tale? Or was it simply another crazy spectacle? In either case, the humor hit home because it dramatized the sense of powerlessness we have before arbitrary arbiters of digital reputation.

Reagle documents the baleful effects of the resulting free-for-all, exploring the "racket of reputation management." On platforms like Yelp, "some reviews are shown by default and included in the overall star rating while others are shunted" to obscurity — but the algorithms (or human actions) behind such determinations are opaque. As Reagle observes, "Merchants cry foul when they decline to advertise on Yelp and a negative review by a Yelp employee soon follows." A similar dynamic was uncovered at the Chinese search engine Baidu years ago, when employees downranked companies impertinent enough to decline an invitation to advertise.

Reagle also discusses Facebook "like farms," which can inflate the apparent popularity of a person or place with a storm of automated approbation. The tactics and countertactics of manipulators and platforms may seem like an esoteric concern, but remember that the economy of online reputation is zero-­sum: Space on the newsfeed is limited, and likes may be a critical signal to increase one post’s salience and decrease another’s. Platforms like to paint themselves as heroic battlers of spam likes and fake links, but Reagle shows how their extreme opacity leaves ample room for self-serving approaches to the quasi-judicial role they now play.

Reagle is also insightful (and funny) as he discusses the many failures of compressed meaning inherent in any "5-star" rating standard. One man’s 2 is another man’s 5. A review of a carbon-­monoxide alarm on a housewares site had the headline "Saved our son’s life-4/5 stars." (The same reviewer gave a nutritional scale 5/5 — perhaps it saved the whole family?)

The more commensurable the rating, the more it tends to flatten complex judgments into blunt calculations. And while platforms try to convince us that their masses of up- or down-votes, likes or faves, instantly report the zeitgeist, savvy media scholars know the "returns" from those cultural voting booths are endlessly manipulable. If you thought black-box voting at real ballot boxes was a problem, just try to understand exactly how Google or Amazon distinguishes between fake and real comments and links.

In a perfect world, Internet comments could accelerate science and democracy, commerce and charity. The endless space after articles would overflow with the collective wisdom of crowds — expertly moderated by fair-minded community-builders. Blogs would prefigure a radically democratic space of mutual dialogue — or at least a high-tech version of the FCC’s old "fairness doctrine," offering one and all a chance to correct, clarify, or contradict the author’s line. Civility would reign as the "unforced force of the better argument" nudged authors and readers alike toward an ideal speech situation, unachievable in older, more unidirectional media.

Where authors hope for good-faith engagement, nit-picking and nastiness too often reign.

That dream lives on in a few blessed blogs and well-curated comments sections, as Reagle recounts toward the end of his book. With enough effort, expertise, and good faith, a comments section can showcase the worthwhile, efface the worthless, and downrank the dubious. But in mass media and mega-platforms — where most of the action is — comments sections are all too often cybercesspools of trashing and trolling, obsessive annotators, and regressive instigators. The digital grass roots may simply have been "astroturfed" by a clever marketer, as David Golumbia’s work on "commercial trolling" has shown. Where authors hope for good-faith engagement, nit-picking and nastiness too often reign.

In light of work like Phillips’s and Reagle’s, it’s hard to take an uncomplicated pleasure in the unraveling of Ulgine Barrows nowadays. Thurber’s story could be that of one more ambitious woman driven out of the workplace by jealous or condescending men. Just as aspects of work dematerialized into online public spaces, misogyny followed. A modern-day Erwin Martin might have targeted his work rival’s Facebook account, sent anonymous hate messages, or rallied some men’s rights-ers to dox her Tinder pictures.

The worst aspects of trolling culture fester in platforms grown too large to moderate content properly; similarly, fake or manipulated reviews are easier to slip by entirely automated platforms than ones influenced more directly by human judgment. Skimping on labor also allows big platforms to pay for high-priced legal and lobbying help to assure they remain largely unregulated.

In a Wild West world of unaccountable platforms, we can only expect the problems Reagle documents to grow. He wisely "fears that the future of online comment will continue to move toward large commercial platforms in which people have little privacy and see mainly the posts of the like-­minded, the popular, and those who pay to reach us — a neutered filter bubble that serves advertisers rather than users."

But he also offers worthy precedents for improving online culture, such as a 2013 action by New York State’s attorney general to crack down on fake reviews. Europe’s recent "right to be forgotten" decision is also a good step toward correcting the balance of power between web giants and their users.

The one thing that probably won’t do much to improve online life, however, is competition. We are all familiar with network effects: The rich get richer online (and Google’s fundamentally citation-­modeled ordering of web pages results in a "Matthew effect" as profound as the one Robert Merton discerned in the academy almost 50 years ago).

Here Phillips’s realism is refreshing: She roots trolling in "the mainstream world — its behavioral mores, its corporate institutions, its political structures and leaders" — and predicts little can change for the better in digital spaces as long as there is so much cruelty and capriciousness in the larger culture.

No magical technology will suddenly civilize the web. That’s a task for professionals — who, one hopes, will someday be paid properly for the labor of building and maintaining community online. Reagle aptly quotes Xeni Jardin, a leading authority on tech culture, who believes "online communities rot without daily tending by human hands."

The original sin of Internet culture was the exploitation of user-generated content to enrich a lucky few at the top of dominant platforms. Spreading that wealth — toward the moderators, watchdogs, and auditors who could detect and respond to the problems Reagle and Phillips so expertly illuminate — would be a good first step toward taming trolls and shaming sock-­puppeteers and astroturfers. Where "spontaneous order" failed, good old-fashioned editorial judgment may yet succeed.

Frank Pasquale is a professor at the University of Maryland’s Francis King Carey School of Law and author of  The Black Box Society: The Secret Algorithms That Control Money and Information (Harvard University Press).