To the Trickster Go the Spoils

William H. Press had been messing around with the Prisoner’s Dilemma, the classic game-theory conundrum, as a sort of side project. That is apparently what you do in your spare time when you’re a computer scientist and computational biologist at the University of Texas at Austin, not to mention president of the American Association for the Advancement of Science. Press wrote a computer program to assist in this happy diversion, but it kept crashing and he couldn’t figure out why.

He was still mulling it when he ran into Freeman J. Dyson at a conference. Dyson, who is 88, is a professor emeritus of physics at the Institute for Advanced Study, in Princeton, N.J., a renowned mathematician, and the author of a number of popular books, including Disturbing the Universe. In 1951 Dyson was given a professorship at Cornell even though he lacked a doctorate. Press was tenured at Harvard while in his twenties. They make quite a pair.

Press explained to Dyson what he’d been up to, and Dyson, intrigued, promised to give it some thought. Dyson has described himself as “quite ignorant” about the details of the dilemma before Press talked to him about it. A couple of days later, Dyson sent Press an e-mail that said in part: “Thank you for this problem, which kept me busy over the weekend.”

Along with the note, there were some formulas. When Press examined them, he was elated. He wrote back that Dyson’s calculations were “truly wonderful.” Dyson hadn’t actually answered the more minor question that first interested Press; instead he had discovered a wildly successful, and apparently new, approach to a well-worn scenario. Says Press: “The joy of collaborating with Freeman is that, once he gives you the answer, you can work out that it’s obvious, but you never could have seen it for yourself.”

That answer is, according to some, “amazing,” “undoubtedly a breakthrough.” Because of it, “the world of game theory is currently on fire.” Or, say others, it’s a more modest accomplishment—important for sure, but not responsible for any infernos. Or maybe it’s not so wonderful after all.

So why does this matter, except to the tiny subset of the population that spends weekends doing math? It matters because the Prisoner’s Dilemma is thought to have a wide range of applications. In biology, for instance, it may help provide models for the evolution of cooperation. It’s also thought to apply to foreign-policy situations like two countries’ acquiring nuclear stockpiles. (If both countries agree to disarm, they can spend their resources elsewhere. Everyone wins. But if one country disarms, and the other doesn’t, the latter country can nuke with impunity.)

First, some background.

One way to get a sense of the Prisoner’s Dilemma is to watch the British game show Golden Balls. In the show’s final round, each of the two contestants is presented with a choice: “split or steal.” If both choose to split, each walks home with half of the prize money. If both choose to steal, each walks away with nothing. If one steals and the other splits, then the stealer walks away with all of the prize money. Neither knows what the other has chosen until after they’ve made their own choices.

The upshot, just as in the classic version involving two prisoners locked in separate cells, is that cooperation pays, but sometimes selfishness can pay even more.

In 1978 and 1979, Robert M. Axelrod, a political scientist at the University of Michigan at Ann Arbor, ran a tournament to discover which strategies were the most effective at the so-called Iterated Prisoner’s Dilemma—that is, playing the game not just once, but dozens or even hundreds of times in row. Is it best never to cooperate? Or always? Can you outsmart your opponent, cooperating at first, then defecting once you’ve established trust? Unlike Golden Balls, there was no prize money or bald, grinning host in Axelrod’s contest. The outcome was determined by a computer program.

Axelrod found that a strategy called “Tit for Tat” usually emerges victorious. If you’re playing Tit for Tat, you always mirror your opponent’s choices. If your opponent cooperated the first time, you cooperate the next. If your opponent doesn’t cooperate, neither do you. If your opponent starts cooperating again, you follow suit. Ever since then, for three-plus decades, it’s been widely accepted that Tit for Tat is the way to go.

What Dyson and Press came up with are several mathematical strategies that, according to a paper they wrote, actually outperform Tit for Tat. The strategy long touted as unbeatable may be more vulnerable than previously thought.

The new strategies, called Zero Determinant, are like a sophisticated coin flip. If the opponent cooperated in the previous turn, the computer runs a formula to decide what to do next. If the opponent didn’t cooperate, it runs a different formula. The program doesn’t pay attention to the history of choices nor does it have any theory of the opponent’s mind. It’s a formula based only on the previous decision. But playing against a dumb opponent—one who is just trying to do the best he or she can—the formulas regularly rise to the top. “I don’t think there’s a deep explanation,” says Press, when asked how the formulas work. “I think it’s a little tweak buried in the math.”

One blogger has developed an online simulator that uses the formulas and allows you to play the role of the dumb opponent. When you’re playing against those strategies, the computer’s choices feel random: Sometimes it cooperates, sometimes it doesn’t. There doesn’t seem to be a pattern. And yet, when I played it, the computer’s score consistently beat mine. It was kind of dispiriting, like being out-evolved.

I asked William Poundstone, who wrote a 1993 book titled Prisoner’s Dilemma that explores the problem’s history and implications, what he thought of the paper. “I was really surprised,” he says. “This challenges how we think about the dilemma.” What it shows, according to Poundstone, is that “if you’re a super-smart player, you can have the upper hand in the game.”

One very general way to look at it is that before, with Tit for Tat, the moral of the story was that cooperation is best for everyone involved. Kindness triumphs. Now it’s to the trickster go the spoils.

Press chuckles at statements like that. People love to search for a human narrative in the cold numbers. Press agrees with a blog post on the n-Category Café by Michael Shulman, a postdoctoral fellow in mathematics at the University of California at San Diego, in which he deems such statements overly simplistic and over the top. That said, Shulman writes that the new strategies are a “discovery of the first order” with “fascinating implications.”

Axelrod is less impressed. The Michigan professor, whose computer tournaments and 1984 book The Evolution of Cooperation are touchstones among those who study the dilemma, says similar approaches (including one with the foreboding title “Grim’s Trigger”) have been tried before but have proved unsuccessful.He writes in an e-mail that Press and Dyson “haven’t really come up with a way to ‘beat’ Tit for Tat.” But Press seems to differ: “We’re about 25 years too late to the party,” he says, referring to Axelrod’s tournament, “but perhaps if we had been there, we would have won.”

Axelrod’s criticism is an outlier among the accolades, though the merits of the new strategies are likely to be discussed for years to come, just as Axelrod’s tournament results have been pored over since the Carter administration. Which goes to show that even a problem that’s been studied to death may yet have some life left in it.