In July, the former federal prosecutor David Hoffman released a report on allegations that the American Psychological Association had colluded with the Department of Defense to change the APA’s ethics code, giving psychologists cover to participate in torturous interrogations.
Hoffman had been commissioned by the association itself, following allegations by the New York Times reporter James Risen. When Risen made the assertions, in October 2014, the APA put out a press release denying wrongdoing and, in effect, calling Risen a hack. But the Hoffman report substantiated Risen’s contentions.
The APA created ethics tasks forces composed of members who had interests that would incline them to back the military’s interrogation practices. Critics of the association’s policy were not consulted. The APA appears to have crafted a corrupted “consensus” by excluding those who might disagree.
This case is a particularly disturbing example of a problem throughout the social sciences: the crafting of false consensus statements to promote ideological or political goals. False consensus does great, sometimes irreparable, damage to science.
Consider another instance: a group of 20 international scholars were convened in 1986, under the auspices of Unesco, to release a proclamation challenging what they described as “misuse of scientific theories and data to justify violence and war.” “IT IS SCIENTIFICALLY INCORRECT to say that war or any other violent behaviour is genetically programmed into our human nature,” the Seville Statement reads, complete with that all-capitals confidence. The signatories allow that genes play a role in behavior, but they disavow a role for genetics in creating a predisposition toward violence. The Seville Statement was influential and was endorsed by numerous scientific organizations, including the APA in 1987, and is typical of consensus statements in social science.
Rallying toward consensus usually reflects not the strength of an argument but its weakness.
Political positions are represented as certainties through the language of scientific truthiness. Only they are usually dead wrong, the Seville Statement no exception. In the decades since its release, a wealth of scientific information has emerged linking genetics to violence. As a 2008 article in the journal Nature noted of the statement, “The decades since have not been kind to these cherished beliefs.”
It’s odd that consensus is so sought out, given that its very definition is wobbly. Dictionary.com defines it as “majority of opinion” and “general agreement or concord.” But it’s not clear what percentage of agreement you need before a consensus can be declared: 51 percent? 90 percent? 99 percent?
In the social sciences, consensus has usually been declared despite continuing debate among scholars. In many cases, consensus statements may have less to do with highlighting true agreement than with bullying recalcitrant scholars into line with an ideological dogma valuable to professional advocacy organizations.
Who cares if one has a consensus, anyway? Argument by consensus is a well-known logical fallacy, so it’s remarkable to see scientists who should know better striving for and boasting about it. Within social science, as highlighted by the Seville Statement, what scholars agree on as consensus opinions one year are often on the scientific trash heap the next.
Rallying toward consensus, I’d argue, usually reflects not the strength of an argument but its weakness. Claims of consensus can function as a deliberate effort to stifle debate, belittle skepticism, and distract from inconsistencies and inadequacies in data.
Such claims may be more common when a research field is politically loaded and inconsistencies in data are inconvenient. After all, I don’t think physicists feel compelled to release a consensus statement that gravity works. The faux-consensus phenomenon is abetted by professional advocacy groups like the American Psychological Association (of which, for the record, I am a fellow). These organizations, while appearing scientific, are motivated to benefit their members. The benefits include identifying “truths” about social problems that their members can fix, and securing grant support, political influence, and prestige for the profession. Stating that the evidence for a particular belief is inconsistent, and that it’s difficult to say anything definitive, isn’t helpful for that kind of professional advocacy.
Undoubtedly the appeal of consensus statements has been propelled by influential consensus among scholars in other areas, primarily among climate scientists on global warming. Surveys of such scholars suggest that easily upward of 90 percent agree climate change is happening, with human behavior contributing. Of course, the strength of climate science is based on the quality of the data. Nonetheless, the consensus has had obvious political impact and has propelled climate science to the front of public consciousness.
My own area of research, video games and society, is often contentious but has seen its own recent rash of consensus statements. Almost all kids (97 percent, according to a 2008 study) and young adults play video games, and action games such as Grand Theft Auto 5 have been setting records for entertainment-media earnings. The impact of these games on public health has been arguably nonexistent, conceivably even beneficial. For instance, youth violence has dropped by more than 80 percent during the video-game era. But claims about the games’ effects, whether for good or bad, continue to have political appeal.
As the scholars James Ivory and Malte Elson wrote in 2013, video-game research has always been a contentious mess, with inconsistent results and the poor quality of studies making clear declarations difficult. Even so, over the past year there was a burst of consensus statements. In each case, the statement was immediately followed by criticism that the claim was premature or unsupported by data.
The most recent of these regarded brain games, cognitively challenging and touted as having the potential to make players smarter and to prevent dementia in the elderly. In October 2014, a group of scholars led by the Stanford Center on Longevity released “A Consensus on the Brain Training Industry From the Scientific Community.” I’m sympathetic to much of the statement, as it takes the industry to task for exaggerating benefits of such games. (There’s little evidence that they really make you smarter.) But how much of a consensus did the statement have? Not much, it turns out. Soon after, more than 130 scholars wrote an open letter that, while agreeing with some parts of the statement, took the group to task for not covering a full range of research studies and for prematurely claiming a consensus on an issue still under debate.
Argument by consensus is a well-known logical fallacy, so it’s remarkable to see scientists striving for it.
Video-game addiction is a similar topic. For decades, researchers have contested how to define it, assess it, and estimate its prevalence. Yet in 2014, 14 scholars declared an “international consensus” on assessment of the disorder. This declaration left out the vast majority of scholars in the field, including those at important laboratories like the International Gaming Research Unit, at Nottingham Trent University.
In October 2014, several scholars used a survey to claim a consensus among scholars on video-game and other media violence, the area in which I do most of my research. However, as noted in The Guardian, the report itself documents considerable disagreement. Further, the study got an unusual post-peer-review mulligan to change the operating definition of “consensus” once the initial analyses were found to be flawed. In fact, only 58 percent of media scholars agreed that media violence (66 percent for video games specifically) causes aggressive behavior (left undefined). On the more pertinent issue of media violence causing societal violence, the rate of agreement was only 35.2 percent.
Most other surveys have produced even lower rates of agreement. A report presented at the 2013 International Communication Association conference found that only 10.1 percent of media scholars agreed that video games cause aggression in society. John Colwell, of the University of Westminster, presented research in July at the European Congress of Psychology finding that only about 15 percent of criminologists, psychologists, and media scholars agree that violent video games are a cause of youth assaults. Research that I conducted on clinicians who work with youths and families found agreement about that claim to be at 39.5 percent. Older clinicians and those who held negative views of youth were more likely to believe that video games spur assaultive behavior.
Agreement in surveys can depend both on the wording of statements and on the sampling strategies, and evidence for a consensus in this case remains thin at best. Indeed, particularly on the key issue of violent media’s causing violent behavior, results suggest that this is a minority view among scholars.
Here, too, we can see the problematic influence of professional advocacy organizations, which may promote false consensus statements for their own perceived gain, just as with the APA-Defense Department debacle. In 2005 the psychology association released a policy statement on video-game violence that has been criticized for assigning a task force of scholars to review their own research and declare it beyond further debate. The APA’s response to the criticism was to assign a second task force to review the issue again. The majority of the members of the second task force had also taken public positions against games, although only one had a direct history of games research. This situation provoked 238 scholars, including myself, to write an open letter to the APA asking it to retire its policy statements on video games and other media. Two years later, neither the APA nor its task force has made any effort to engage this large group of scholars or learn of their concerns.
The consensus craze has not yet run its course. An October 2015 conference on media is being organized under the auspices of the National Academy of Sciences. Among its purposes is to “develop a consensus statement on what is a healthy media diet and guidelines for parents on how to raise happy, healthy children.” However, a look at the panelists reveals that particularly on issues like video-game violence, only scholars who endorse the more extreme views of harmful effects are represented. This highlights the trend of developing consensus through the intentional exclusion of those who might be expected to disagree.
Consensus statements may serve a legitimate function when a real consensus exists, insofar as it can draw attention to an important issue. But repeated statements of policy or consensus seem to serve primarily as attention getters, attempts to make a profession seem important. In pursuit of that goal, discrepant data or opinions are inconvenient.
Major professional advocacy organizations such as the APA and the American Academy of Pediatrics have dozens of policy and consensus statements. Some might reflect actual consensus, since the positions taken, such as a 2008 APA resolution against genocide, are so obvious that one would have to be a psychopath to disagree. Others, like a series of APA statements on abortion declaring “that termination of pregnancy be considered a civil right of the pregnant woman” appear to be an intrusion of social science into moral declarations. One might disagree or agree with the consensus statement, but what basis science has for weighing in on such a matter is unclear.
When the American Academy of Pediatrics issues a consensus statement that speaks to behavioral issues like the effects of spanking, it runs the risk of sacrificing scientific objectivity and precision for politicized policy positioning. It may be time to view such organizations as more akin to guilds, looking for influence and power, than to neutral and objective purveyors of scientific information.
We need to ask ourselves if science is something we want handed to us by diktat instead of inquiry; if we want skepticism toward and falsification of assertion repudiated rather than encouraged; and if we want dialogue or conformity. There’s nothing wrong with scholars’ stating their opinions on a topic, even as a group. But to claim a consensus when there is none isn’t science. It’s pseudoscience — and an aggressive, mean-spirited one at that.
