In an innovative study that pushed the boundaries of research ethics, scholars posed online last fall as potential applicants to graduate programs to determine if graduate-study directors would betray ideological or political bias in responding to their e-mail.
The researchers involved in the experiment planned to make the ruse known this past weekend to the more than 250 graduate-study directors they had tricked. If the directors object to having been deceived, they can at least take solace in knowing that no significant bias was detected in their answers to queries from fictitious students whose e-mails mentioned previously working for either the Obama or the McCain presidential campaigns.
A paper on the study’s findings was released on Monday, along with a paper on a new, related study examining the role Americans’ political leanings play in their decisions to seek advanced degrees. The studies’ authors argue that their results affirm other research that the liberal leanings of the nation’s professoriate cannot be blamed on indoctrination or efforts to screen conservatives out of the academic pipeline. Instead, they said, liberals’ greater interest in advanced degrees and in academic careers appears to be the chief factor accounting for why college faculties tend to be more liberal than society as whole.
“There are just many more liberals than conservatives in the ranks of graduate students,” said Neil L. Gross, an associate professor of sociology at the University of British Columbia and co-author of both of the unpublished studies, in an interview last week. The relative paucity of conservatives in the professoriate “does not seem to be the result of bias or discrimination against them,” he said, but instead mainly appears to be the result of self-selection among those who have the option of seeking such careers.
Neither of the papers rules out the existence of political or ideological bias in how college faculty members interact with students. Moreover, the study that deceived graduate-study directors makes clear that students actually might experience such bias once they apply to or enroll in graduate programs, as well as in other stages of their academic careers.
But, in the paper summarizing the phony e-mail experiment, the authors argue, “if political bias toward graduate students were robust in the fields we studied, our methodology would very likely have detected it.”
Missing Something?
April Kelly-Woessner, an associate professor of political science at Elizabethtown College and co-author, with her spouse, of several other studies on the political leanings of academics, said neither of the new papers should be interpreted as saying conservatives face no bias on college campuses. “If discrimination exists against conservatives in the academy, it is going to take subtle forms, and the people guilty of it aren’t even going to recognize that they are doing it,” she said.
David Horowitz, founder of Students for Academic Freedom and a prominent critic of perceived ideological bias among college faculty members, was harsher in his critique of the e-mail experiment, arguing that it was “flawed at the outset.” The experiment focused on how college administrators handle an activity in which they are likely to try to hide any political bias they might have—much, he said, as he believes professors who abuse bright conservative students in class go on to give them good grades to demonstrate their own fairness. And, he argued, the experiment had failed to look at how liberal or leftist assumptions are built into many academic fields—such as women’s-studies programs that teach that gender is a social construct—and how students who reject such ideas are unlikely to be allowed to advance.
“The methodology here obscures the obvious,” Mr. Horowitz said. “There is no explanation for the overwhelming percentage of liberals—let alone leftists—in academia other than bias in the hiring process.”
Matthew C. Woessner, Ms. Kelly-Woessner’s husband and an associate professor of public policy at Pennsylvania State University, said, “It is impossible, in one study, to determine definitely whether conservatives are the victims of discrimination in academia.” But, he said, the studies released on Monday “do a fine job of chipping away at the larger question.”
Meet ‘Jeff’ and ‘Kevin’
Mr. Gross and Ethan A. Fosse, a doctoral student in sociology at Harvard University and a co-author of both new papers, previously conducted research concluding that liberals are drawn to certain academic careers because those careers are “typecast” as the province of people with liberal views, just as some men are discouraged from going into nursing because it is stereotyped as a career for women.
The Woessners have produced similar research arguing that liberals are disproportionately likely to possess, and conservatives to lack, the sorts of values and interests that predispose people toward academic careers, such as a willingness to postpone marriage, a relative indifference toward financial success, and a comparatively strong interest in writing original works. Other scholars have argued that the liberalism of many professors can be tied to class dynamics, their tendency to live in cities, or their relative lack of religiosity.
Most of the research done on the issue so far has been based on analyses of survey data, rather than any sort of experimentation.
The experiment involving e-mails to graduate directors was led by Mr. Gross, the editor of Sociological Theory, a journal of the American Sociological Association. In conducting it, he was joined by Mr. Fosse and by Joseph Ma, an undergraduate at the University of British Columbia.
The three are hardly the first scholars to have disguised their identities to extract information from research subjects, although the practice remains controversial. A twist they used was applying an “audit” approach often employed by researchers or civil-rights officials trying to find evidence of discrimination against certain applicants for employment or housing. Typically, the technique involves giving two people false profiles that leave them similar in every way except for the trait, such as race, for which it is suspected they might face bias, and approaching the subjects of the investigation to see if they are treated much differently.
The researchers’ paper says they designed their study based on the tendency of prospective graduate-school students, especially in the social sciences and humanities, to write e-mails to directors of graduate study to introduce themselves and determine if they would be good fits for the programs. Last fall they sent two e-mails, three weeks apart, to directors of graduate study in well-regarded departments of sociology, political science, economics, history, and English—fields chosen because of their reputations as centers of the controversy over alleged political bias in academe. Each of the e-mails came from one of two fictitious college seniors majoring in the field in question: “Kevin Cook,” purportedly of the University of California at Irvine, and “Jeff Allen,” purportedly of the University of California at Santa Barbara.
The e-mails were each about 200 words long and expressed interest in academic specialties that are well established and do not have distinct ideological reputations. Both fictitious students were closely matched in their expressed interests and academic backgrounds. Their e-mails matched in all but one respect: While one made no mention of politics, the other said either “When I was a sophomore, I also spent a few intense months working for the Obama campaign, which was quite a learning experience,” or the same thing in reference to the McCain campaign. (The researchers did not send both Obama- and McCain-oriented e-mails to the same graduate-study directors, out of concern that the receipt of three similar e-mails would arouse suspicion.)
In looking out for evidence of political discrimination, the researchers compared how likely the directors were to respond to the various types of e-mails, and how quickly. They also had three recent college graduates in the social sciences read each response from a director and rate it in terms of the amount of information it provided, its emotional warmth, and the level of enthusiasm shown toward the applicant. The raters were kept in the dark about the senders of the responses and which of the three flavors of student e-mails they were answering.
Too Small a Test?
The responses from graduate-study directors averaged about 160 words. The paper summarizing the study’s results says the researchers, in comparing responses, did not find anything more than statistically insignificant traces of bias in any of the areas measured. They used plagiarism-detection software to ensure that the similarity among the responses did not stem from graduate-study directors’ use of form letters or boilerplate, and found that in 90 percent of the cases examined, the response letters had significant differences.
Last week Mr. Gross said the study did not rule out the presence of more subtle forms of bias in graduate education “around ideas.” For example, he said, “we don’t know what would happen if conservative students applied to doctoral programs in sociology and said, ‘I want to write a dissertation on why welfare is a bad thing.’” But, he said, the field experiment showed that, at least among graduate-study directors, “there does not seem to be direct political bias in favor of liberals or against conservatives.”
The researchers’ paper says that, because their experiment involved deception, they felt ethically bound not to construct a test that would take a substantial amount of their subjects’ time—as would have been the case if, for example, the researchers had submitted fake graduate-school applications. In seeking not to inconvenience the directors too much, however, the researchers might have made the mistake of doing too little to draw evidence of bias out of them, the paper acknowledges in listing the study’s limitations.
Amy J. Binder, an associate professor of sociology at the University of California at San Diego, said last week that she would have been angry if she had been on the receiving end of a more elaborate field experiment, such as one that involved the submission of fake applications. That said, she argued that the e-mail experiment represents “a very thin test of the whole universe of experiences that undergraduates have when they are thinking about what their future careers will be and whether they are interested in graduate school or not.”
“When you are tapped to be the director of graduate studies for your department, you have a professional role to play, and that is, at least, to respond to people who express interest in applying to your department,” Ms. Binder said. “If I were approached with a letter like that, I would probably go out of my way to say ‘Please, do apply.’”
Mr. Gross argued, however, that people commonly display bias when engaged in activities that they do fairly automatically, without much thought, and believe their bias will not be detected. E-mailing a response to a potential applicant is “a good example of a situation where no one would think that anyone is watching,” he said.
Tracking Young Liberals Over Time
The second paper released on Monday is based on a study conducted by Mr. Gross, Mr. Fosse, and Jeremy Freese, a professor of sociology at Northwestern University. They based their analysis on data from the National Longitudinal Study of Adolescent Health, which surveyed students in Grades 7 through 12 in the 1994-95 academic year and then returned to them with surveys administered in three waves, lasting until 2007-8.
The researchers were able to use the health study in examining the influence of political views on academic careers because the third wave of data collection, consisting of in-home interviews conducted in 2001-2, asked the study’s participants, who were then 18 to 26 years old, whether they were conservative, liberal, or “middle of the road,” and instructed them to place themselves on a five-point spectrum ranging from very conservative to very liberal. Six years later, the study participants were asked the same question, and whether they were enrolled in a master’s or doctoral program and intended to complete a doctorate.
After statistically accounting for other traits that might make some people likelier than others to enroll in graduate programs and seek doctorates, the researchers found that people who identify as liberals are more likely to pursue such a course in life. Moreover, they said, the tendency cannot be explained away by students’ exposure to liberal ideas in college or graduate school, or by factors such as liberals’ relative lack of materialism or interest in marriage at an early age.
The researchers were unable to rule out the possibility that cognitive or personality differences between liberals and conservatives leave liberals more likely to enter graduate school. Mr. Gross said he planned to publish the results of both studies in books that he has in the works.
