When a study claims to prove there’s no bias against women in science, no surprise that it gets an avalanche of media attention. It’s newsy and notable that a rigorous study provides such comforting news.
One problem: The 2015 study, performed by Wendy M. Williams and Stephen J. Ceci, isn’t rigorous. It’s plagued by five serious methodological flaws.
1. Bias goes well beyond the hiring hurdle. The two authors have become known for their public claims that no bias exists against women in science, as in an April 2015 essay on CNN, “The Myth About Women in Science.” But their study concerns bias solely in hiring. It’s not evidence that no bias exists at all. In many instances, only after women start work do they run into problems.
Gender bias across the academic workplace affects female scientists in distinct patterns. For example, because of widespread cultural beliefs that women are less competent at science than men are, women often have to provide more evidence of their skills to be seen as equally competentn. “Even when I went up for promotion,” said a Latina biologist, questions were raised about “whether or not I would continue to be doing the things that I was doing once I got full professor. … I had never heard that kind of a comment ever expressed in previous deliberations.” That sort of double standard has been documented over and over again for decades. Two-thirds of 557 female scientists reported encountering it in a recent survey.
Another distinct pattern, “the tightrope,” leaves women negotiating a narrow space between being seen as too feminine to be competent or too masculine to be likable. To quote a woman at MIT: “To get ahead here, you have to be so aggressive. But if women are too aggressive, they’re ostracized, … and if they’re not aggressive enough, they have to do twice the work.” The tightrope has been documented repeatedly in lab studies. Three-fourths of the scientists interviewed for Joan’s recent co-authored book, What Works for Women at Work, reported that pattern.
In addition, in that recent survey of female scientists, 64 percent of those with children reported a third pattern of gender bias — one triggered by motherhood. Said an immunologist in the report, there’s “this perception that if you’re a mother, you can’t be a high-achieving scientist.”
2. The superstar problem. Williams and Ceci are not experimental social psychologists, which explains why their study is flawed by significant methodological errors. Their study presented subjects with three job candidates and asked which one they should hire. The female candidate for the hypothetical academic job in science was described as “extremely strong,” according to the narrative “notes” provided by the experimenters.
Participants in the study were told that the chair of the search had called this female candidate “a real powerhouse,” and that her recommenders praised her “high productivity, impressive analytic ability, independence, ambition, and competitive skills” with comments like “Z produces high-quality research and always stands up under pressure, often working on multiple projects at a time.” Participants also were told of her tendency to “tirelessly and single-mindedly work long hours on research, as though she is on a mission to build an impressive portfolio of work,” and that her job talk received a 9.5 (out of 10) rating.
What a superstar! That’s the problem. Research shows that women superstars actually tend to be ranked higher than similarly situated men in some contexts. After all, who knew a woman could do it?!
The women who have problems succeeding in academic science typically are the ones who are merely excellent. These women often find that they have to provide more evidence of competence than their male colleagues do. For example, bias creeps in when people hear job talks by candidates and hold the women to higher standards than the men. Williams and Ceci conveniently eliminated that form of bias below the superstar rank by granting their imaginary female candidate superstar status up front. A more standard study design would have asked subjects to rate identical job talks from a man and a woman.
Equality for superstars is different than equality for women in general. To paraphrase the late feminist U.S. Rep. Bella Abzug, We’ll know we have equality not when Alberta Einstein is hired as an assistant professor, but when a female schlemiel gets as far as a male schlemiel. Or, in this case, when the woman who is merely excellent has opportunities equal to those of her male peer.
3. The social-desirability problem. Doing research outside your own field of expertise can be a challenge. Williams is a developmental and educational psychologist, and Ceci is a developmental psychologist. Neither is in experimental social psychology, which is where people are trained to study race and gender bias. Social psychologists know that experiments have to be carefully designed to eliminate “social desirability” effects. That’s a fancy way of saying that people in a research study will know they shouldn’t be biased, so they will give you politically correct answers.
This study’s experimental design gave many clues that subjects were being tested for bias. In a typical social-psychology experiment, participants are told to make hiring recommendations and are given a convincing cover story that explains why they are being asked to do so. The experiments are designed so that the subjects think they are actually helping to make the hiring decisions.
That’s not the way Williams and Ceci’s experiment was designed. Their subjects were told to “imagine you are on your department’s personnel/search committee.” They were given obviously contrived narrative reports on the candidates, not at all the way science professors are hired. In case participants didn’t get the point that this was a hypothetical situation, they were told that the superstar female scientist was named “Z.” Faculty members are not dimwits. It was easy to figure out what was really going on.
The Williams-Ceci experimental design does provide some important data. It proves that scientists know, and can produce, the politically correct answer if they are cued that they are participating in a study about gender bias in science. The experiment produces no data about what the same scientists will do if they think they are in an actual hiring situation.
4. It’s wonky — we warned you. Williams and Ceci do something no social psychologist would do. They compare scientists in a design that varies across five axes: gender of the study respondent; gender of the target (“Z”); marital status of the target; whether the target is relocating for a partner’s job or vice versa; and whether the target’s partner is employed, has a home-based business, or is a homemaker.
A typical social-psychology study would cross all of the factors, drilling down to two at a time, for a four-condition analysis. In sharp contrast, the Williams-Ceci analysis includes only a haphazard select few of the conditions. Given that all of the factors were considering at once, they should have had 72 different scenarios! They would have needed to vary the gender of the study participant; the gender of the candidate; the candidate’s marital status (married, single, divorced); whether the candidate is a parent (has kids, has no kids), and the partner’s employment (employed, works at home, homemaker).
It is impossible to tell which of the factors they included in producing their result. The study design means we just don’t know. Social psychologists call that the problem of “confounding variables.” What we have here is not clear findings; it’s mud.
5. The “work-devotion schema.” The mud means that Williams and Ceci fail to pinpoint the effect of bias triggered by motherhood, which is a particularly strong form of gender bias. Recall that paragon of virtue, Z, who tended to “tirelessly and single-mindedly work long hours on research, as though she is on a mission to build an impressive portfolio of work.”
In other words, even if Z had children, she didn’t let them affect her work life. That reminds us of the law-firm partner who, after a female colleague returned to work two weeks after giving birth, said, “Now, that’s the responsible way to have a baby.”
Responsible? Is it responsible for a new mother to be “single-mindedly” focused on her work? Most people would say no: She has a baby to take care of. Is it responsible for a new father to be single-mindedly focused on his work? But of course: He has a family to support.
And there’s the rub. The “single-mindedness” requirement — the sociologist Mary Blair-Loy calls it “the work devotion schema” — applies only to men with stay-at-home partners and to women without children. No wonder only 50 percent of tenured female scientists are married with children — more than twice the rate of childlessness among American women in general.
Williams and Ceci have written about this issue themselves, but they see having children as a “lifestyle choice.” We see it as something different. If science designs its professional ideals around a man married to a homemaker, that’s sex discrimination. Kids aren’t for everyone, but for adults who want them, having children is a key goal. Requiring women to sacrifice having children as the price of an academic career in science guarantees a paucity of women.
Why smart people say silly things. Why did so many smart people — both those who did the study and those who read it — overlook these serious methodological issues? It’s called confirmation bias. People tend not to interrogate findings that confirm what they already believe.
A lot of people want and need to believe there’s no bias against women in academic science. Take a look at this 2015 study of the public comments on a rigorous study in 2012 documenting bias against women in science. Here are some of the comments:
- “The reason why there is gender bias is because there are only a limited amount of experiments that can be conducted in the kitchen.”
- “Women get pregnant. This is a real disadvantage and risk for any project leader. So given the same qualifications, I would rationally go for the man. Not saying it’s right, just saying there are logical reasons behind it.”
- “The successful males I train simply seem to be hungrier and more willing to make the personnel sacrifices required to get ahead of the competition.”
We know it is comforting to believe that sexism in science is over, and that the tables have turned and women are now the preferred item on the menu. Fine, whatever: Enjoy your comfort food. Just don’t call it scholarship.
