In a basement laboratory at Haverford College, Monica Kishore and Peter J.J. O’Malley, two undergraduates, have hit a snag in a fluid-dynamics experiment they’re conducting.
The two physics majors are doing research in the laboratory of Jerry P. Gollub, a professor of physics. The experiment they have helped to design requires them to find a way to get tiny plastic beads to hover at the interface between a blue liquid and a clear liquid. Then they will run an electric current through the liquids, and a camera will track the particles’ motions.
But when Mr. O’Malley, a senior, follows their usual protocol of sprinkling the beads onto the liquid, some of them float on the surface instead of sinking to the interface.
“That’s a problem,” he says flatly.
For decades, students like Ms. Kishore and Mr. O’Malley have been doing original research in professors’ laboratories. And it is just these unexpected problems — and the troubleshooting required to solve them — that catapult students involved in undergraduate research past the cookbook-style class experiments with step-by-step instructions and expected outcomes.
The belief that undergraduate research attracts students to careers in science — and makes them better candidates for such work — has gained almost universal acceptance in academe. But until recently, few researchers have studied whether or not that’s actually the case.
In the past few years, a small cadre of social scientists have, with grants from some of the largest supporters of undergraduate science research, begun systematically studying the effects on students. Three large studies verified some widely held notions about undergraduate research but challenged other assumptions.
They found that undergraduates learn and grow significantly from their research experiences, but require a strong mentor relationship to do so. Such a time commitment may cost faculty members more than they gain from the additional lab help, one of the studies suggests. And the end result may only slightly increase the number of students going on to earn Ph.D.'s.
“There is a widespread belief in the benefits of these programs, but our study was the first to really document it,” says Anne-Barrie Hunter, a researcher at the University of Colorado at Boulder who worked on one of the three studies.
More Than Anecdote?
Investigating the purported benefits of undergraduate research is an important endeavor, particularly in light of the money spent on such programs.
The National Science Foundation spends some $50-million yearly to support about 4,500 students just in its largest undergraduate-research program, the summer Research Experiences for Undergraduates. The Howard Hughes Medical Institute supports an additional 3,300 students. Over all, some 40 percent of students majoring in the life sciences and physical sciences do research with a faculty member, according to two surveys: the National Survey of Student Engagement, which canvassed more than 65,000 students at 209 colleges and universities, and a study performed by the Reinvention Center, at the University of Miami, which surveyed administrators at 75 research universities to get estimates.
“It’s so hard to know, are you making a difference?” reflects Peter J. Bruns, Howard Hughes’s vice president for grants and special programs.
The three recent studies have tried to delineate what difference undergraduate research makes. Elaine Seymour, David Lopatto, and Susan H. Russell each took a different tack: Ms. Seymour’s team, including Ms. Hunter, did in-depth interviews with students at four liberal-arts colleges; Mr. Lopatto developed a survey at those four colleges and then expanded it to other institutions; and Ms. Russell performed several surveys of thousands of students, some of whom had received grants from NSF for research. (See “Researching Undergraduate Research” at left.) The National Survey of Student Engagement also began to ask questions on such topics this year, according to George D. Kuh, its director and the director of the Center for Postsecondary Research at Indiana University at Bloomington.
The three studies largely agreed on their conclusions — they found similar gains in undergraduate learning and effects on students’ career paths — and their results largely reflected the accepted wisdom.
Nancy H. Hensel, executive officer of the Council on Undergraduate Research, which provides training and support for faculty members who involve undergraduates in research, says the studies “confirmed what a lot of people felt intuitively and from their own observations.”
Over this summer, The Chronicle talked with 30 undergraduates doing research at the University of Pennsylvania and Haverford College for firsthand accounts of their experiences and to place the new reports in context.
The students describe learning how to use lasers, magnetic resonance instruments, and microscopes that cost over $100,000. They pick up computer programming languages and how to analyze data in spreadsheets.
Some of the results were just what scientists might expect: The vast majority of students enjoy doing research and say they have learned a lot from it. The three studies found similar cognitive and personal benefits for students, including understanding how scientists work, learning laboratory techniques, and gaining self-confidence.
“There seems to be a tremendous amount of growth,” says Mr. Lopatto, a professor of psychology at Grinnell College. “Students report they can work more independently; they feel they can tolerate obstacles to their work better than they used to; they feel ready for challenge.”
Students also learn how to handle uncertainty and how to work on problems without clear solutions. “In research and in life,” says Adriane Q. Wotawa-Bergen, a junior at the State University of New York at Buffalo who is spending the summer in an electrical-engineering laboratory at Penn, “there’s not always necessarily the answer.”
Their advisers cite this experience as one of the key benefits for students. “They hear us floundering and flailing too,” says Jennifer A. Punt, a professor of biology at Haverford who has seven undergraduates in her lab this summer. “Sometimes it’s plain scary, but it’s exhilarating. We can share that [with our students] and share the confusion and the wonder.”
Charles Slominski, a sophomore at Penn, says he has learned that “mistakes happen” in the laboratory. His adviser, Alain F. Plante, an assistant professor of earth and environmental science, says, “That’s a huge step to learn.”
And indeed, finding out what research is really like is not always pleasant. Mr. Slominski describes much of his time in the lab — washing glassware, weighing samples — as “tedious,” while Evan Reed, a junior at the College of New Jersey who has come to Penn for the summer, says, “Every week it seems like a new problem arises.”
Joshua Magarick, a junior at Penn, experiences similar frustrations when his computer-science research doesn’t go smoothly. “I feel like I’ve wasted my time and their money,” he says, “but that’s how research is.”
Breeding Grounds for Ph.D.'s?
The strongest, and perhaps most often made claim for undergraduate research — that it leads to more Ph.D.'s in science — is also among the hardest to test. Tracking students 10 years after they graduate is no simple task, and is likely to be subject to bias since those who stay in academe would be the easiest to find.
But the social scientists who have studied undergraduate research experiences have done the next-best thing: They’ve asked the students about their plans. Here, the results showed that the story is not as simple as “do research, get inspired, go to graduate school.”
Doing research as an undergraduate may only slightly increase the likelihood of students’ going to graduate school. Mr. Lopatto’s survey showed that more than 80 percent did not change their existing plans to go to graduate school, although it did sometimes strengthen their interest in continuing their studies. Just 3.5 percent said that research changed their plans in the direction of attending graduate school.
In interviews that Ms. Seymour and Ms. Hunter’s team at the University of Colorado at Boulder performed, similar results popped out: Undergraduate research did not seem to have prompted new plans for graduate study, though for many students it did increase interest. (However, Ms. Russell’s large survey found a larger effect: 29 percent of NSF-financed undergraduate researchers had new expectations of earning a Ph.D. after doing research.) “Students are very savvy these days,” says Ms. Hunter, who is a co-director of Ethnography and Evaluation Research, an independent research unit at Colorado. “They come into college with the expectation that they’re going to go on.”
Many of the students that The Chronicle talked with agreed with that conclusion. Mr. O’Malley, the senior doing research on fluid dynamics at Haverford, says he had planned to go to graduate school even before starting college or doing laboratory research. “Probably most significant is it didn’t change my mind,” he says.
Those results may be due to selection bias: Students who do research, particularly those who have won competitive grants to support them or those at highly selective colleges like Penn and Haverford, tend to already be among the strongest students, and may have already given thought to future science careers. (In fact, Myles G. Boylan, a program officer at the NSF, suspects that Ms. Russell’s survey may have found a stronger effect on Ph.D. expectations because it included weaker students, with 15 percent of the respond-ents reporting grade-point averages below 3.0.)
“Are we picking winners or are we creating winners?” Mr. Bruns, of the Howard Hughes Medical Institute, asks. “I don’t know.”
Some students used the undergraduate research experience to help them decide against future academic pursuits. Mr. Magarick, the Penn junior doing computer-science research, says he has “no interest in entering academe because I don’t have the patience for it.”
Ms. Russell, director of the Survey Research Program at SRI International, a nonprofit research institute, found that 17 percent of students who had received NSF grants said they learned through their lab experiences that “research is not for me,” while Mr. Lopatto found that 4.5 percent of students said research made them less likely to go to graduate school. “But a lot more said, This is the best experience of my undergraduate years,’” Ms. Russell says.
What Doesn’t Work
At Penn’s medical school, Ashwin S. Nathan, a junior, smiles as he ticks off the many steps he performs to execute a single experiment in his tissue-engineering project in the laboratory of Robert L. Mauck. He makes a plastic scaffold, grows cells on it to imitate the knee meniscus, uses an instrument that pulls on the scaffold to deform it, and then studies how the strain affected the expression of genetic material in the cells.
He enjoys working in the lab, but didn’t during his first research experience at Penn, in another research group: “It was just washing dishes,” he recalls.
Ms. Hunter, of the University of Colorado, who performed the in-depth interviews with students, says that Mr. Nathan’s experience fits with her hypothesis: that dedicated mentors and an “authentic” research experience produce better results. In a small study at Colorado that she hopes to expand, she found that “students that didn’t have a positive experience were doing more-mundane and routine tasks rather than being involved in the actual science of the project.”
Another problem students reported to Ms. Hunter was poor relationships with mentors. Ms. Russell also found that in an open-ended question in her survey about improving research programs, by far the most common responses suggested increased or better faculty guidance.
Ms. Wotawa-Bergen, the junior from Buffalo, seems to try hard not to complain about her experience at Penn. She does research in the laboratory of Gianluca Piazza, an assistant professor of electrical and systems engineering, and when asked if she has enjoyed her summer so far, she glances at the ceiling and says, “Um, I like it.”
But it gradually becomes clear that it hasn’t met her expectations. She has done a lot of work that repeats previous research, and snafus with equipment have meant that her experiments have been delayed. (Mr. Piazza says about her project, “There’s always a novelty to it, but it’s not going to be transformational.”)
Ms. Wotawa-Bergen has been at Penn for six weeks when a reporter visits. Mr. Piazza has traveled during all but two, and she has met individually with him only once. She shrugs it off: Other students she knows haven’t even met the faculty members running their labs.
Making a Difference
Ms. Russell was able to pull data from her large-scale surveys that define the most important characteristics of undergraduate research — those that correlate with increased confidence or a stronger desire to take on postgraduate studies. Such factors include working longer in a laboratory, co-authoring a paper that was submitted to a journal, attending conferences, and serving as a mentor to other student researchers.
Emily M. Hinchcliff, a senior at Haverford who does immunology research in the laboratory of Dr. Punt, has participated in most of those activities. She did research during one previous summer and plans to continue during the school year; she attended a conference in California this summer; and she has been a mentor to a sophomore, Raven G. Harris. “It was definitely a very different lab experience,” says Ms. Hinchcliff of her relationship with Ms. Harris, “being able to teach her and talk with her about her results, have her fresh perspective and my somewhat more jaded perspective. I think it improved both of our abilities to critically analyze our data.”
In Ms. Russell’s results, some research activities that may seem important had little correlation with good outcomes. They included students’ helping design their project, giving an oral presentation, and writing a final report about their research.
Understanding what works and what doesn’t could play a role in shaping future efforts. “There’s a movement to see what constitutes a good research experience, so there’s a base line to give faculty,” says Ms. Hunter, of Colorado.
This year the Council on Undergraduate Research published a book, Developing & Sustaining a Research-Supportive Curriculum: A Compendium of Successful Practices. And Mr. Lopatto, whose survey about undergraduate research experiences is available to any university that wants to assess its own program, says he hopes to pull together the results to give general advice to faculty members.
What About Faculty Members?
Of course, the benefits of undergraduate research don’t accrue only to the students. All of the 13 faculty members from Haverford and Penn whom The Chronicle interviewed say they have enjoyed working with undergraduates. They mention personal satisfaction and advances in their research.
“People say, How can you get undergrads to do publishable work?” says Dr. Punt, of Haverford. “You don’t get them to do it; you just expect it to happen.”
Francis R. Blase, an associate professor of chemistry at Haverford, cites the personal rewards of working in the lab with her students. “I have so much fun with them,” she says. “They’re young, they’re fun, they’re enthusiastic, they have good ideas.”
But scratch the surface and concerns start to arise. “It’s a huge time investment,” says Mr. Plante, of Penn. “You do have to do a lot of hand-holding.” The assistant professor admits that the time spent working with the students may not help his case for tenure. “It’s risky,” he says.
For faculty members at liberal-arts colleges like Haverford, continuity in research is an issue. “My seniors leave,” says Dr. Punt, “and they each have developed an intuition, some skills, some gifts with their hands that you can’t replace.”
And Robert S. Manning, an associate professor of mathematics at Haverford, admits, “It’s pretty rare that an undergraduate project would create a paper that wasn’t already under way.”
Indeed, the faculty members interviewed by Ms. Hunter and her colleagues from Colorado tended to present a less rosy view. In the interviews, faculty members mentioned the costs of doing research with undergraduates twice as often as the benefits.
Ms. Hunter says the family strains for faculty members can be profound: “They’re dedicating their lives to other people’s children and maybe not putting in as much as they’d like with their own family.” They also struggle with balancing their own research goals with the need to educate their students.
Colleges also don’t do a good job of compensating faculty members for their efforts, she says. “Institutions see student labor help as a reward enough,” she says, but “faculty honestly state they could get their research done a lot faster if they weren’t working with a student.”
Indeed, one of Ms. Russell’s recommendations to the NSF is to encourage colleges to include serving as a mentor to undergraduates as a factor in tenure and promotion decisions. “There’s this national clamoring for getting kids to stay in science and engineering,” she says, “so it would make sense for institutions to find ways to incentivize those activities among professors.”
http://chronicle.com Section: Research & Publishing Volume 53, Issue 50, Page A12
