Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    College Advising
    Serving Higher Ed
    Chronicle Festival 2025
Sign In
Teaching

How One Professor Is Trying to Paint a Richer Portrait of Effective Teaching

By Emma Pettit June 16, 2016

In his research, Philip B. Stark pinpointed something that he believed professors already suspected to be true: that student evaluations of their teaching are biased.

Philip B. Stark, associate dean of the division of mathematical and physical sciences at the U. of California at Berkeley
Philip B. Stark, associate dean of the division of mathematical and physical sciences at the U. of California at BerkeleyCourtesy of Philip Stark

Mr. Stark and several other researchers recently examined student evaluations in online courses and found that implicit gender bias had seeped into end-of-semester evaluations. The students routinely rated professors higher when they thought they were male, even though the classroom experiences were standardized and students and professors never interacted in person.

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

In his research, Philip B. Stark pinpointed something that he believed professors already suspected to be true: that student evaluations of their teaching are biased.

Philip B. Stark, associate dean of the division of mathematical and physical sciences at the U. of California at Berkeley
Philip B. Stark, associate dean of the division of mathematical and physical sciences at the U. of California at BerkeleyCourtesy of Philip Stark

Mr. Stark and several other researchers recently examined student evaluations in online courses and found that implicit gender bias had seeped into end-of-semester evaluations. The students routinely rated professors higher when they thought they were male, even though the classroom experiences were standardized and students and professors never interacted in person.

The scores also did not correlate with how much students actually learned, as measured by the final examination.

“Whatever it is the students are responding to, it’s certainly not what they’re learning,” said Mr. Stark, who is associate dean of the division of mathematical and physical sciences at the University of California at Berkeley.

Mr. Stark’s research built on existing studies that suggest a professor’s race, age, accent, and even physical attractiveness could alter evaluation scores.

When he was chair of the statistics department, Mr. Stark analyzed those studies and eventually published a paper concluding that student-evaluation surveys were a poor measure of effective teaching. He was also aware of Berkeley’s reliance on survey feedback during the faculty-review process.

Every semester students ranked their professors’ teaching effectiveness on a scale of one to seven. Department and university committees used an average of that score — and sometimes little else — to inform their decisions. (At Berkeley, professors undergo assessments every two to three years at the start of their careers, then less frequently as they progress.)

As chair, Mr. Stark revamped the process. He had professors submit portfolios of materials they had created for their classes, including syllabi, exams, and lecture notes, as well as examples of student work. He sent other professors into classrooms to observe their peers before major reviews and write up assessments that those being evaluated could read and respond to. Student evaluations were not eliminated, and their input was still valued, said Mr. Stark. He just aimed to widen the lens through which to view a professor’s teaching.

Deandra Little, director of the Center for the Advancement of Teaching and Learning at Elon University, said many colleges are bolstering their assessment process with metrics other than student-evaluation scores. Mr. Stark’s system is unique because many departments are not recommending peer evaluations so frequently, said Ms. Little.

Now, armed with statistical evidence of bias in student evaluations, Mr. Stark wants to graft a similar approach onto the entire mathematical- and physical-sciences division, which encompasses five departments, for next fall. He and others in the division agree that the evaluations are flawed. But how to mitigate those flaws is still up for debate.

Out With the Old

Elizabeth Purdom, an assistant professor in the statistics department, started teaching at Berkeley in 2009. She remembers that her first evaluations were fairly negative. The class was not smooth sailing, she said.

ADVERTISEMENT

But even as Ms. Purdom gained experience, the numbers on her evaluations stayed low. And the written portion and numerical rating often did not align, making it difficult to establish any trend. Once a student wrote that the course was the best stats class she’d ever taken. But then she gave Ms. Purdom a five out of seven on the teaching-effectiveness question.

“Well, that number is not really useful,” Ms. Purdom thought at the time.

The departmental committee that reviews professors brought up those low scores even after her ratings had improved, Ms. Purdom said. The people who conducted her reviews also typically relied on her average score instead of the median, which meant one low rating could tank — or at least drag down — a large pool of high marks.

Ms. Purdom was eager to receive any feedback that might be more useful, so in 2013 she agreed to act as a guinea pig for Mr. Stark’s new evaluation system.

ADVERTISEMENT

A professor in another department observed one of her classes and wrote up a synopsis. Ms. Purdom said that professor gave her a wealth of positive feedback and several concrete suggestions, which gave her confidence in her teaching for the first time.

“Up until that time I was sort of like, OK, maybe I’m not one of these people who is good at teaching,” Ms. Purdom said.

The written observation, along with a teaching portfolio she had constructed, went into her dossier for her midcareer review. Those materials were a stronger foundation than just her student-evaluation scores and a brief teaching statement — the documents typically used to judge a professor at that time, Ms. Purdom said.

The statistics department still uses peer evaluations, as well as teaching portfolios, in tandem with the student scores to evaluate professors for their major career reviews. L. Craig Evans, interim chair of the mathematics department, said that process would have benefited him last fall.

ADVERTISEMENT

As chair, he reviewed multiple professors’ promotion cases with little more than “a single number and raw teaching comments from the students,” Mr. Evans said. He wished he had had a fuller perspective.

“When students evaluate how a course went, they have a view. I don’t think it’s an entire view,” Mr. Evans said.

In With the New

Though Berkeley has cautioned for several years against relying too heavily on student evaluations, the practice still happens, and the university has struggled to avoid it, said Frances Hellman, dean of the division of mathematical and physical sciences.

“All of us cling to this hope that it will be a reasonable metric,” Ms. Hellman said.

ADVERTISEMENT

Ms. Hellman knows firsthand that student evaluations can be unreasonable, or occasionally “kind of merciless,” she said. (She remembers one student’s remark on her hair, which said she looked as if she stuck her finger in a light socket every morning.)

The Committee on Teaching for Berkeley’s Academic Senate reviewed the universitywide policy for evaluating teaching and, in 2015, published its findings. The committee concluded that “student course evaluations alone do not portray a complete picture on which to conduct an evaluation.” The group recommended requiring a teaching dossier that would include peer observation as part of a professor’s merit and promotion materials.

Juan M. Pestana, a professor in the department of civil and environmental engineering and chair of the Academic Senate’s teaching panel, said it was too early to tell if departments were heeding the panel’s suggestions. But there is an active conversation on the campus about the best ways to measure effective teaching, he said.

Ms. Hellman said she supports drafting and circulating new suggestions on how to evaluate teaching to the five departments in her division for the fall. But she said she’s not convinced that peer evaluations would be less influenced by implicit biases than student evaluations are. And she’s skeptical that asking faculty members to watch one of their peers’ lectures would do much to strengthen the observed professor’s teaching.

ADVERTISEMENT

Mr. Stark also understands the potential shortcomings of peer evaluations, but for a different reason. Asking faculty members to sacrifice time and energy to perform additional duties is “a hard sell,” he said. But he added that such work is key to actually improving teaching, not just assessing it.

Department chairs in Ms. Hellman’s division will talk with Mr. Stark throughout the summer to hammer out the specifics of how a department might put peer-assessment and teaching-portfolio requirements into practice. What teaching criteria to examine, how often to prescribe evaluations, and which professors are qualified to do the assessing are all potential points of discussion. She foresees a process that blends all options — student, peer, and self evaluations — to paint a richer portrait of a professor. She hopes it will measure how hard professors are trying to be effective instructors.

“Effort, by and large, will lead to better teaching,” said Ms. Hellman. “Just like it leads to better everything else.”

A version of this article appeared in the July 8, 2016, issue.
We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
Teaching & Learning
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
Emma-Pettit.png
About the Author
Emma Pettit
Emma Pettit is a senior reporter at The Chronicle who covers the ways people within higher ed work and live — whether strange, funny, harmful, or hopeful. She’s also interested in political interference on campus, as well as overlooked crevices of academe, such as a scrappy puppetry program at an R1 university and a charmed football team at a Kansas community college. Follow her on Twitter at @EmmaJanePettit, or email her at emma.pettit@chronicle.com.
ADVERTISEMENT
ADVERTISEMENT

Related Content

How to Read a Student Evaluation of Your Teaching
The Inherent Unreliability of Student Evaluations
Why I Stopped Reading My Student Evaluations
Don’t Shrug Off Student Evaluations

More News

Protesters attend a demonstration in support of Palestinian activist Mahmoud Khalil, March 10, 2025, in New York.
First-Amendment Rights
Noncitizen Professors Testify About Chilling Effect of Others’ Detentions
Photo-based illustration of a rock preciously suspended by a rope over three beakers.
Broken Promise
U.S. Policy Made America’s Research Engine the Envy of the World. One President Could End That.
lab-costs-promo.jpg
Research Expenses
What Does It Cost to Run a Lab?
Research illustration Microscope
Dreams Deferred
How Trump’s Cuts to Science Funding Are Derailing Young Scholars’ Careers

From The Review

Vector illustration of a suited man with a pair of scissors for a tie and an American flag button on his lapel.
The Review | Opinion
A Damaging Endowment Tax Crosses the Finish Line
By Phillip Levine
University of Virginia President Jim Ryan keeps his emotions in check during a news conference, Monday, Nov. 14, 2022 in Charlottesville. Va. Authorities say three people have been killed and two others were wounded in a shooting at the University of Virginia and a student is in custody. (AP Photo/Steve Helber)
The Review | Opinion
Jim Ryan’s Resignation Is a Warning
By Robert Zaretsky
Photo-based illustration depicting a close-up image of a mouth of a young woman with the letter A over the lips and grades in the background
The Review | Opinion
When Students Want You to Change Their Grades
By James K. Beggan

Upcoming Events

07-31-Turbulent-Workday_assets v2_Plain.png
Keeping Your Institution Moving Forward in Turbulent Times
Ascendium_Housing_Plain.png
What It Really Takes to Serve Students’ Basic Needs: Housing
Lead With Insight
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin