A Spat Over Teaching Evaluations Roils a Department

When economics faculty members at James Madison University received their annual evaluations this summer, some of them were taken aback. Had their teaching gotten much worse all of a sudden?

Six economics professors who spoke with The Chronicle said their overall performance scores, which encapsulate teaching, research, and service, dropped — most by at least 2 points on a 9-point scale in a single year. For five professors, lower teaching scores were to blame.

The teaching evaluation — conducted by the academic-unit head for economics, who leads the department and supervises its faculty — didn’t capture teaching well, faculty members said. The reviews, they alleged, stripped out qualitative factors like instructional methods and focused on “countable” metrics like class grades. Several professors said they were dinged because their students were earning too many D’s and F’s.

The six economics faculty members — representing more than one-third of a department that, until recently, had 15 tenured or tenure-track professors — saw their overall performance rating fall from “excellent” to “satisfactory.” Faculty members who don’t achieve an “excellent” rating can lose out on promotions, merit pay, and financial awards from the university. “Satisfactory” is the minimum rating to remain employed.

Five of the six faculty members appealed their scores, arguing that the evaluations failed to measure teaching abilities and methods in a valuable way. They’ve met with the provost, who they said listened to their concerns. As of this week, the deadline to finalize evaluations has passed, and the professors said that the business school’s dean did not overturn their lower ratings.

What’s happening in James Madison’s economics department touches on two of the most heated debates in higher ed: how to best evaluate faculty members on the quality of their teaching, and whether grade inflation has become a pervasive problem.

Amid growing skepticism of tenure among policymakers and budget shortfalls, colleges are facing pressure to better quantify the wide-ranging duties of a professor — efforts that often prompt faculty resistance for lacking nuance. Meanwhile, student grades around the country are going up, and scholars are debating whether professors are making more of an effort to teach effectively and help struggling students, or if they’re simply making their classes easier.

In an interview, James Madison administrators denied the economics professors’ claims that their department head had asked them to inflate grades and said the university isn’t lowering the bar for students. Officials want professors to meet students where they are and lift them up.

But the economics faculty members who spoke with The Chronicle see the shift in teaching evaluations as evidence of a larger problem at James Madison. As the university records its highest enrollment numbers, some professors feel that quality teaching isn’t valued. They fear they’re effectively being directed to water down grades so as to graduate as many students as possible.

For some, after a year of tension between faculty and their department head, the conflict was the last straw. One professor has resigned, another is retiring early, and others are actively job hunting.

Shifting Expectations

The disputed evaluations were conducted by the economics department’s new academic-unit head, Charlene M. Kalenkoski. At James Madison, academic-unit heads manage the departments. They are hired by their college’s dean, in contrast to department chairs, who are typically elected by their department’s faculty. James Madison professors said they feel that having an academic-unit head takes away accountability measures that faculty members usually have with a chair.

Kalenkoski, also a professor of economics, was hired from Texas Tech University in 2022. She declined an interview request from The Chronicle, and she did not respond to a written request for comment on the evaluation process and specific claims made by faculty about how she conducted the reviews.

For each category — teaching, research, and service — Kalenkoski started each faculty member with a score of 5 on the 9-point scale, and then added or subtracted points for different reasons. That’s according to an appeal letter shared with The Chronicle by S. Kirk Elwood, one of the economics professors who protested their evaluation.

Some factors, like teaching a class one didn’t want to teach or developing a new course, added points. Others, like low average course grades, student complaints, and unresponsiveness to emails, decreased scores. Professors who shared their evaluations with The Chronicle received scores of 4.4 to 6.7 over all, and 4 to 7 in the teaching section.

Faculty members wrote the guidelines for annual evaluations, which are spelled out in James Madison’s faculty handbook and the economics department’s governance document. Allowing faculty members to write and formalize their own evaluation guidelines was a new directive from the provost’s office last year. The academic-unit head made edits and approved the final version.

But the economics professors take issue with how the guidelines were applied in 2023, saying their evaluations diverged dramatically from previous years.

This year’s assessments were short and offered little feedback. Professors said student evaluations were used differently, with instructor-effectiveness scores, which are calculated based on students’ numerical ratings, appearing to count as a main factor. James Madison’s faculty handbook states that student evaluations should “not be the primary method” for evaluating teaching performance.

In several evaluation letters, Kalenkoski included versions of the same line: “Please work to meet students where they are in terms of skills and preparation and provide remedial and extra assistance as needed in order to reduce the number of D and F grades.”

Kalenkoski told some faculty members that she didn’t review or consider the qualitative data they’d submitted, such as course materials, exams, or projects. That’s not required, but it had been a convention in the past. Students’ written comments were also not considered, faculty members said.

“There was no attempt to evaluate what was going on in my classrooms,” said Scott R. Milliman, an economics professor. “The point was, my GPAs were too low, period. And it didn’t matter whether there was integrity to those grades or not.”

Elwood’s 2022 evaluation letter from the previous academic-unit head, which he shared with The Chronicle, included detailed feedback on Elwood’s teaching methods, syllabi, course materials, student evaluation comments, and grading system. The earlier review also commended Elwood for having high academic standards. In 2021 and 2022, he received a 7.4 overall rating, categorized as “excellent.” In 2023, he received a 4.4 overall rating with a 4 in teaching — barely “satisfactory.”

“I was praised in my previous years for holding the line on things like grades,” said Elwood, who’s decided to retire from James Madison after this year. “Not only was the criteria new, it was a 180-degree-turn difference from what was expected before.”

Wrestling Over Grades

How to best evaluate teaching has been a longstanding, fraught question. A vast body of research has found that student evaluations are biased, with women and people of color consistently scoring lower than white men even when they’re all teaching the same material. Peer evaluations of teaching have similar problems.

Adrianna Kezar, director of the Pullias Center for Higher Education at the University of Southern California, said the economics department’s criteria at James Madison didn’t seem wildly out of line, but they were unusual in some ways.

Factors like teaching a new course or revising a concentration are generally included in annual evaluations, Kezar said, but they’re classified as service. The teaching section usually focuses more on what happens in the classroom. Sometimes that includes student evaluations, classroom observations, or teaching summaries, she said.

Many of the more qualitative measures of teaching, like portfolios and observations, are time-intensive and more often used in consideration for promotion and tenure, she said.

“A lot of the measures that we know are good are tough to do regularly because of just people power,” Kezar said. “I think we’re wrestling with what’s the interim annual criteria, and that’s why we relied on some of these ones that aren’t particularly good, like student evaluations, because they’re short and summative and easier to get at.”

Kezar added that she doesn’t consider grade distributions a “valid” measure to include in an evaluation. Rather, they’re typically used to rethink course materials and prerequisites. For example, Kezar said, if a class has lots of D’s and F’s, the department might look at revamping the prerequisite and making sure students are adequately prepared as they advance through the curriculum.

In one-on-one evaluation meetings, some economics faculty members said, Kalenkoski told them there’s a collegewide goal to improve grades. The College of Business has one of the lowest grade distributions across the university, according to institutional data.

But Michael E. Busing, dean of the College of Business, said improving learning is the goal, and higher grades are just the effect of improved learning.

“At no point would anyone in the College of Business, or anywhere else that I know of, say we need to inflate the grades,” Busing said. “Rather, we would look at a grade distribution and say, What’s happening here? Possibly, the students are not mastering the material in this course. And so when we see something like that, we ask, Is there a better way for us to teach this material?”

In ECON 200, a general-education course that’s also required for all College of Business students, Busing said 25 percent of the students enrolled in the fall of 2022 withdrew or received D’s or F’s.

“That is sending a signal to me that we need to take corrective action,” Busing said, “and again, I just want to be very clear, not to inflate grades, but to do a better job of teaching and give support for students.”

Milliman, the economics professor, said he’s been trying to meet students where they are by hosting review and help sessions outside of class time. He wants to give “grades of integrity,” he said, and better engage his students. But with distraction from cellphones and Covid learning loss, he said, there’s only so much he can do.

“I want to recognize where they are,” he said of his students, “but I don’t want to meet them there. I want to pull them up.”

Busing emphasized that the college wants to bring students up to standard and that they “are not looking to lower any bars,” because the college’s accrediting body looks at student success after graduation. If students aren’t mastering the material, Busing said, “that’s no good in the workplace.”

Perceived Pressure

In this year’s teaching evaluations, some economics faculty members see a university that is shifting its priorities.

Last year, James Madison was reclassified as an R2 doctoral university with “high-research activity.” One economics faculty member, who asked to remain anonymous because he’s on the job market, said Kalenkoski told him that teaching doesn’t matter that much for an R2 institution.

Heather J. Coltman, James Madison’s provost, said that the university had already met the R2 criteria for research spending and doctoral students years ago and that expectations for faculty haven’t changed in recent months.

“Around all of that, right, is this sense of identity,” Coltman said. “The sense of, Hey, we’re a competitive national university now. We got there by being who we are, and so how can we make sure it’s sustained?”

Meanwhile, James Madison’s enrollment has been increasing steadily for decades. Professors say they have more students than ever. But competition is heating up as the college-age population drops off across the country.

Some faculty are perceiving pressure from the administration to improve grades to fill seats and boost popularity.

“Is it the incentive of the dean and maybe the university at large to give higher grades so that we make sure that we’re popular and everybody likes us and we have good student evaluations so that we make sure we fill the seats, as opposed to worrying about the education as much?” Elwood asked.

Busing acknowledged the impending enrollment cliff but said he’s not focused on popularity, only learning. A spokesperson said the university isn’t actively seeking to grow.

Kezar, of the University of Southern California, said the debate at James Madison is a “microcosm” of larger, widespread issues in higher ed. These days, colleges are constantly grappling with the pull between enrollment, budgets, and the academic mission, she said.

Andre R. Neveu, one of the professors who left James Madison partly due to the new evaluation measures, said he believes the university hasn’t given faculty enough support to match the increased student population.

“I understand the need to survive,” Neveu said. “With that comes acknowledgment that that creates significant challenge for faculty, and they need additional support. If you’re not going to support them, then you’re simply trying to fulfill your enrollment mission without the educational component.”

James Madison provides support to faculty and students through its Center for Faculty Innovation, which offers teaching support and consultations, and an Early Student Success system, which Coltman said brings together professors, advisers, and tutors to help struggling students, among other programs.

Neveu, however, said he would’ve liked to see more commitment to preserving educational quality.

“Is it possible they’re on the move over the next 20 years to fix this? Sure,” Neveu said. “Do I need to sit here and fight tooth and nail every day to maybe see change over the next 20 years by the time I retire? That’s where I decided I couldn’t stay.”

‘Going Back to Basics’

The appeal process and response from the administration has left several economics faculty members dissatisfied with the university’s handling of the evaluations.

Now economics professor Bill Grant and his colleagues are trying to revise their department’s governance document to make the criteria more specific.

“We will not have a criteria where your teaching is demoted because you gave out too many D’s and F’s,” said Joanne M. Doyle, another economics professor. “I mean, I can’t imagine it. If that’s the case, then I throw my hands up and I give up.”

An administrator from the provost’s office has stepped in to mediate the process, as the document must be agreed upon by faculty and get sign-off from the academic-unit head, dean, and provost.

Grant said faculty members are “going back to basics” to spell out in the document the department’s longtime standards. They also want to clarify how student evaluations and grade distributions should be used. For grades, he hopes to add a clause that prohibits the academic-unit head from counting a professor’s grading against them.

“There’s a broad range of GPAs that can be consistent with high-quality teaching,” Grant said. “We plan to basically assert that and say there’s not one specific GPA average that shows you’re doing it right and anything that’s not that is wrong.”

Grant submitted the following proposed passage to Kalenkoski for consideration, per an email thread he shared with The Chronicle: “The AUH will not use the annual evaluation process to pressure any faculty member to change their grading system or use GPAs as a factor for determining faculty teaching-evaluation ratings unless there is evidence that the faculty member’s teaching effectiveness is compromised by their grading system.”

Kalenkoski declined it.