Advice

The Art of the Bogus Rating

September 27, 2006

If you look up Matthew L. Julius on RateMyProfessors.com, you'll find this comment about the associate professor of biological sciences at St. Cloud State University:

"Dr. Julius is the single greatest instructor that has ever graced a classroom. His patience and kindess can only be truely appreciated once you've seen the raggedy and motely group of students that he lets work in his lab."

He would feel flattered, except he wrote the comment himself (and yes, the misspellings were intentional). In fact, his RateMyProfessors page is peppered with gags like that one, which he composed purely to amuse his graduate students. Mr. Julius says he and his colleagues take great sport in posting remarks about themselves and each other on the site -- so much so that it has become a medium for their inside jokes.

RateMyProfessors invites anonymous students to grade professors on criteria such as easiness, helpfulness, clarity, and "hotness." Professors like to dismiss the site as flawed and frivolous at best, and untrustworthy and malicious at worst. However, faculty members are using the site more than they let on, and in all sorts of ways.

Based on a recent statistical analysis, Patrick Nagel, president of RateMyProfessors.com, estimates that as many as a quarter of its 2-million visitors each month are professors. Some, like Mr. Julius, are using it for amusement. But others -- both candidates and search-committee members -- are using, and misusing, the site as a source of information in the hiring and tenure processes.

Mr. Julius isn't surprised: "The world is littered with professors who've said they don't check it but do." He confesses he's hooked. While he mainly regards the site as a lark, he says, he can't resist peeking at it several times a semester to see if anything new has been posted about him, his colleagues, and his friends. He also admits to occasionally taking guilty pleasure in seeing weak teachers get bad ratings.

It's fascinating reading, says Margaret Soltan, an associate professor of English at George Washington University who maintains a blog called University Diaries. While she's never posted anything herself on RateMyProfessors.com, she says she gets a big kick out of randomly reading the comments there. "Of course, many of the comments are purely idiotic," she says, "but some are remarkably astute and observant, it seems to me, and even thoughtful and humane at times."

Perhaps that's because they weren't written by students. A number of the remarks about Mr. Julius on the site, for example, were actually written by his teaching assistants and colleagues:

  • One of his TA's posted a remark poking fun at an undergraduate who wrote on a final exam that forest fires had caused rabbits to evolve horns: "PHD Julious is good teecher. He has envolved beyond all other biology teechers and will one day have horns, like a rabbit!"

  • Then there's this one, written by Neal Voelz, a colleague in the St. Cloud department: "Its clear that Dr. Julius is being tutored by Dr. Voelz. His teaching is outstanding."

  • Mr. Julius says he has no idea who wrote this one -- "Unbelieveably helpful and caring for his students. Nice eyes!" -- though all of his graduate students have accused him of writing it.

His one regret about having posted fake reviews? "Now I kind of wish I hadn't done it because there are all these real ratings on there, and every time I get a nice one, everybody assumes I wrote it myself," he says.

Mr. Julius and company at St. Cloud aren't alone. Of the more than 50 people interviewed for this story, nearly every one had either manipulated the site or knew someone who had.

Hadley Mozer and Jon Lemmond -- friends since their undergraduate days at Houston Baptist University -- have made RateMyProfessors.com into their own mutual-admiration society and a means of keeping in touch. Mr. Mozer is an assistant professor of English at Flagler College, and Mr. Lemmond is A.B.D. in history at the University of California at Santa Barbara. Mr. Mozer gives his friend high marks for his ability to write like an undergraduate: "His [postings] are better than mine. There's an art to writing the bogus review. Note the typos, the misspelling, the awkward syntax. These things have to be attended to if you're going to be an aficionado."

Richard Scott Nokes, an assistant professor of English at Troy University, sees no shame in posting faux reviews. He freely admits having primed the pump himself after a friend asked him if he was listed on the site and he found that he wasn't: "I created my own first rating in which I rated myself extraordinarily high. I made myself very difficult, too, and gave myself a chili pepper." (Professors who are considered "hot" get chili peppers by their names.)

Mr. Nokes also invites readers of his blog, Unlocked Wordhoard, to review him on RateMyProfessors.com. "Feel free to rate me whether you've had a class with me or not. Good or bad, I don't care, just as long as it's amusing," he wrote.

Apparently, it's fun for the whole family, as even Mr. Nokes's sister has gotten in on the action. She gave him a mediocre rating (3's across the board) and wrote this comment: "I have heard that he is actually the dumb one in his family. I hear his sister is way smarter and doubly hot! I'd still take his class just to see if I could meet his sister."

If he thought anyone at his university took the site seriously, Mr. Nokes says, there's no way he would have posted his own rating.

While RateMyProfessors.com's welcome message boasts of "better, more accurate ratings" and its guidelines warn professors not to post their own ratings, this reporter was able to post fictitious ratings for imaginary English professor Jim Jones at Algonquin College in Pembroke, Ontario, without registering or providing a name or e-mail address.

Still, some might expect professors would be above such high jinks -- after all, they're supposed to be the adults here. But they are adults who have spent a lifetime in school.

Perhaps that's why academics are so drawn to the site, says Rob Franciosi, an English professor at Grand Valley State University. Professors have been evaluated their whole lives, he says, adding: "All through school it's the grade, the number on the test that they worship. I think some professors are really fascinated and also horrified by being graded again in public."

Jeff Snyder, director of the music business program at Lebanon Valley College of Pennsylvania, definitely falls into the "horrified" camp. He says it pains him to visit RateMyProfessors.com and see some of the petty comments about him like this one: "Stick to selling pianos."

He even compares the experience of reading his ratings to watching a horror movie. "You're fascinated by it, you feel like you need to see it, but you want to cover your eyes because of the unfairness of the criticism," he says. He finds it scary to think that "I had somebody in one of my classes who absolutely hated my guts, and I had no idea."

It stings even more knowing that those comments are on the Internet for all the world to see, he says. Which is why, despite his impulse to steer clear of the site, he still checks it occasionally. And it's why he has resorted to posting a couple of phony rebuttals of his own. Mr. Snyder insists he was simply defending his reputation -- fighting fire with fire, so to speak: "It's the only way that I can respond. It's all bogus, so I might as well play the bogus game."

The name of that game is image management, and many professors are playing it.

"The main form of capital that a professor has is his or her reputation, his or her credibility as a knowledgeable, truthful person," says Kenneth Westhues, a professor of sociology at the University of Waterloo, in Canada, who has studied RateMyProfessors. "And so professors very attentively guard their reputations. That's why they are so interested in RateMyProfessors.com."

He says the practice of padding one's ratings is rampant among professors. While conducting research for his 2004 study on the site, he found numerous instances of professors at his own institution trying to manipulate their ratings. He says he has never done that himself because he believes it's unethical. He declined to "out" his colleagues but said it was funny to see how some described themselves.

Doctoral students are doing it, too. An anthropology student at a state university in the East, who spoke on condition of anonymity, says she beefed up her ratings after receiving some bad reviews -- which called her "nice" but "confusing" and "unorganized," and described her teaching skills as "non existent" -- after she taught her first class last spring.

"I figure if you can vote for yourself for president, you can rate yourself on RateMyProfessors.com, so I did. I gave myself a 4 for easiness, a 5 for helpfulness, a 4 for clarity, and I said, 'pretty good class.'" She also gave herself a chili pepper, because "given the option between a chili pepper or no chili pepper, you take the chili pepper."

She justified her actions by saying, "I'm paranoid that search committees for jobs I'm up for are on RateMyProfessors right now reading my reviews." She herself has, out of curiosity, looked up faculty members of departments where she has applied, so she assumes they're doing the same about her.

Like Mr. Snyder, she's troubled by the public nature of the site. "Bad departmental evaluations don't bother me nearly as much as RateMyProfessors because of how public it is. Google my name and RateMyProfessors is on the first page of hits. Right out there for everyone to see is the handful of students out of 155 that actually didn't like me. This is what someone might call the perfect storm of biased samples."

Ms. Soltan, the George Washington professor, thinks academics ought to develop thicker skin. She sees little difference between the public space of a classroom and the public space of the Internet, except for the size of the audience.

In fact, Ms. Soltan says she prefers RateMyProfessors over the old-fashioned kind of appraisal. "In my experience in-class evaluations are always very positive. Everybody says nice things about everybody because the students are indifferently just checking off yeah, OK, OK, OK. To use RateMyProfessors, students have to positively care enough to open it up and write something."

Mr. Julius doesn't see the big deal either: "It's no different than what's gone on on campuses forever; they've just made it electronic."

Of course, that's easy to say when you have tenure and a smiley face next to your name on the site, Mr. Julius admits, as do both he and Ms. Soltan. It's much harder to laugh off a nasty remark when you lack job security or the face by your name is puke-green or sickly purple.

Mr. Westhues, the sociologist who has studied RateMyProfessors, says job seekers and those with numerous bad ratings are right to feel paranoid.

"Nobody on a search committee is going to admit to having looked up somebody on RateMyProfessors -- but they'll do it, sure," he says. "I know of no instance where a college or university administration has admitted to using the ratings on RMP in the formal evaluation of a professor -- and I myself would be heartily opposed to them doing so -- but do they do it surreptitiously? I'm sure they do. And that's what makes the site so potentially damaging."

A former animation instructor at a state university in Pennsylvania, who asked that her name not be used, claims that administrators fired her based mainly on some bad ratings on RateMyProfessors. Last spring, after she received a letter saying her contract would not be renewed and citing student complaints, she decided to examine her personnel file. In it she found copies of her RateMyProfessor.com reviews, with negative comments highlighted. She says her personnel file also included copies of her male colleagues' ratings from the site. She has filed a grievance.

In an interview with The Chronicle, her department head insisted repeatedly that his university does not use RateMyProfessors to evaluate professors. When asked to explain the presence of those pages in her file, he said, "I don't know how they got in there." The vice president for human relations, who was present when the pages were discovered, said in an interview, "It was a mistake. She was told that it was an error, and the pages were removed."

The former animation instructor isn't buying that explanation. She says, "The mistake is that they got caught. And, yes, the pages were taken out of my personnel file, but that was long after I was not renewed. It's a little late for that to be perceived as a positive action."

Mr. Nokes can't imagine anyone would take seriously what they read on RateMyProfessors, least of all search committees: "I've been on lots of search committees, and I've done everything that you do to check up on candidates, and I've never looked at RMP in that way. It didn't even occur to me to take it seriously in any way, so job seekers needn't worry," he says. "At least not here."

While some academics are appalled by the thought of hiring committees and administrators consulting RateMyProfessors, others see no reason why they shouldn't.

Gene C. Fant, Jr., chairman of the English department at Union University, has not yet peeked at the site during a search only because his department hasn't done one in a few years. "But if we were doing a search now, I would look at it just to see if there were any red flags," Mr. Fant says. He would never disqualify a candidate solely on the basis of those ratings, but "if there were a lot of negative comments, it might be something that I'd have on my radar."

Heiko Schoenfuss, an associate professor of anatomy at St. Cloud State, says he would peek, too. "I mean, I Google people to see if I want to collaborate with them or have them as a colleague at my institution, so why ignore any source of information, as imperfect as it might be?" he says. "Of course, I think it's a biased source like any other source, but, honestly, we know that our teaching evaluations aren't necessarily unbiased either."

Mr. Schoenfuss notes that only a few years ago, no one took blogs seriously, but now, suddenly, everybody has a blog, and everybody is looking at them. He sees RateMyProfessors as just another way to measure the pulse of student satisfaction and student concern.

But who's to say what's a good rating? A very low rating can mean the professor is either a bad teacher or extremely demanding; a very high rating can mean the professor is a good teacher or too easy. The only consensus among professors seems to be that's it's best to be somewhere in the middle.

The worst thing, some professors say, is not to be rated at all.

Perhaps that's why an anthropology instructor, who did not want to be identified, posted a comment on RateMyProfessors for a colleague without telling her: "No one had rated her, and she told me that her boyfriend, who is a professor here, checks every day to see if she's been rated. So I gave her a chili pepper, so that her boyfriend would be proud, and I said that she's a great teacher, which she is."

Does the instructor plan to tell the colleague? "I'll let her think that a student wrote it, because I'm sure they were all thinking it and just never got around to writing it down."