Ah, autumn. The falling of leaves. A new batch of excited freshmen and graduate students. Some different faces among colleagues, perhaps. The roar of a football crowd. And alas, the reading and writing of promotion-and-tenure letters.
For some fortunate reason, I have none to write this year, which must be a first, but unfortunately, I have 11 to read. And after many years of serving on promotion-and-tenure committees, I have finally come to the conclusion that these letters are nearly worthless. The ones I read and the ones I have written.
Think about it. We hardly need letters to evaluate candidates within our own discipline. We are capable of evaluating their research. Letters are strictly for the members of collegewide and universitywide committees, who, through lack of discipline-specific knowledge but mostly lack of time, cannot evaluate the research of candidates outside of their fields. So we call on experts, those renowned scholars from distinguished and preferably higher-ranked institutions, who can vouch for the quality of the candidate’s record. They have, for lack of a better term, letterhead value.
And they write so well and so cogently. Today I have read the expressions “highly commendable,” “groundbreaking,” “impeccably rigorous,” “carefully designed,” and “recognized nationally"—all phrases I wish I could think of when I am the writer. Instead, I come up with “doing good work,” “interesting,” and “innovative.” At least I didn’t say “cool.”
This process is absurd. Consider that the evaluators are selected by the candidate’s department, sometimes with input from the candidate. They are not a random sampling of experts. Indeed, they are far from random and are often biased, whether subtly or blatantly. The most egregious cases of bias involve choosing the candidate’s former professors or the department head’s former colleagues and friends, but other, subtler forms exist as well.
Suppose the candidate has an article accepted for publication in the most prestigious journal in her field. Her department head asks the journal’s editor to write a letter on her behalf. The editor, of course, believes that the paper he accepted is excellent. What else would he think? Is he going to change his mind and say he made a mistake in accepting the paper? Ideally the editor would look at the candidate’s entire corpus of work, but that is too much trouble. The editor, after all, has numerous letter requests, not to mention many manuscripts, awaiting his attention. So in addition to a few casual observations about the candidate’s other research, he writes a detailed review of the paper he accepted, heaping dollops of laudation, knowing that any future success of the paper is a shared success. Kind of like having your kid get into Harvard when you went to a third-tier state university. You, too, get credit.
I once read a letter from a journal editor concerning a candidate up for promotion to full professor who had published four articles in that journal and was on its editorial board. The editor noted that the journal was A-level (in fact it was clearly B-level), and that the candidate had done an extensive amount of refereeing for the editor. Naturally the letter was favorable. Naturally I wanted to transfer it into the “stuff that should never have been written” folder, also known as my recycle bin.
Not only are external letters nearly useless, but the whole process is flawed.
At least half of all academics are exposed to the scientific method of research: stating a testable hypothesis, collecting data, analyzing those data, and drawing a conclusion with the admission that we could be wrong. That process is widely accepted as the correct way to investigate an issue.
In the promotion-and-tenure process, we try to do the same thing. Whereas a scientist might hypothesize that a drug has no positive benefit, we might hypothesize that someone should not be promoted. Whereas the scientist goes about collecting data, we do the same thing in gathering information about the candidate’s research record. Whereas the scientist, upon obtaining statistical evidence that admits only a small possibility of error, concludes perhaps that a drug is effective, we often likewise analyze the data and conclude that the candidate should be promoted. In our case, there is no admission of a margin of error.
The scientist does it correctly. We do not. Our margin of error in evaluating tenure candidates is pretty high, because our sample is not random and far too small. Nonetheless, on that basis, we make a case to the higher authorities that this candidate should be promoted.
If we conducted our research like that, we would be laughed out of the profession.
What we ought to do is make the process more random. For example, each department could compile an extensive list of experts, perhaps at least 100. It could then randomly choose a set. A random sample of experts would at least attempt to remove the subtle biases.
Naturally, I cannot tell you what percentage of letters I have read that are favorable, but my estimate is more than 90 percent. Random letters would very likely produce favorable percentages a good bit lower. Would that result in a smaller percentage of candidates being tenured? Possibly, but after all, tenure is a lifetime contract. The hurdle should be high.
If promotion to full professor is not granted, it is not the end of the world for the candidate. Could a good candidate get three or four negative letters simply because the luck of the draw chose some hard-nosed experts? It could. I suspect that four letters is not enough. Frankly, I would prefer to see six to 10. I cannot imagine a deserving candidate’s being denied promotion with 10 letters.
Perhaps there are other solutions, and I would like to hear some. I just know that we are trying to answer an important question, and doing it poorly.
Well, time to return to reading letters. I will faithfully read them, for I do not want anyone to think I am not doing my job by playing the game the way it has been played for years. And I will not take these letters with a grain of salt. I will take them with an entire 26-ounce box of Morton’s. I will spend much time trying to read what is said and digging deeply for what is not said. I will try to discern the quality of the journals based on impact factors, sponsoring institutions of the editors, and the candidate’s citation records.
When all is said and done, it is an imprecise process, but I will be fair. And next year, I probably will have to write some promotion-and-tenure letters. If I don’t, my letterhead value will go down. I suppose it will be like what happens when the paparazzi stop following a celebrity. Come to think of it, that’s not a bad thing.
