A Long Shortlist

The last time I led a search committee in history, the task of compiling a list of semifinalists required a lot of long meetings. One committee member at the time insisted that it was impossible to properly evaluate people on the basis of cover letters and letters of recommendation alone. That led us to create an unmanageably large pool of applicants to ask for further information, with the unfortunate result that we probably misled candidates into thinking that they had a more serious chance for our job than they actually did.

I didn’t want to make those mistakes again in this search. As I mentioned in the first installment of this series -- designed to give you a window on a search in progress -- I was asked to lead the committee because we are looking to hire a faculty member in a specialty similar to my own.

When we met to winnow the list of applicants, one member of the committee proposed that we each reveal our tentative “long shortlists” at the outset of our initial meeting. But I thought we owed it to the candidates to discuss them each individually, no matter how briefly. So that is what we did.

I had directed the committee members to read the files with an eye to placing candidates in one of three categories: clearly yes, clearly no, and maybe. At this point, I reminded them, all we needed to do was create a list of semifinalists to ask for dissertations, chapters, or other publications, along with syllabuses of courses taught. So we did not need to have detailed conversations now about the relative strengths and weaknesses of the better candidates.

We eliminated some applicants from consideration immediately. Most often, their fields were not exactly what we were looking for. Although we specified both a chronological and a subject specialty in our ad, some people seem to have read the two categories as “either/or” rather than “both/and.” Thus some applicants had the chronological expertise we sought, but not the topical; for others, the reverse was true.

We eliminated other applicants for incoherent cover letters; lack of evident interest in research and publication (a must for our research university); failure to indicate clearly how much of the dissertation had been completed (we suspected this revealed someone who would not have a degree in hand next fall); lack of experience in the teaching fields we sought; subspecialties of little interest to us or our university in general; and lack of appropriate language skills for their subject areas.

At the other end of the spectrum, some applicants had ended up at, or near, the top of all our lists; they were easy to deal with. But others received a mix of votes --- “yes” for some of us, “maybe” for others. These applicants aroused the most discussion. (Fortunately, no one was “yes” on one list and “no” on someone else’s.)

In discussing the maybes, we reviewed our notes on the candidates’ cover letters, CVs, and letters of recommendation. For an assistant professor on the committee, learning to distinguish the true raves from the letters of perfunctory praise was, he said, a revelation. One letter would call a candidate “a brilliant and innovative thinker” while another would speak blandly of an applicant’s “solid” research and “competence” in the classroom. One lukewarm letter aroused from all of us the observation that the candidate had made a grave mistake in seeking a letter of recommendation from the scholar, and the scholar should have declined to write one if he felt he could not write a good letter.

Because the field in which we are searching is relatively small, several referees had written letters for more than one applicant. Comparing the wording of those letters proved enlightening: When the same referee expounded for several lengthy paragraphs on one candidate’s many strengths, but offered only brief, pro forma comments about another, we knew which one he preferred.

I did call one referee I knew personally to ask for a further assessment of the applicants for whom he had written. He had tried to do the best he could for each candidate, which I appreciated, but, as I told him, since he knew the applicants and we did not, it would be helpful to us if he made some explicit comparisons. I reported what he told me to the committee, and we took his comments into account in our deliberations, although we did not consistently concur with his assessments.

At least we did not experience what happened in another of our department’s searches some years ago: A referee, the mentor of two candidates, proclaimed each of them “the best student” he had ever taught. All that accomplished was eliminating both of them from consideration because we knew his letters were untrustworthy.

I paid close attention to what might appear to be a mundane aspect of the CVs: the professional memberships listed by the candidates. Remarkably, some applicants failed to indicate that they were members of the appropriate professional organizations, not just the American Historical Association, but also the smaller ones appropriate to the field in which we are searching. When all such groups have low dues for graduate students, it’s definitely a mark against a candidate who has not joined them and who thus is not receiving the journals (and current book reviews) regularly. Nonmembers have to pay more to attend professional meetings, and they usually cannot participate in panels at those meetings. Thus memberships are, to me, an important early sign of the professional engagement we seek from new colleagues.

In the end, we placed 12 candidates on our long shortlist, with five others not yet wholly rejected -- “reserves” in case we made too many errors in selecting the first 12.

Our semifinalists, a group including both women and members of minority groups, range widely in experience. Three applicants are very junior and probably won’t make it into the next round; each has written only a few dissertation chapters. We considered rejecting them out of hand, because we have several other strong candidates who have completed their dissertations and have several years of teaching experience. We even have applicants with recently published books who are interested in leaving smaller institutions for ours. Yet one of the most junior people had enthusiastic support on the search committee, and as long as we were looking at one in that category, we decided we should look at the other two we had assessed similarly. If the existing chapters are truly brilliant, one of these candidates might make it onto our list of invitees.

We decided not to interview candidates at the annual convention of the American Historical Association, although the dean would have paid for travel for two of us. Instead, we are choosing finalists strictly on the basis of written materials. I have experienced AHA “slave market” interviews from both sides of the table; my younger colleagues have been through it only as candidates. I have spent enough hours inside hotel rooms interviewing large numbers of applicants to be dissatisfied with the results. In 30 or 45 minutes one cannot learn much more about an applicant than is evident from written materials. As the hours drag on, candidates blend into each other and interviewers tire. Too much decision-making depends on chance. (Which interviewers are in the room? How exhausted are they? How exhausted is the candidate?)

Thus I recommended to the other committee members that we avoid the convention, and they readily concurred. Almost all the written material we have requested from the semifinalists has now arrived, and we are independently surveying the various manuscripts. So far, my reaction is that we have a strong pool of candidates, and we may well have difficulty selecting three, or possibly four, for on-campus interviews. But I’ll have to see what my colleagues think after we have all done our homework.