> Skip to content
FEATURED:
  • The Evolution of Race in Admissions
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Find a Job
    • Post a Job
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Find a Job
    • Post a Job
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Find a Job
    • Post a Job
Sign In
ADVERTISEMENT
Advice
  • Twitter
  • LinkedIn
  • Show more sharing options
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Copy Link URLCopied!
  • Print

Why Your ‘Objective’ Screening Rubric Produced Biased Results

Five things that search committees can do to move more women and people of color forward in the executive-hiring process.

By  Allison M. Vaillancourt
February 16, 2021
Allison-Feb.16-gears-5688996_1280
Image by mohamed Hassan from Pixabay

When you were invited to serve on the search committee for a high-profile leadership post, you thought it would be a chance to diversify an executive team that had no diversity at all. Here, finally, was a tangible way to respond to equity demands and disrupt previous hiring practices that seemed to yield the same leadership phenotype over and over again at your institution.

At the first search-committee meeting, you were both pleased and surprised when the chair announced that the group had two primary goals: (1) Recommend at least three finalists who would work well with the president’s cabinet, and (2) ensure that diverse candidates were both identified and fully considered.

We’re sorry. Something went wrong.

We are unable to fully display the content of this page.

The most likely cause of this is a content blocker on your computer or network. Please make sure your computer, VPN, or network allows javascript and allows content to be delivered from c950.chronicle.com and chronicle.blueconic.net.

Once javascript and access to those URLs are allowed, please refresh this page. You may then be asked to log in, create an account if you don't already have one, or subscribe.

If you continue to experience issues, contact us at 202-466-1032 or help@chronicle.com

When you were invited to serve on the search committee for a high-profile leadership post, you thought it would be a chance to diversify an executive team that had no diversity at all. Here, finally, was a tangible way to respond to equity demands and disrupt previous hiring practices that seemed to yield the same leadership phenotype over and over again at your institution.

At the first search-committee meeting, you were both pleased and surprised when the chair announced that the group had two primary goals: (1) Recommend at least three finalists who would work well with the president’s cabinet, and (2) ensure that diverse candidates were both identified and fully considered.

To guide this work, the search chair said he wanted to apply some new theories about unconscious bias in hiring that he had gleaned by reading Jennifer L. Eberhardt’s Biased and Abigail J. Stewart and Virginia Valian’s An Inclusive Academy. Both books, he said, had helped him realize how much “gut feelings” had guided candidate selections in the past. This time, he was committed to using a highly structured assessment process that would be both equitable and methodical. Everyone on the committee applauded his good intentions. “Finally,” you said to yourself, “a search committee that knows what it is doing and is committed to a fair process.”

The committee created a rubric to evaluate candidate dossiers, and the assessment tool produced a diverse slate of candidates for first-round, one-hour video interviews. Soon after those interviews, however, things took a wrong turn. The committee’s score sheets revealed that white male candidates had outperformed women and people of color on almost every criterion of the carefully crafted rubric, including “grasp of emerging trends in higher education,” “strong public-speaking skills,” and “comfort with conflict.”

ADVERTISEMENT

The committee was puzzled. How was that possible? How did diverse candidates whose impressive dossiers indicated such promise fail so miserably during a short on-camera interview? Was there a problem with the evaluation rubric? No; it worked just as it was intended. And that was the problem.

Let me pause here to acknowledge that the scenario I have described is fictional. While it would be amazing to have a search-committee chair kick things off by citing the work of Eberhardt, Stewart, and Valian, I’ve never actually seen that happen. That said, in my 30 years in higher-education administration and now as an organizational consultant, I have served on and observed several hiring committees that relied on detailed assessment rubrics to evaluate candidates. Such rubrics tend to yield a diverse pool of candidates when the dossiers are reviewed, but things start to fall apart once the interviewing begins. Why? Because of tightly held perceptions of how leadership behavior should be demonstrated in higher education.

Ijeoma Oluo — in her 2020 book, Mediocre: The Dangerous Legacy of White Male America — wrote that she and her racial-justice colleagues often utter the words “works according to design” in response to actions or decisions that so obviously benefit white men at the expense of people of color. “Although the phrase may seem alarmingly cold-hearted,” she wrote, “it is our way of reminding ourselves that the greatest evil we face is not ignorant individuals, but our ignorant systems.”

What that means in leadership searches is that committee members often rely on narrow visions and demonstrations of leadership to assess candidates. Candidates who do not look or sound like the leaders we have come to expect end up being evaluated less favorably. White male candidates score well against the evaluation criteria because they act in accordance with the visual and auditory expectations that come to mind when we think about the majority of higher-education leaders we have seen for as long as we can remember.

So, what is the solution? Are we doomed to homogenous leadership teams until the end of time? Of course not, but achieving different results will take more than good intentions. It will require some different goals, including a commitment to reimagining what a campus leader looks and sounds like.

ADVERTISEMENT

Here are five things that search committees can do to move more women and people of color forward in the executive-hiring process:

  • Don’t be swayed by the math. Using rubric scores works only if the assessment instrument and the evaluators are completely unbiased and the measures are easy to quantify. Leadership attributes are, of course, highly subjective. If certain types of people seem more likely to be screened out by your assessment tool, consider the possibility that there is something wrong with it or the way it is being used. While initial assessment scores can be useful in getting a sense of how candidates compare against one another, be sure to take time to discuss why each candidate scored well or poorly. Slowing down to consider the various factors that explain a candidate’s rubric score can reveal a reliance on faulty models of what a campus leader must look and sound like to be successful.
  • Avoid the “likeability” trap. Most of us gravitate to people who are like us. If you hear comments like “It would be fun to work with her” or “I felt an immediate connection,” be open to the possibility that affinity bias is at play. It is normal to prefer people who are like you and to be wary of those who are not. Make it a practice to both name and analyze how candidates make you feel.
  • Challenge the charisma requirement. In his 2019 book, Why Do So Many Incompetent Men Become Leaders?, Tomas Chamorro-Premuzic notes that we are drawn to people who have a strong sense of self, make bold declarations, and discuss visionary plans. He adds, however: “There is a world of difference between the personality traits and behaviors it takes to be chosen as a leader and the traits and skills you need to be able to lead effectively.” Too often, charisma gets people their leadership roles, but it does not help them to succeed on the job. Charisma can blind search committees to a candidate’s lack of other critical leadership attributes.
  • Consider that perceptions of “professionalism” and “gravitas” are often based on white male characteristics. Have you ever been on a search committee when a candidate’s passion was cited as evidence of a lack of emotional control? How about when a quiet and reserved candidate was labeled “uninspiring” or “obviously not interested” in the position? I have observed introverted thinkers — who paused a beat before responding — get tagged as “too tentative” or “less prepared” than other candidates. Be open to the possibility that you and your search-committee colleagues are evaluating a candidate’s style based on what makes you comfortable rather than what is essential for the job you are seeking to fill.
  • Think about which candidates might offer new ways of thinking about old problems. Give greater consideration to candidates who are most likely to challenge assumptions, interrupt default ways of thinking, force difficult debates over complex issues, and ask “Why?” over and over again. While it may not be easy or even pleasant, constructive conflict typically yields better analysis and results than comfortable conversations do.

Spending time in search committees exploring the criteria to be used in evaluating candidates is a valuable strategy for interrupting decision-making patterns that give some groups advantages over others. These are not easy conversations, but they can reveal that we tend to favor candidates for leadership roles based on our default vision of what a leader looks like rather than being open to the possibility that we might benefit from something entirely different.

A version of this article appeared in the March 5, 2021, issue.
We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Leadership & GovernanceFinance & Operations
Allison M. Vaillancourt
Allison M. Vaillancourt provides organizational consulting services as a vice president and senior consultant in Segal’s organizational effectiveness practice. Prior to joining Segal, she spent three decades as an administrator and faculty member at large public research universities. Browse her previous columns in the Management Corner series on administrative-career issues. She is on Twitter @VaillancourtA.
ADVERTISEMENT
ADVERTISEMENT
  • Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
    Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
  • The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
    The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
  • Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
    Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
  • Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
    Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
1255 23rd Street, N.W. Washington, D.C. 20037
© 2023 The Chronicle of Higher Education
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin