> Skip to content
FEATURED:
  • The Evolution of Race in Admissions
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
ADVERTISEMENT
News
  • Twitter
  • LinkedIn
  • Show more sharing options
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Copy Link URLCopied!
  • Print

Productivity Metrics

What is the best way to assess faculty activity?

By  Vimal Patel
February 29, 2016
Productivity Metrics 1
Eric Petersen for The Chronicle

Questions about faculty productivity are nothing new. But the growing use of metrics to assess faculty activity has raised the stakes at a time when colleges already face growing pressure to demonstrate accountability and compete with peer institutions.

Meanwhile, questions about how to measure a scholar’s influence in social media, known as “altmetrics,” are expected to add to the debate over faculty productivity.

One company that colleges turn to for help with metrics is Academic Analytics, which allows colleges to compare the peer-reviewed publications, journal citations, federal research grants, and other honors of their faculty members with those at peer institutions. The company says it has more than doubled the number of colleges it works with, to about 100, over the past five years.

We’re sorry. Something went wrong.

We are unable to fully display the content of this page.

The most likely cause of this is a content blocker on your computer or network. Please make sure your computer, VPN, or network allows javascript and allows content to be delivered from c950.chronicle.com and chronicle.blueconic.net.

Once javascript and access to those URLs are allowed, please refresh this page. You may then be asked to log in, create an account if you don't already have one, or subscribe.

If you continue to experience issues, contact us at 202-466-1032 or help@chronicle.com

Productivity Metrics 1
Eric Petersen for The Chronicle

Questions about faculty productivity are nothing new. But the growing use of metrics to assess faculty activity has raised the stakes at a time when colleges already face growing pressure to demonstrate accountability and compete with peer institutions.

Meanwhile, questions about how to measure a scholar’s influence in social media, known as “altmetrics,” are expected to add to the debate over faculty productivity.

One company that colleges turn to for help with metrics is Academic Analytics, which allows colleges to compare the peer-reviewed publications, journal citations, federal research grants, and other honors of their faculty members with those at peer institutions. The company says it has more than doubled the number of colleges it works with, to about 100, over the past five years.

At public colleges, the pressure to measure faculty productivity often comes from legislators. But all types of institutions are increasingly paying more attention to remaining competitive with their peers, says Peter Lange, chief academics adviser at the South Carolina-based company and a former provost of Duke University.

Such efforts haven’t always been executed with finesse. Texas A&M University, for example, issued a report in 2011 that listed faculty members’ names in red or black — like a corporate balance sheet — depending on whether the research and tuition dollars they generated covered their salary and expenses. Such heavy-handed efforts usually crumble under faculty opposition. Texas A&M abandoned its plan amid faculty objections to the perceived corporatization of the university as well as the accuracy of the data.

ADVERTISEMENT

Winning over faculty members — or at least avoiding a revolt — is key to the long-term success of evaluation efforts, and administrators must strike a balance between their needs and faculty concerns.

“The most important thing,” says Gary A. Olson, president of Daemen College, “is to make sure you have faculty buy-in. If you have them helping in the production of the measurement instrument, you have the best chance of coming up with an instrument that everybody’s happy with.”

Pleasing everyone, though, may be impossible. Many faculty members, especially in the arts and humanities, are distrustful of faculty analytics.

“They’re trying to run creative thinking through a machine,” says Mark Usher, chair of the classics department at the University of Vermont. While the university works with Academic Analytics, it does not require the use of company data for evaluation, and recently asked each of its academic units to develop its own faculty-productivity metrics.

Mr. Usher says the metrics aren’t meaningful without context and often aren’t even accurate. He echoes faculty members on many campuses who have complained that reports based on metrics often show deflated grant awards and incorrect journal citations, and omit publications that should be included (and vice versa).

ADVERTISEMENT

For example, he says, Academic Analytics had included Acta Astronautica, a publication he’d never heard of, as a classics journal. “What is that?” he asks, “Like, the study of UFOs?” (It’s an astronautics journal.)

Company officials acknowledge that assessing faculty productivity in the arts and humanities is tougher than in the sciences and engineering, where quantified measurements are the norm.

TAKEAWAY

Can Metrics Measure Professors?

  • Colleges are increasingly using data to measure their faculty members’ productivity and to compare them with professors at peer institutions. Professors complain that such metrics provide an inaccurate and incomplete picture of their activities, but colleges say the careful use of data from an outside source provides credibility.
  • Gaining faculty approval is key to the long-term success of any effort to measure faculty productivity.
  • Administrators need to be sensitive to disciplinary differences: A metric that works in civil engineering might not work in English or, for that matter, chemical engineering.
At the Massachusetts Institute of Technology, visiting committees, comprising members from academe, business, industry, and government, have reviewed each department since the 19th century. These days, those committees receive data compiled by Academic Analytics, which allows comparisons with peers. But administrators must be cognizant of disciplinary differences, says Lydia Snover, director of institutional research at MIT, and must put the data in context with other indicators of productivity, which the company’s data don’t measure.

Peer-reviewed publications may be an effective productivity metric for some departments. For others, like computer science, which produce fewer papers, the metrics might be citations per publication, or conferences per faculty member, or honors and awards. Federal grants could be a productivity metric for a department like chemistry, but not for engineering, since engineering faculty members at MIT receive a large share of grants from private sources that aren’t captured by Academic Analytics’ data.

Moreover, Ms. Snover says, colleges must make clear that faculty-productivity metrics will be placed in the context of a faculty member’s broader body of work. “A lot of this,” she says, “is just to be able to provide some comparative data that isn’t hearsay. It’s not perfect. It’s impossible to be perfect in these areas.”

ADVERTISEMENT

Some say the next faculty-productivity battlefield might be altmetrics, a term used to describe alternative methods of gauging scholarly impact, including the use of blogs, news coverage, and social media. How many times was a tweet about your research retweeted, or “liked” on Facebook? Such measures have made headway in Britain but are still a gray area in the United States, says Anthony J. Olejniczak, chief knowledge officer of Academic Analytics.

“It’s not exactly clear where that line between what is scholarly and what is media is ultimately going to be settled,” he says. “But a blurring of that line is clearly something that has been happening in the last few years.”

Proponents of faculty analytics say the quality and accuracy of data have improved and will continue to do so as technology evolves. But the debate over how meaningful those data are won’t be settled anytime soon.

Vimal Patel covers graduate education. Follow him on Twitter @vimalpatel232, or write to him at vimal.patel@chronicle.com.

A version of this article appeared in the March 4, 2016, issue.
Read other items in this The Trends Report: 10 Key Shifts in Higher Education package.
We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Vimal Patel
Vimal Patel, a reporter at The New York Times, previously covered student life, social mobility, and other topics for The Chronicle of Higher Education.
ADVERTISEMENT
ADVERTISEMENT

Related Content

  • Commentary: Academic Analytics: Buyer Beware
  • How One University Measured Faculty Productivity — and Nobody Got Hurt
  • Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
    Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
  • The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
    The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
  • Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
    Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
  • Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
    Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
1255 23rd Street, N.W. Washington, D.C. 20037
© 2023 The Chronicle of Higher Education
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin