Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Virtual Events
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Virtual Events
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    An AI-Driven Work Force
    AI and Microcredentials
Sign In
Student Success

Dean at UC-Irvine Seeks to Solve Higher Ed’s Existential Crisis With Data — Lots of It

By Steven Johnson November 27, 2018
Richard Arum
Richard Arum

In 2011 the sociologist Richard Arum bemoaned the lack of rigor in American higher education.

In 2016 he critiqued the “inadequate and wrongheaded approaches,” especially those focusing on labor-market data, that drive most studies of student outcomes.

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

Richard Arum
Richard Arum

In 2011 the sociologist Richard Arum bemoaned the lack of rigor in American higher education.

In 2016 he critiqued the “inadequate and wrongheaded approaches,” especially those focusing on labor-market data, that drive most studies of student outcomes.

Now, as dean of the University of California at Irvine’s School of Education, he gets to design his own study, leading a new effort to track students’ academic, social, and personal behaviors to gauge their success.

Irvine is the pilot site of a longitudinal study aiming to rethink how colleges measure their own students’ success, thanks to a $1.1-million grant from the Andrew W. Mellon Foundation. What researchers learn at Irvine will inform a national study of several colleges, managed by a consortium at the University of Michigan at Ann Arbor.

“Every other industry in the world right now has figured out a way to use data to better serve customers and clients, and meet their needs and improve institutional performance,” said Arum, the project’s lead investigator. Although higher education has pushed for more “evidence-based decision making,” he said, “we’re a real laggard in using those techniques and tools to improve our own work.”

Academics, in the liberal arts in particular, have been divided on how to best justify their usefulness to the public — or whether such justification is even necessary. In recent years, partisan divisions have expanded to the point that simply earning a college degree is now seen as both a marker and a driver of political affiliation, troubling a higher-education world already unnerved by declining public funding. The Irvine project seeks to meet skeptical policy makers on their home turf: measurement.

“In a society hungry for data and evidence, simply claiming these values of the liberal arts no longer suffices,” said Mariët Westermann, a vice president at the Mellon Foundation, in a news release. Modeling and measuring student outcomes won’t just prove higher ed’s worth to the public, the thinking goes — it will also improve academic quality, because it will help colleges hone their practices.

New Methods to Measure Student Success

The study, called the Next-Generation Undergraduate Success Measurement Project, will trawl depositories of student data, combining institutional records with survey responses and data from learning-management systems like Canvas and Blackboard. Researchers hope to determine what practices affect how students learn, grow, and enter the work force.

ADVERTISEMENT

In the first year, Arum’s team will gather experts from more than a dozen institutions, including Michigan, Columbia University, and Indiana University, to flesh out the study’s design.

One of those experts is Charles F. Blaich, director of Wabash College’s Center of Inquiry. He led the Wabash National Study of Liberal Arts Education, a similarly ambitious research effort that used multiple data sources to track student pathways. He has noted the challenges of studying student outcomes.

Those decade-old studies relied on unwieldy surveys, longer time frames, and full-time freshman populations, Blaich said. But faster methods have sprung up to measure student success. And the new effort at Irvine is poised to take advantage of them.

For example, the advent of data-rich learning-management systems, which can track a student’s every moment and click on assignments, adds a layer of information that many colleges have yet to examine. And Irvine’s study of two cohorts, freshmen and juniors, as well its substantial mix of juniors who transferred from California’s community colleges, help shift the focus from traditional first-time freshmen, a less representative slice of college students than they used to be.

ADVERTISEMENT

In the project’s second and third years, Arum’s team will collect data on 1,050 Irvine students — 500 freshmen, 500 continuing and transfer juniors, and 50 freshman-honors students — and will track not just how they perform academically but also, to a modest degree, what they do outside of course work.

“We make a mistake to think that it’s all about academics in higher education,” Arum said. Students spend most of their time socializing, in extracurricular activities, at work, or “at a place like UCI, time with their family, commuting from home.”

As part of the project, in exchange for a small stipend, a freshman in Irvine’s honors program might fill out a one-minute survey pinged to her smartphone that asks whom she’s with and what she’s doing, Arum said. That could happen 10 times, at random, in a given month.

The goal is not to get an exact picture of what each student is doing at any given time. Rather, if you “multiply that by the number of students you have in the project,” Arum said, “all of a sudden you have a very detailed description of what the student body as a whole is doing, and how they’re feeling about it.”

By combining that “experiential” information with routine administrative data, like a student’s demographic profile and course progression, as well as data from a learning-management system, like the writing assignments a student uploads to Canvas, the project will seek to stitch together an aggregate picture of how students move through an undergraduate education.

The Variables That Matter

It’s too early to use those data to tailor education to each individual student, Arum said. Rather, he plans to use the information to identify what variables matter in measuring student success, and to improve practices at an institutional level. At Irvine, that may mean working with faculty members to streamline how they teach and organize their curricula.

ADVERTISEMENT

Arum sees health care, businesses, and public administration as models for this multifaceted approach. Hospitals, for example, track mortality and infection rates to improve treatment units and health systems, but they’re on their way to personalizing medicine to individual patients. Colleges, meanwhile, are “just starting” to pull together data from across campus units, and often don’t yet know what variables to track, he said.

“These are kind of the methods you want to be using in public administration, in social-welfare organizations,” Arum said. And they’re just the kinds of methods that he and the funders at the Mellon Foundation hope can better inform colleges as they enter the next few years of public debate.

Follow Steven Johnson on Twitter at @stetyjohn, or email him at steve.johnson@chronicle.com.

A version of this article appeared in the December 7, 2018, issue.
We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
Scholarship & Research
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
About the Author
Steven Johnson
Steven Johnson is an Indiana-born journalist who’s reported stories about business, culture, and education for The Chronicle of Higher Education, The Washington Post, and The Atlantic.
ADVERTISEMENT
ADVERTISEMENT

More News

Photo illustration showing Santa Ono seated, places small in the corner of a dark space
'Unrelentingly Sad'
Santa Ono Wanted a Presidency. He Became a Pariah.
Illustration of a rushing crowd carrying HSI letters
Seeking precedent
Funding for Hispanic-Serving Institutions Is Discriminatory and Unconstitutional, Lawsuit Argues
Photo-based illustration of scissors cutting through paper that is a photo of an idyllic liberal arts college campus on one side and money on the other
Finance
Small Colleges Are Banding Together Against a Higher Endowment Tax. This Is Why.
Pano Kanelos, founding president of the U. of Austin.
Q&A
One Year In, What Has ‘the Anti-Harvard’ University Accomplished?

From The Review

Photo- and type-based illustration depicting the acronym AAUP with the second A as the arrow of a compass and facing not north but southeast.
The Review | Essay
The Unraveling of the AAUP
By Matthew W. Finkin
Photo-based illustration of the Capitol building dome propped on a stick attached to a string, like a trap.
The Review | Opinion
Colleges Can’t Trust the Federal Government. What Now?
By Brian Rosenberg
Illustration of an unequal sign in black on a white background
The Review | Essay
What Is Replacing DEI? Racism.
By Richard Amesbury

Upcoming Events

Plain_Acuity_DurableSkills_VF.png
Why Employers Value ‘Durable’ Skills
Warwick_Leadership_Javi.png
University Transformation: a Global Leadership Perspective
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Virtual Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin