> Skip to content
FEATURED:
  • The Evolution of Race in Admissions
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
  • News
  • Advice
  • The Review
  • Data
  • Current Issue
  • Virtual Events
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Career Resources
    • Find a Job
    • Post a Job
    • Career Resources
Sign In
free-report-banner03.png

Puzzle pieces art

The Puzzle of Student Data

Key strategies for using critical information, responsibly.

Strategic Thinking
  • Twitter
  • LinkedIn
  • Show more sharing options
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
  • Copy Link URLCopied!
  • Print
By  Taylor Swaak

When information-technology and higher-education leaders reunited in Philadelphia in October 2021 for the first in-person Educause conference in two years, buzzwords floated around the panel rooms:

“Data analytics.” “Data-informed” action. “Innovation.”

Indeed, college leaders nationwide — many of them still shouldering deep state-budget cuts that followed the 2008 recession — are increasingly looking to harness data, especially about their students, to make strategic and cost-effective decisions. They’re also facing increasing pressure from the public and social activists to close equity gaps for marginalized students and improve student-success metrics such as retention and graduation rates.

We’re sorry. Something went wrong.

We are unable to fully display the content of this page.

The most likely cause of this is a content blocker on your computer or network. Please make sure your computer, VPN, or network allows javascript and allows content to be delivered from c950.chronicle.com and chronicle.blueconic.net.

Once javascript and access to those URLs are allowed, please refresh this page. You may then be asked to log in, create an account if you don't already have one, or subscribe.

If you continue to experience issues, contact us at 202-466-1032 or help@chronicle.com

When information-technology and higher-education leaders reunited in Philadelphia in October 2021 for the first in-person Educause conference in two years, buzzwords floated around the panel rooms:

“Data analytics.” “Data-informed” action. “Innovation.”

Indeed, college leaders nationwide — many of them still shouldering deep state-budget cuts that followed the 2008 recession — are increasingly looking to harness data, especially about their students, to make strategic and cost-effective decisions. They’re also facing increasing pressure from the public and social activists to close equity gaps for marginalized students and improve student-success metrics such as retention and graduation rates.

This reality, experts say, has sparked a shift in the past decade to “predictive” and “prescriptive” data use. Previously, colleges collected students’ graduation rates, demographics, financial-aid, and other data largely because this data was required to operate and meet federal reporting requirements. Now, they’re also using data for predictive analytics, adopting early-alert systems that flag advisers when students show signs of dropping out. Faculty members can now evaluate and adjust their learning materials by tracking how often, and for how long, students engage with them in learning-management systems. ID-card-swipe data can determine which facilities students are using most often.

free-report-box02.png

Done well, analyzing and acting on student data can boost student performance and deliver returns on investment. Georgia State University’s adoption of predictive analytics in advising, for example, has helped raise its graduation rate eight percentage points since 2010, with each percentage point adding $3.18 million in revenue annually. Amarillo College, a community college in rural Texas, has leaned on online student surveys to inform its distribution of emergency aid, nearly doubling its graduation and transfer rate in five years.

Still, the use of data to drive decisions is a fairly new shift for many in higher education, and colleges are weighing their ability to know more than ever about their students against the ethical and privacy-related risks that come with amassing more and more information — including through vendor partnerships.

What Data Should You Collect to Begin With?

In a seemingly endless universe of data, what to collect beyond what’s required can be tricky — especially if a college is strapped for resources. Experts offer this advice for college leaders:

  • Have a clear strategic mission that outlines your college’s student- success goals.
  • Think about how deep you want your student-data insights to go. Are you OK with top-line trends, or do you want to know students in a more disaggregated way?
  • Given your mission and goals, what do you believe you actually need in order to answer the questions at hand? Aim for data minimization.

“We have a lot more” data, but “that doesn’t mean it’s better,” says Loralyn Taylor, associate provost for institutional effectiveness and analytics at Ohio University. “It actually makes it harder for institutions to separate the wheat from the chaff in terms of what’s actually useful.”

Flashy tech alone won’t ensure success. Experts say student-data collection and analysis done in the absence of a supportive campus culture, data-defining governance boards, faculty data-literacy training, and transparency, among other things, can actually do more harm to students than good.

“Every school has to have a comprehensive plan for how they are going to change” to do this work right, says Allison Calhoun-Brown, senior vice president for student success and chief enrollment officer at Georgia State. “The innovation is not the technology. The innovation is the change that accompanies the technology.”

Risks to Consider

View this report as a PDF


student-data-free.jpg

With predictive analytics, the ethical dilemma to keep in mind is that they may not capture students’ unique circumstances and mind-sets. “The model, by definition, is based on every person that came before [you],” Laura Jensen, vice provost for planning and effectiveness at Colorado State University, says. “It’s not based on [you] in any way, shape, or form.”

With enrollment algorithms, colleges risk “widespread missteps” and discrimination when they focus too intently on data like ZIP codes, GPA, and test scores, and on engagement metrics such as attendance at recruitment events, adds Amelia Parnell, vice president for research and policy at the National Association of Student Personnel Administrators.

These algorithms “don’t know my parents, they don’t know my friend circle, they don’t know my study patterns,” says Parnell. “It’s a tool to be used in a suite of other pieces for a decision. I know it’s tempting when you get a model that continues to come back with certain levels of predictability, to say, ‘Oh, let’s just go there first.’ … [But] that’s where you miss the richness” of a more diverse population.

Behavioral analytics, too, can form an incomplete picture. The University of Iowa, for example, collected data on a semester’s worth of ID-card swipes at the dining hall, generating a “connectedness” score for students based how many meals the students had, and with how many different people. Not addressed with that score, though, is the reality that a student with one or two close friends but no broader network can have a low score but still be happy.

red6
Further Reading
  • This College Is Asking Personal Questions — and the Answers Have Helped Boost Completion Rates
  • The Surveilled Student
  • As Cyberattacks Roil Colleges, Many Look to Faculty Members and Students for Help

This all underlines the importance of making sure analysis isn’t happening in a vacuum with a machine, experts say. Human beings should still always be reviewing analyses and considering context.

Nothing with data is “cookie cutter,” says Vanessa Roof, assistant director of enterprise data solutions at the University of Nebraska system. Colleges should still ask “and then what?” and have conversations.

The Role of Faculty Data Literacy

Improper analyses are also less likely the more data-literate faculty and staff members are.

Faculty members are constantly surrounded by data. Professors can track their own students’ performance and activity in learning-management systems. Depending on the institution, they may have ready access to aggregated institutionwide data filterable by student demographics and fields of study.

But the ability to make responsible use of that data isn’t an ingrained skill: In the 2018 AIR National Survey of IR Offices, 59 percent of respondents were either neutral or disagreed with the statement that faculty data-literacy levels were “high” at their institution.

“Data literate,” as a simplified definition, means the ability to access, interpret, and communicate data in context. It means knowing to ask questions that foster critical thinking rather than cherry-picking data to justify a decision that has already been made. It means knowing the difference between correlation and causality.

Having faculty members who aren’t data literate can come with repercussions. At Dartmouth College, misinterpretation of Canvas data spurred false cheating allegations against 17 medical students. Or it can be less overt; a faculty member may simply not attempt to draw on data at all, either not knowing it exists or how to access it, or finding it overwhelming rather than beneficial.

That’s a problem because “faculty are key” in this work, says Nic Richmond, chief strategist and vice chancellor for strategy, analytics, and research at Pima Community College. “In terms of the people our students have the most contact with, it’s their instructors in the classroom. … There’s nothing like being in a class, building a relationship directly with a student. … You get to know them.”

What’s Effective in Faculty Training

Ellen Mandinach, director of the Data for Decisions Initiative at WestEd, a nonprofit research and development company, has compiled dozens of data-literacy skills that should be covered when training faculty members. Some of the most pressing for her:

  • Knowing when you have bad data. Is the data complete? Is it timely/recent? Is it from a reliable source?
  • Responsibly using data. Are you asking the right question about the data? Is it granular enough? Do you have everything you need to make a decision?
  • Communication. Can you communicate to a student, to other faculty members, what the data is showing?

Just as with data collection, experts say a campus culture that encourages data use and has set institutionwide expectations — and time — for faculty professional development is critical. It also helps to deliver a clear message that the chief reason for using data is to better serve students, and not to weed out and punish a faculty member whose students are underperforming, Richmond says.

Pima Community College’s Actions

  • Adopted a business-intelligence platform with reports on enrollment trends, persistence, and retention.
  • Offers on-demand training for faculty members on how to access and analyze the data.
  • Emphasizes that focus is on better serving underperforming students, not punishing faculty members.

Key Result
About 80 faculty members had taken a custom training as of December 2021, helping them to further inform their teaching. The college’s goal is to reach 50 percent of its nearly 250 full-time faculty members by the end of this academic year.

At Pima, faculty members have a wealth of data “at their fingertips.” The college, which served about 33,000 students in 2020-21, used capital funding to adopt a business-intelligence platform in 2015 — called “Pima Reports” — that includes folders of interactive data reports on topics such as enrollment trends, persistence, and retention.

Faculty members can “slice and dice” that data by factors like student demographics, subject code, course number, and instructional delivery method, Richmond says. More personally identifiable and sensitive data, such as grade distributions and financial aid, are kept in more restricted folders.

But simply having this platform doesn’t mean faculty members will know how to access or analyze it, she notes. And that’s where data-literacy training has come in.

Pima launched an analytics academy in fall 2019: a cohort of 20 volunteer administrators and faculty and staff members who met for four days over the course of the semester. Sessions included how to navigate the Pima Reports system, where to find existing student- and faculty-engagement surveys, an overview of mandated data the college collects for Ipeds, and a crash course on the ways faculty members should and shouldn’t use data.

Then Covid-19 hit. In-person training was no longer an option, and Pima was juggling myriad competing priorities that Richmond admits sometimes placed data training on the back burner.

For now, Richmond’s division is offering a monthly, two-hour session virtually — one hour of training, one hour of hands-on practice — on how to use the Pima Reports platform. Leaders run through how to log in, where to locate the reports, and how to filter and search them. They’re also offering this service on demand and customized for individuals, teams, or departments that request training.

Richmond said about 80 faculty members (including adjuncts) had taken a custom training as of December. The college’s goal is to reach 50 percent of its nearly 250 full-time faculty members by the end of this academic year.

The analytics academy is likely to return in some form, Richmond adds, but discussions are still underway about who should take it, the timing and intensity of the sessions, and the amount of hands-on practice needed to be effective.

Brooke Anderson, who teaches writing and humanities courses, took Pima Reports training in October. She says accessing and disaggregating data has provided evidence of what she’s already observed in the classroom: that much teaching on campus is still benefiting the “traditional student,” notably, those who are full-time, “economically comfortable,” white, and male. Those students tend to do better in composition classes, which traditionally dock points for failure to use standard, written English.

The data will inform how she teaches from now on, Anderson says, reinforcing her “student-centered and student-directed” approach. For example, she and her students develop the rubric for writing assignments together. She doesn’t penalize them for grammar or typos — though she’s happy to provide guidance if they ask.

“When they do not see themselves as the problem, they are more likely to not interpret faculty feedback and grades as a reason to drop out of school or not pursue a field that interests them,” she says.

Knowing more about their students allows faculty members to take action. They can make changes as simple as pulling a student aside to ask what support they need to succeed, or swapping in new materials, and as complex as reconfiguring a lesson plan or redesigning a course.

Taking action could also mean piloting new tools and scaling up gradually. The University of Central Florida, for example, expanded the use of adaptive-learning software — which crafts personalized learning paths for students based on their performance — after a pilot program in a few online foreign-language courses saw a 23-percent increase in the number of students earning A, B, or C grades.

Institutional Action

Action can even expand to the institutional level, as with Georgia State, which tracks 800 factors the university has identified as putting students at academic risk.

An early-alert system lets an adviser know when a student displays any one of those risk factors, and the adviser reaches out to the student within 48 hours. An adviser might call, text, or email, for example, if a student has enrolled in both an organic-chemistry course and a calculus class in the same semester — a combination that’s considered “toxic.”

Protecting the Data You Do Have

The more data a college collects, the more is at stake if the data is poorly defined, mishandled, or subject to a breach. Colleges can be proactive by:

  • Ensuring collegewide data governance, or protocols around the ownership, definitions, usage, storage, protection, and destruction of student data.
  • Working toward creating central repositories for data, such as with a data warehouse.
  • Confirming they retain ownership of all of their students’ data when they enter third-party vendor partnerships.
  • Prioritizing cybersecurity. While tech considerations such as multifactor authentication are important, colleges should also allocate resources to help faculty members and students adopt cyber-safe habits (like recognizing phishing emails).

“When we’ve had innovations at Georgia State, they haven’t been because we had the luxury to say, ‘We have enough time [and resources] to innovate,’ it’s because we haven’t had the luxury not to innovate,” says Tim Renick, executive director of the National Institute for Student Success at Georgia State.

Other colleges have found themselves in a similar position of adopting campuswide initiatives to improve student success and retention.

At Hillsborough Community College, a chatbot named Gwen will text students who’ve stopped attending classes with 15 credits or fewer left in their degree, offering assistance for course scheduling, setting up advising appointments, or filling out a financial-aid application.

Many colleges are distributing emergency aid to students who have unpaid balances or financial holds on their accounts, recognizing that more than one-third of college dropouts leave because of financial pressures. Dillard University, in New Orleans, La., for example, built up a philanthropic fund that provides microgrants averaging $2,500 per academic year to students struggling with the cost of attendance.

Another historically Black college, Morgan State University, in Baltimore, homed in on sophomore retention as a way to boost its graduation rates. The university runs a Second Year Experience program that encourages sophomores to participate in either an internship, a research experience, study abroad (pre-Covid-19), a service-learning course, or a campus-to-career trip to, for example, a local radio station or museum.

Morgan State University’s Actions



  • Used data to identify sophomore Pell Grant recipients as a priority for retention efforts.
  • Launched outside-the-classroom “experiential learning” opportunities, such as internships and campus-to-career trips.
  • Collaborated and shared best practices with two other HBCUs — Dillard and Howard Universities — that were doing similar work.

Key Results



  • Morgan State saw its four- and six-year graduation rates reach historic highs, increasing nine percentage points and 13 percentage points, respectively, between May 2016 and May 2020.
  • The percentage of Pell Grant recipients taking 30 credits a year rose to 28 percent from 21 percent.

These opportunities are about “getting students to see the importance of what they’re doing in the classroom,” says Kara Turner, Morgan State’s vice president for enrollment management and student success.

Between May 2016 and May 2020, Morgan State saw its four- and six-year graduation rates reach historic highs, increasing nine percentage points and 13 percentage points, respectively.

Keeping It Sustainable

Resources aren’t limitless, though,and colleges with tight budgets may worry more broadly about how financially sustainable investments in data and analytics are.

Georgia State’s chatbot, through the engagement platform Mainstay, costs about $4 per student a year, a rate the company says is “fairly typical.” The University of Central Florida paid $150,000 in 2020-21 for student licenses for its main adaptive-learning system, Realizeit — roughly $30 per student.

Both Renick and Allison Calhoun-Brown, Georgia State’s senior vice president for student success and chief enrollment officer, emphasize that colleges don’t have to break the bank to do this work. Georgia State, they note, isn’t collecting 800 “discreet personal details” about every single student. The 800 risk factors are based on data the college has long collected in order to be accredited: registration records, majors, minors, degree programs, attendance, and midterm grades, to name a few.

Outside partnerships can also fill resource gaps. Amarillo College, which dispatches a 10-question basic-needs survey to students each semester, leans on more than 60 partners, including lawyers doing pro bono work and staff at a nonprofit health-care facility, to get students the assistance they need. (While it’s drawn from $15.6 million in federal stimulus aid to help meet students’ needs for things like stable housing and child care, that money will run out.)

What’s most important, Renick and Calhoun-Brown say, is the approach — the decision to be proactive rather than reactive.

Taking initiative for identifying and providing help rather than “waiting for students to diagnose their own problems and figure out where to get help … that methodology works at all levels,” says Renick.

Communicating With Students

No matter a college’s resources, experts emphasized that no action will be successful if it’s communicated poorly to students.

“Communicating with students can seem easy, intuitive, and low stakes,” the former New America policy analyst Alejandra Acosta wrote in a multipart 2020 report. “But having good intentions does not guarantee success, and ineffective communication can cause serious harm.”

It helps if the information a college gathers translates clearly and quickly into action to support the student. At Amarillo, for example, Claudia Rubi Zaragoza Hernández, a sophomore, told The Chronicle in the fall that she and friends she’s spoken with aren’t bothered by the college’s questions on topics like food and housing security. “If they collected data and we don’t know why and the college doesn’t do anything with that information, I think we’d be more concerned,” she said. “But at least here … we get benefits from it.”

Especially when communicating sensitive information — perhaps a student is at risk of failing a class or is behind on payment — a message meant to inform and guide can come across as invasive and creepy if framed incorrectly. Action based on predictive analytics can also, if not communicated properly, give students a feeling of being pigeonholed or doomed to a self-fulfilling prophecy.

“Don’t tell me what I can or can’t do,” one student said in a case study published in 2019 that discussed colleges using predictive data to recommend course pathways. “Who can decide my future besides myself?” another asked.

It can even feel discriminatory. Amelia Parnell, of Naspa, recalls an anecdote she heard at a conference a few years ago about a Black man who was feeling “targeted” by “constant emails” about support services. “The campus had made the decision that they were going to focus [outreach] on Black men who were enrolled because they wanted to retain them, because the numbers were showing they were not performing as well,” Parnell says. “And he was like, ‘They don’t know me. They don’t know that both my parents went to college. My dad’s a doctor. I know how to do college.’”

The equity implications expand beyond threatening students’ feelings of belonging. Messaging without clear instructions — perhaps on how to make an advising appointment or fill out a financial-aid form — could confuse a first-generation student who needs these services urgently “but does not know how to schedule one or have relatives and friends who can help,” Acosta wrote in her report.

So what to do? Acosta offered a few strategies for effective communication:

  • Social proof. Students are more likely to take action if their college highlights that other students are doing it too.
  • Loss aversion. Making students aware of what they may lose as a result of inaction (in a positive way) is likely to encourage them to change their behavior.
  • Minimizing hassle. Offer clear and straightforward instructions so that students know exactly what they have to do, how to do it, and can act immediately.
  • Timing. Periods such as course registration, midterms, or financial-aid deadlines may be the optimal time to reach students. Holiday break or summer vacation? Less so.
  • Preserving choice. Avoid messaging that suggests students have no other option if they don’t take a certain action. Some may not be able to.

And even after all of that action, evaluate.

Colleges should be circling back and evaluating any action they take, says Denise Nadasen, the Association of Public and Land-Grant Universities’ assistant vice president for data and policy analysis — even if they’ve followed all the right steps for responsible data collection and use on the front end. It will ensure the most efficient use of college’s limited resources.

“If we have evidence to show that they are [working], then we should continue supporting them and putting resources there,” she says. “If they are not … then we need to step back, and find out why.”

We welcome your thoughts and questions about this report. Please email the editors or submit a letter for publication.
TechnologyStudent SuccessLeadership & Governance
Taylor Swaak
Taylor Swaak is a staff reporter at The Chronicle of Higher Education, covering how innovations in technology are changing the student experience. She aims to hold institutions accountable for technology that is misused or contributes to inequity, as well as uplift success stories that could inspire other ideas.
  • Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
    Explore
    • Get Newsletters
    • Letters
    • Free Reports and Guides
    • Blogs
    • Virtual Events
    • Chronicle Store
    • Find a Job
  • The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
    The Chronicle
    • About Us
    • DEI Commitment Statement
    • Write for Us
    • Talk to Us
    • Work at The Chronicle
    • User Agreement
    • Privacy Policy
    • California Privacy Policy
    • Site Map
    • Accessibility Statement
  • Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
    Customer Assistance
    • Contact Us
    • Advertise With Us
    • Post a Job
    • Advertising Terms and Conditions
    • Reprints & Permissions
    • Do Not Sell My Personal Information
  • Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
    Subscribe
    • Individual Subscriptions
    • Institutional Subscriptions
    • Subscription & Account FAQ
    • Manage Newsletters
    • Manage Your Account
1255 23rd Street, N.W. Washington, D.C. 20037
© 2023 The Chronicle of Higher Education
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin