Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    College Advising
    Serving Higher Ed
    Chronicle Festival 2025
Sign In
Teaching

Grades and Tests May Miss Measuring What Matters Most in Learning

By Dan Berrett November 15, 2012
“Deep approaches” to learning figure prominently in the new edition of the National Survey of Student Engagement.
“Deep approaches” to learning figure prominently in the new edition of the National Survey of Student Engagement.Melanie Burford for The Chronicle

As pressure mounts on colleges to document what their students learn, it remains tough to judge from outside the classroom how much knowledge they gain from their academic experience.

The traditional measure of learning is the course grade. Nothing says academic success more succinctly than an A.

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

As pressure mounts on colleges to document what their students learn, it remains tough to judge from outside the classroom how much knowledge they gain from their academic experience.

The traditional measure of learning is the course grade. Nothing says academic success more succinctly than an A.

But an A is subjective. Skeptics note that course requirements vary depending on the professor, the department, and the institution. Grades are often inflated.

Alternative methods to document learning have arisen in the form of standardized tests of critical thinking, which are meant to assess students’ ability to analyze material at a collegiate level. The strength of such tests is in their ability to provide results that can be compared across institutions.

But what if neither of those methods says much about the teaching, expectations, and assignments that students encounter in their courses?

According to this view, the nature of teaching and learning should be measured instead of relying solely on an outcome like a grade or a test. Students should be exposed to courses and assignments that require them to analyze information and apply it to new contexts, reflect on what they know, identify what they still need to learn, and sort through contradictory arguments.

Such opportunities are described in research literature as “deep approaches to learning.” They figure prominently in Thursday’s release of data from the National Survey of Student Engagement. While Nessie, as the survey is known, has long sought data on those practices, this year’s report replicated and extended the previous year’s findings, which showed that participation in deep approaches tends to relate to other forms of engagement, like taking part in first-year learning communities and research projects.

Deep approaches have been a subject of education research since at least the mid-1970s, after Ference Marton’s and Roger Säljö's pioneering work in Sweden. Those educational psychologists analyzed how students responded to an academic article. They found that one group used “surface” approaches, like rote memorization, while the other took “deep” ones, in which students sought to understand the material’s purpose, meaning, and significance.

Students responding to Nessie were asked to describe how often—very much, quite a bit, some, or very little—during the current academic year they had analyzed the basic elements of an idea; synthesized information into new and more-complex interpretations; judged the value of an argument; and applied concepts to new contexts. The goal is for colleges to use the results to drive improvements in how they deliver undergraduate education.

“The stuff we measure in Nessie is process,” said Alexander C. McCormick, the survey’s director and an associate professor in the School of Education at Indiana University at Bloomington. “College students should be having experiences that call upon them to use more higher-order thinking, regardless of whether or not it connects to given tests. Students should have to do more analysis and synthesis than memorization.”

ADVERTISEMENT

A big divide emerged among the seniors who participated in the survey. Those who reported engaging most regularly in deep approaches to learning did so more than twice as often as did those in the bottom quartile.

The different levels of participation in the deep approaches manifested themselves in other behaviors that facilitate learning. Seniors in the top quartile in participating in deep approaches, for example, spent about five more hours per week preparing for class than did their peers at the bottom.

Chicken-and-Egg Problem

Nessie’s measures of deep learning figure prominently in other newly released research, in which the value of those measures is both reinforced and challenged.

Corbin M. Campbell, an assistant professor of higher education at Columbia University’s Teachers College, and Alberto F. Cabrera, a professor of higher education at the University of Maryland at College Park, looked at responses to Nessie’s deep-learning questions from about 1,000 students at a large, public research institution.

ADVERTISEMENT

Ms. Campbell and Mr. Cabrera found that the three deep approaches that Nessie measures—higher-order, integrative, and reflective learning—were tightly interconnected and resulted in deep learning, as the two researchers describe in a paper to be presented on Thursday at the annual meeting of the Association for the Study of Higher Education.

The results, they write, suggest that faculty members and institutions can use Nessie’s deep approaches to assess students’ progress over time.

But the two professors also found that those measures of deep learning bore no relationship to students’ grade-point averages, a result Ms. Campbell found surprising.

That finding gave rise to what the authors describe as a chicken-and-egg problem: Was the lack of a relationship between deep learning and grades evidence of a problem with the Nessie measures or with grading?

ADVERTISEMENT

“The results are intriguing,” Ms. Campbell said in an interview. “There’s something amiss here.”

Perhaps, she said, students are engaged in deep learning but are not being rewarded with good grades. Or perhaps students are receiving high marks but not learning deeply. “My sense,” she said, “is that it’s the former, not the latter.”

The Nessie measures of deep learning also seem to have a disconnect with standardized tests of critical thinking.

Thomas F. Nelson Laird, an associate professor in the department of education leadership and policy studies at Indiana University and principal investigator of the Faculty Survey of Student Engagement, wrote in 2008 that he had found relationships between Nessie’s deep approaches and tests of moral reasoning and of intellectual dispositions like curiosity and open-mindedness. He could not find a link between the deep approaches and scores on the California Critical Thinking Test, which colleges and employers use to gauge test takers’ ability to analyze, make inferences, and carry out inductive and deductive reasoning.

ADVERTISEMENT

Similarly, Robert D. Reason, now an associate professor in Iowa State University’s department of education, wrote in a 2010 paper that he could find no relationship between Nessie’s deep approaches and scores on the Collegiate Assessment of Academic Proficiency, or CAAP. It serves purposes similar to those of the California test and is often used to evaluate colleges’ academic programs.

Mr. Laird said the disparity between the deep approaches and the results on standardized tests probably reflects an inconsistency between what most people in higher education mean by the term “critical thinking” and how it is actually tested. Most academics, he said, see critical thinking as the ability to analyze ambiguous or contradictory arguments and reach a conclusion in situations where the right answer is unclear, if one even exists.

The tests, he said, do something different. They test inductive and deductive reasoning, and they contain little ambiguity. “Critical-thinking tests have clear answers,” he said.

Higher-education experts at ACT, the testing service that created the CAAP, acknowledged that there is a limit to how well a multiple-choice test can measure students’ skill at sorting through problems with uncertain answers. But they also said that facets of their test that assess students’ ability to form, judge, and extend arguments can offer insights into how well students solve ambiguous problems.

ADVERTISEMENT

In the end, argued Mr. Laird, no tool, on its own, can capture learning across disciplines or institutions. Researchers should seek out several measures to paint a picture of what happens during a student’s college education.

The strength of Nessie’s deep approaches to learning, he said, is that they call upon the sorts of skills that both faculty and employers seem to want students to develop. While Nessie data suggest that colleges can do a better job of giving assignments that truly tap into critical-thinking skills, Mr. Laird said students, too, bear some responsibility for taking advantage of such opportunities.

“If they’re not in college to get this stuff out of college,” he said, “maybe it won’t matter what we do.”

We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
Teaching & Learning
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
berrett-edletter-portrait.png
About the Author
Dan Berrett
Dan Berrett is a senior editor for The Chronicle of Higher Education. He joined The Chronicle in 2011 as a reporter covering teaching and learning. Follow him on Twitter @danberrett, or write to him at dan.berrett@chronicle.com.
ADVERTISEMENT
ADVERTISEMENT

Related Content

Economy Affects Students’ Academic Performance as Well as Spending Decisions

More News

Vector illustration of large open scissors  with several workers in seats dangling by white lines
Iced Out
Duke Administrators Accused of Bypassing Shared-Governance Process in Offering Buyouts
Illustration showing money being funnelled into the top of a microscope.
'A New Era'
Higher-Ed Associations Pitch an Alternative to Trump’s Cap on Research Funding
Illustration showing classical columns of various heights, each turning into a stack of coins
Endowment funds
The Nation’s Wealthiest Small Colleges Just Won a Big Tax Exemption
WASHINGTON, DISTICT OF COLUMBIA, UNITED STATES - 2025/04/14: A Pro-Palestinian demonstrator holding a sign with Release Mahmud Khalil written on it, stands in front of the ICE building while joining in a protest. Pro-Palestinian demonstrators rally in front of the ICE building, demanding freedom for Mahmoud Khalil and all those targeted for speaking out against genocide in Palestine. Protesters demand an end to U.S. complicity and solidarity with the resistance in Gaza. (Photo by Probal Rashid/LightRocket via Getty Images)
Campus Activism
An Anonymous Group’s List of Purported Critics of Israel Helped Steer a U.S. Crackdown on Student Activists

From The Review

John T. Scopes as he stood before the judges stand and was sentenced, July 2025.
The Review | Essay
100 Years Ago, the Scopes Monkey Trial Discovered Academic Freedom
By John K. Wilson
Vector illustration of a suited man with a pair of scissors for a tie and an American flag button on his lapel.
The Review | Opinion
A Damaging Endowment Tax Crosses the Finish Line
By Phillip Levine
University of Virginia President Jim Ryan keeps his emotions in check during a news conference, Monday, Nov. 14, 2022 in Charlottesville. Va. Authorities say three people have been killed and two others were wounded in a shooting at the University of Virginia and a student is in custody. (AP Photo/Steve Helber)
The Review | Opinion
Jim Ryan’s Resignation Is a Warning
By Robert Zaretsky

Upcoming Events

07-31-Turbulent-Workday_assets v2_Plain.png
Keeping Your Institution Moving Forward in Turbulent Times
Ascendium_Housing_Plain.png
What It Really Takes to Serve Students’ Basic Needs: Housing
Lead With Insight
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin