Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Virtual Events
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Virtual Events
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    A Culture of Cybersecurity
    Opportunities in the Hard Sciences
    Career Preparation
Sign In
Measuring Stick

A ‘Stealth Assessment’ Turns to Video Games to Measure Thinking Skills

Avoiding complications like test anxiety is one benefit of a method for evaluating difficult-to-capture abilities

By Travis Kaya November 7, 2010
This computer game, ostensibly about ecology, can actually measure a variety of critical-thinking skills in students, says Valerie J. Shute, a professor at Florida State U., who is playing the game on her laptop.
This computer game, ostensibly about ecology, can actually measure a variety of critical-thinking skills in students, says Valerie J. Shute, a professor at Florida State U., who is playing the game on her laptop.Sarah Kiewel for The Chronicle

Colleges no longer simply want to know what their students know, but how they think.

Higher-order thinking skills are “something that schools are paying a little bit more attention to these days,” says Jeffrey Steedle, a measurement scientist at the Council for Aid to Education, whose Collegiate Learning Assessment essays are used at several hundred colleges to test students’ abilities to synthesize arguments and write persuasively. “It’s largely in response to the recognition that these skills are needed to be competitive in the global marketplace.”

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

Colleges no longer simply want to know what their students know, but how they think.

Higher-order thinking skills are “something that schools are paying a little bit more attention to these days,” says Jeffrey Steedle, a measurement scientist at the Council for Aid to Education, whose Collegiate Learning Assessment essays are used at several hundred colleges to test students’ abilities to synthesize arguments and write persuasively. “It’s largely in response to the recognition that these skills are needed to be competitive in the global marketplace.”

But educators also say that paper-and-pencil examinations have limits—for one thing, knowing that you are being tested can drag down performance—and they are looking for new methods to measure skills like critical thinking, creativity, and persistence.

Valerie J. Shute, an associate professor of educational psychology and learning systems at Florida State University, believes she has a solution in “stealth assessment"—the administering of tests without students’ knowing.

To do that, Ms. Shute and other stealth-assessment researchers have turned to video games, which let educators watch students solve complex tasks while immersed in virtual worlds. How students react to new challenges and put evidence together—without the pressure of test proctors breathing down their necks—can reveal a lot about creative problem-solving skills that traditional testing cannot deliver. “A lot of important stuff happens when playing games,” Ms. Shute said. “You’re just doing. You’re in the process.”

Ms. Shute, who first studied stealth assessment with a video game for undergraduate students more than 20 years ago, sees applications for students of all ages. She is now helping two of her graduate researchers test the technology with sixth graders in Florida, And she is already looking for new ways to use the technique in her graduate courses.

“Everybody likes to play,” she says. “And so much could be done using games.”

Matthew Ventura, an associate research scientist at the Educational Testing Service who worked on stealth-assessment research with Ms. Shute, says the technique will be especially helpful in eliminating test anxiety, which can hurt students’ performance. That and its potential to test intangibles like creativity and problem solving, he said, make the technique attractive to educators and video-game developers alike.

Virtual Worlds, Real Skills

Educators have long believed that video games and virtual worlds could be used to supplement classroom instruction, although not necessarily as testing tools.

In 1986, when Ms. Shute was a postdoctoral fellow at the Learning Research and Development Center at the University of Pittsburgh, she designed a computer game to teach undergraduates the principles of microeconomics. The game was set in the virtual farming community of Smithtown, where students could play around with different economic factors—like product price, labor costs, and population size—to see how they affected the market. A student who decided to raise the price of coffee, for instance, would see demand for his or her product fall as a result. The software also let students plug in hypotheses about the outcome of their experiments.

ADVERTISEMENT

On its surface, the game was a way for students to get acquainted with basic economic ideas—and maybe have a little fun in the process.

But it was also teaching students a deeper lesson about scientific inquiry. If they changed too many inputs at once, they found it difficult to determine what caused a sudden change in the market, and had to go back and experiment with the inputs to determine exactly what was happening.

The question that grew out of Smithtown, Ms. Shute says, was, “Wouldn’t it be lovely to actually pass along the log files of what students did in order to look at their scientific-inquiry skills?”

Since then, Ms. Shute has been working on a framework to help educators design and execute stealth assessment.

ADVERTISEMENT

She looks first to the core competencies—critical thinking, empathy, persistence—that she wants to test, then breaks them down into smaller goals. She can then tie those theoretical skills to actual tasks in a video game. For instance, an instructor looking to test a student’s grasp of systems thinking—understanding the complex relationships among parts of a whole—might ask players to complete tasks that show information gathering, developing hypotheses, and tracing causal relationships.

Rather than conducting a one-off exam, Ms. Shute says, continuing stealth assessment “enables you to have a very systematic representation of the stuff you’re interested in.”

But for Ms. Shute, stealth assessment is not just about gathering data. It’s also about improving teaching.

If instructors know where students need the most help, they can quickly tweak their courses—and their games—to make up for those deficiencies. Students who need help developing critical-thinking skills, for instance, may be asked to repeat a level or to take on additional tasks in the game until their performance is satisfactory—all without interrupting their play. “The idea of stealth assessment is really to make it merge into the fabric of the learning environment,” Ms. Shute says. “My goal is to blur the distinction between learning and assessment.”

ADVERTISEMENT

Crunching all the data on student performance and tweaking computer games were major tasks in the days of Smithtown, but today’s technology is making it possible to test increasingly complex critical thinking using virtual worlds.

Taiga Park, a computer game developed by the University of Indiana’s Center for Research on Learning & Technology, is one of Ms. Shute’s favorite vehicles for stealth assessment. On its surface a game about ecology, Taiga Park requires players to look for the cause of a widespread fish die-off in a virtual river by “interviewing” park rangers, environmental scientists, and the owners of a logging company. While students learn about pH levels and runoff, they also come away with lessons on data analysis, complex cause-and-effect relationships, and communication.

With the stealth-assessment framework behind it, Oktay Donmez and Yoon Jeon Kim, two of Ms. Shute’s graduate students at Florida State, are planning to test Taiga Park with about 50 sixth graders. Starting in the spring, the researchers will study the students’ performance data from the game to see how effective stealth assessment is at measuring their complex thinking.

“My goal is not only to design, develop, and implement stealth assessment within a game, but also to test its effectiveness,” Mr. Donmez says.

ADVERTISEMENT

Although much of the research in the field—especially when dealing with elementary and secondary students—is centered on video games, Ms. Shute is quick to point out that stealth assessment is widely applicable as a tool, not just a game.

Working with Florida State graduate students in a course called “How to Write Excellent Literature Reviews,” Ms. Shute found that she could use routine assignments—like peer reviews and summaries of research material—to analyze her students’ higher-order thinking skills. All assignments can be linked back to a larger skill, she says. “Evidence is everywhere.”

Ms. Shute also hopes that stealth assessment might engage students unmoved by traditional teaching and testing.

“We have this whole group of kids who are not engaged with school, and appropriately so, because schools are so antiquated,” she says.

ADVERTISEMENT

With field research on stealth assessment still ahead of them, both Ms. Shute and her students are optimistic about the technology, especially for a generation of students who grew up with video games at home.

“They’re going to play video games anyway,” Ms. Kim says. “We can actually embed something that can help them learn.”

We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
Technology
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
ADVERTISEMENT
ADVERTISEMENT

More News

Illustration showing two professors outside a university building sunk down in a large canyon, looking up at an unreachable outside world above them.
Stagnant pay
Professors Say They Need a Raise. They Probably Won’t Get One.
Photo-based illustration depicting a basketball scene with a hand palming a quarter, another hand of a man wearing a suit sleeve, and a basketball goal made from a $100 bill and the Capitol building.
Sports shakeup
A New Normal Looms in College Athletics. Can Trump Help Shape It?
Illustration showing three classical columns on stacks of coins, at different heights due to the amount of coins stacked underneath
Data
These 35 Colleges Could Take a Financial Hit Under Republicans’ Expanded Endowment Tax
Illustration showing details of a U.S. EEOC letter to Harvard U.
Bias Allegations
Faculty Hiring Is Under Federal Scrutiny at Harvard

From The Review

Solomon-0512 B.jpg
The Review | Essay
The Conscience of a Campus Conservative
By Daniel J. Solomon
Illustration depicting a pendulum with a red ball featuring a portion of President Trump's face to the left about to strike balls showing a group of protesters.
The Review | Opinion
Trump Is Destroying DEI With the Same Tools That Built It
By Noliwe M. Rooks
Illustration showing two men and giant books, split into two sides—one blue and one red. The two men are reaching across the center color devide to shake hands.
The Review | Opinion
Left and Right Agree: Higher Ed Needs to Change
By Michael W. Clune

Upcoming Events

Ascendium_06-10-25_Plain.png
Views on College and Alternative Pathways
Coursera_06-17-25_Plain.png
AI and Microcredentials
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Virtual Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin