Why Are SAT Takers Getting an ‘Adversity Score’? Here’s Some Context

Grades, test scores, and a list of extracurricular activities can tell you only so much. To assess an applicant’s performance, admissions officers at selective colleges also try to understand the all-important context of his or her achievements. A new tool might help them do that.

As The Wall Street Journal reported on Thursday, 50 colleges over the last year have been using something called the Environmental Context Dashboard, which includes an “adversity score” for each applicant who took the SAT. The College Board, which oversees the exam, created the dashboard to give colleges a better understanding of test takers’ socioeconomic backgrounds — and the challenges they may have encountered.

So far, at least some participating colleges report that the dashboard has helped them admit more disadvantaged applicants. David Coleman, the College Board’s chief executive, said in a written statement that the tool “shines a light on students who have demonstrated remarkable resourcefulness to overcome challenges and achieve more with less.” And some admissions officials describe the platform as a promising race-neutral tool that could prove especially useful if the Supreme Court one day strikes down race-conscious admissions policies.

It’s already controversial, though. Robert Schaeffer, public-education director of the National Center for Fair and Open Testing (FairTest), believes that the dashboard undermines the very notion that standardized tests are a common yardstick for comparing applicants. “This latest initiative,” he said in a written statement, “concedes that the SAT is really a measure of ‘accumulated advantage’ which should not be used without an understanding of a student’s community and family background.”

One thing perhaps no one would dispute: “Environmental Context Dashboard” is a mouthful. And so far there is some confusion about how the platform works, as well as what it could mean for applicants and colleges alike. So let’s take a close look at some key questions.

What does the dashboard tell colleges?

The new tool was built to inject more-nuanced data into evaluations of applicants, particularly those who face significant obstacles. At a time when admissions officers’ subjective judgments are under intense scrutiny, this data-driven experiment offers a systematic way to make sense of where, literally and figuratively, an applicant comes from.

The dashboard does not take a student’s race or ethnicity into account. It does show levels of disadvantage in each applicant’s neighborhood and high school, incorporating data on the crime rate, the median income, and residents’ educational attainment. It also includes data on the school’s academic rigor, participation in the free and reduced-price lunch program, and “undermatch risk” (the likelihood that typical students in a given school or area will enroll in a less-selective college than their grades and test scores suggest they could). And, yes, colleges see how applicants’ SAT scores compare with those of their peers.

The College Board uses all that data — drawn from the U.S. Census, public records, and other sources — to calculate a student’s “Overall Disadvantage Level,” on a scale of one to 100. The higher the number, the more disadvantaged the applicant.

Right now, test takers don’t get to see that score. Zachary Goldberg, a spokesman for the College Board, said the organization was open to discussing the issue with its members.

Why does this kind of “environmental” information matter?

Students from low-income backgrounds are greatly underrepresented at the nation’s most-selective colleges. And standardized tests play a large role in that story: Applicants with affluent, college-educated parents tend to far outscore their disadvantaged peers, just as white and Asian-American students, on average, fare better on the exam than black and Hispanic students do.

Some researchers believe that richer information about the opportunities available to such underrepresented students would help admissions officers better understand their achievements. One of those researchers is Michael N. Bastedo, director of the Center for the Study of Higher and Postsecondary Education at the University of Michigan at Ann Arbor, whose findings informed the dashboard’s development. In a recent study he led, admissions officers were up to 28 percent more likely to admit a low-income applicant when they had ample information about his or her high school.

The state of Florida bars the use of race in admissions. But recently, Florida State University increased its enrollment of nonwhite students by five percentage points, to 42 percent. The dashboard helped the university do that, according to John Barnhill, assistant vice president for academic affairs. “It can change the way you read an application,” he said. “When you look at a test score, you might prejudge an applicant. But then you look more carefully at the high school, you might think, ‘Boy, in the environment that they’re in, they’re doing pretty well.’”

Sure, many admissions officers already have plenty of information about high schools, but they tend to know more about some schools than others. And some colleges weigh context with more care than others. In any case, another layer of convenient information can aid admissions officers, who are often pressed for time.

Kirk Brennan, director of admission at the University of Southern California, said the dashboard had helped his staff “leaven the adverse effects of the SAT” for some highly qualified students: “It can help us know more about the life they’re living, the culture at their high school, and what they might bring of that experience to the classroom.”

Brennan recalled how neighborhood data had helped him understand the context of a particular low-income applicant who happened to attend an affluent high school. “That allowed me to make the case for a student,” he said, “who might not otherwise have been admitted.”

What are some concerns to keep in mind?

Every measure that admissions officers use is flawed. And the new dashboard is no exception, said Richard A. Clark, director of undergraduate admission at the Georgia Institute of Technology: “My concern, as with all numbers we use in admissions, is that people will make automatic assumptions about students because of it.”

Clark described the dashboard as a helpful signpost. So far, he likes it.

But he also recalled a couple of instances in which applicants’ adversity scores didn’t square with the information in their files, such as their parents’ occupations and the kind of high schools they attended. “Sometimes, there was enough to raise an eyebrow,” he said, “and I questioned it.”

That’s one reason the dashboard concerns Sara Urquidez, executive director of the Academic Success Program, a nonprofit group in Dallas that provides college advisers at public high schools. “Students’ life experiences should not be reduced to an adversity rating,” she said. “This is the College Board making a determination for all of us. The key is, How do you train people not to misuse this?”

It’s an especially important question to ask of a measure that’s somewhat mysterious. Though public information largely informs the dashboard, the College Board has not said how it weights various factors or calculates the disadvantage levels that pop up on a user’s screen. So, yeah, there’s a secret-sauce element to the dashboard.

In the end, an adversity score is just one data point, just like a test score. And it will present admissions leaders with one more opportunity to affirm the importance of contextualizing all the digits before them. “You never take one number in isolation,” Clark said.

Evaluations aside, the dashboard could help admissions officials with another important task: Persuading their presidents and boards to support policies and practices that might increase socioeconomic diversity. After all, the dashboard gives each college an average institutional score, too, based on the demographic characteristics of students they enroll.

Georgia Tech’s score is 29, according to Clark. “That means we don’t have a high adversity score,” he said. “And so this gives us another indicator to say, ‘Hey, we are not doing a great job with enrolling certain kinds of students,’ as we advocate for funding for travel, recruitment, and financial-aid packaging.”

The College Board plans to expand its Environmental Context Dashboard experiment to 150 institutions this fall. After that, it’s a safe bet that any and all institutions will be able to use it. But no whiz-bang tool can guarantee that a given college will admit more underrepresented students, provide them with sufficient aid, or support them when they enroll. All of that comes down to something known as institutional will.

Eric Hoover writes about the challenges of getting to, and through, college. Follow him on Twitter @erichoov, or email him, at eric.hoover@chronicle.com.