When President Obama announced the release of a new federal College Scorecard in his weekly address on Saturday, he said the website would give Americans "access to reliable data on every institution of higher education."
"You’ll be able to see how much each school’s graduates earn, how much debt they graduate with, and what percentage of a school’s students can pay back their loans," he said.
But prospective students who turn to the revamped tool to estimate their future earnings probably won’t get a realistic sense of what they can expect to make after graduating. And if they’re checking out state websites with wage data as well, they’re likely to end up more confused than enlightened.
To understand why, let’s consider a hypothetical student — Jake, a junior in Virginia who is looking at four-year colleges in his state. Jake doesn’t want to take on large amounts of debt, so he’s leaning toward a public college at which he can pay in-state tuition. He has excelled in STEM classes, so he enters "Virginia Polytechnic Institute" in the search field of the government’s College Scorecard. A graphic tells him that his projected annual "salary after attending" is $57,900, well above the national average of $34,343. He also learns that 80 percent of Virginia Tech graduates earn more than someone who has only a high-school degree.
If he dug a little further, hovering on the "information" icon above the graphics, Jake would learn that the earnings figure accounts for recipients of federal student aid only, and reflects the earnings of students 10 years after they first enrolled. The 80-percent figure, he’d discover, is based on the share of former students who earn more than $25,000 (the average annual income of young high-school graduates) six years after enrolling.
Here’s what the website wouldn’t tell him: how students who don’t receive federal aid fare, or how graduates’ earnings compare with those of dropouts. Nor would it show how majoring in English, rather than engineering, might affect his earning potential. That limitation matters. A wealth of research has shown that "what you study is more determinative than where you study it," said Mark S. Schneider, vice president of the American Institutes for Research.
As Jeff Strohl, director of research at the Georgetown University Center on Education and the Workforce, put it, the scorecard "tells you what people make, on average, but nobody is average."
"The misleading part is the variation," he said. "If you’re a French-literature major at Harvard, you’d better not look at these data and think you’re going to earn $87,000" five years out.
Where State Sites Fall Short
For program-level information, Jake could turn to his state’s higher-education website, which provides average earnings at 18 months and five years after completion, by degree. There he’d learn that the median annual wage for Virginia Tech graduates who earned four-year bachelor’s degrees in the five years ending in 2006 was $52,090 five years after graduation.
He’d also discover the wide variation in wages among majors. Five years out, the average English major was earning just $41,094, while the average chemical-engineering major was making $73,889. Those nuances aren’t captured on the federal site, which provides only a single number for an entire institution.
But the state site has some significant gaps, too. Chief among them is the fact that the data are limited to graduates employed in Virginia in positions that are covered by the state’s unemployment insurance. Missing from the data are graduates who work in the neighboring District of Columbia or Maryland (or any other state), along with federal workers, members of the military, and the self-employed, among others. A note on the state’s website cautions users to "exercise great care to understand the limitations of the available data and their meaning."
"Wage outcomes of graduates do not measure the quality or effectiveness of any institution," the note warns.
And while the state site would give Jake a good sense of what he might earn if he graduated and remained in the state, it wouldn’t help him understand the fiscal consequences of dropping out. That’s because, unlike the federal site, it includes only graduates. Tod Massa, director of policy research and data warehousing at the State Council of Higher Education for Virginia, said the state was exploring ways to draw distinctions between those who earn a degree and those who drop out.
"Clearly, you get a better return if you finish than if you don’t, but we can’t say that right now," he said. (The feds can’t either, since they combine everyone who started at a college.)
Meanwhile, the U.S. Education Department has begun collecting earnings data by program level, with plans to include it in future versions of the scorecard.
Now let’s imagine that Jake’s college counselor urges him to expand his college search to private institutions, noting that many of them offer generous financial-aid packages. If Jake restricted his search to Virginia, he would be able to view the same earnings information that he did for public colleges.
But if he ventured outside his home state, he would be likely to be out of luck. That’s because most states that publish graduates’ earnings do so only for public institutions, not private colleges, according to Mr. Schneider. And none allow consumers to compare similar programs across state lines.
So what is Jake, as a prospective student, to do with the inconsistent and incomplete earnings data available to him? He could choose to ignore it, as some overwhelmed students no doubt will. Or he could draw what conclusions he can, piecing together a rough picture of whether a college is likely to be worth the investment in time and money.