For decades some state lawmakers have pushed public colleges and universities to think of themselves as parts of a broader job-training plan. It’s only recently that those efforts have been bolstered by detailed data.
Working with years of income information, a majority of states (at least 37, according to survey results released in October) have now built — or are in the process of building — systems that can pinpoint what graduates earn. Some states can even tell what types of jobs the graduates hold and how much debt they’ve taken on. Ohio’s system, for example, knows that the typical biology graduate of Ohio State University earns $22,565 in his or her first year after graduation.
That’s valuable information, and some states are moving to make use of it. For example, Florida already factors work-force data into decisions on funding for public colleges and universities. It’s not hard to find lawmakers eager to use the information to push students away from “underperforming” degrees and toward programs that lay a pipeline to “in-demand” jobs.
Meanwhile, states continue to spend money to build such data-collection programs. In addition, President Obama’s budget for 2016 would allocate $107 million to the two federal grant programs supporting so-called longitudinal data systems, up 174 percent from the current fiscal year.
The growth of income analysis, says Anthony P. Carnevale, director of Georgetown University’s Center on Education and the Workforce, is “more a relentless brush fire than a spectacular forest fire. The information will arrive a long time before the policy impacts will be known.”
But as work-force data becomes more commonplace — and starts to influence policy decisions — it’s worth pausing to take a look at some of its shortcomings. Even many of the people building the data systems admit the limitations are significant, at least for the time being. Those limits may affect the data’s short-term use and its long-term value.
Flaws and Holes
Here’s how states typically generate work-force data: Most use graduates’ Social Security numbers to match quarterly income information reported by employers with colleges’ transcript data. The states then aggregate the data to show the average earnings among graduates of specific degree programs.
That sounds simple enough. But according to Eugene J. Kovacs, assistant vice chancellor of information-resource management for the State University System of Florida, the data still has missing pieces. Most important, graduates who work in another state are missing from the records of the state where they received their degree.
Those missing graduates can make up a huge chunk of a state’s awarded diplomas. For instance, only 26 percent of the graduates of Colorado’s public colleges and universities have been found in its wage records.
Also not captured in the data are graduates who take jobs with the federal government or who are missing for other reasons. “We don’t know how many we’re looking for,” says Rachel Zinn, director of the Workforce Data Quality Campaign, “because the ‘missing’ students might be employed out of state, or they might be unemployed, or pursuing further education, so they are not working, or maybe they are self-employed and so not in unemployment-insurance wage records at all.” Ms. Zinn’s group is an effort to expand the skilled work force.
The missing data is a problem because it might be undervaluing — or overvaluing — some degree programs. For example, Ohio State’s biology program, whose graduates post low earnings in their first year out of college, might have skewed income results: Only 36 percent of its graduates have been found working in the Buckeye State.
Those gaps could be filled in. Mark S. Schneider, president of College Measures, a research partnership that investigates higher-education outcomes, suggests one solution: for the Internal Revenue Service to hand over missing wage data to states.
Meantime, though, some states are teaming up to share data on workers. More than 36 states participate in one such program, the Wage Record Interchange System 2. Some states are benefiting from the exchange. Florida now has data on about 70 percent of its public-college graduates, with a small share of that coming from the interchange system.
But as Mr. Schneider points out, the interchange comes with its own problems. Two of the largest states, California and New York, do not participate. And about one in four requests for data go unanswered.
With so many holes in the data, is it fair to base funding on such a system? Mr. Schneider argues that the missing data on graduates who work out of state shouldn’t confound policy decisions because most states are chiefly concerned with educational and economic outcomes within their borders.
But several state officials working on data systems note that, with very few graduates coming from some degree programs, sample size is a concern. In smaller fields, it may be hard to confidently make funding decisions with incomplete earnings data.
Of course, even sound data is open to interpretation. It’s not entirely clear, for example, which years of income data should become the basis for funding choices. As Mr. Schneider points out, graduates in some majors historically have earned more after their early years. For instance, Ohio State’s biology majors — the ones who earn just over $22,500 in their first year out of college — make $57,566 by their sixth. That makes the major look much more valuable.
Other experts ask broader questions about using the data as a driver of policy decisions.
“Students and states would be well advised to be thoughtful and know what the data means,” says Tod R. Massa, policy-research and data-warehousing director at the State Council of Higher Education for Virginia.
“I have a degree in studio art,” he says. “For some people, that major will not be relevant.”
States Fill the Void
That was part of the argument higher-education lobbyists made in the run-up to the 2008 reauthorization of the Higher Education Act. That law bans federal agencies from collecting graduates’ income data, precisely to prevent the government from using the figures to influence appropriations and policy.
By that point, states had already begun to pick up the slack. Although the income information had been available for decades, it was not until the 1990s, when Florida started to build its system, that the data was seen as a way to measure educational outcome.
Jay Pfeiffer, the architect of Florida’s system, says it created a pathway for other states. In 2005 the U.S. Department of Education started issuing grants to states interested in building their own longitudinal data systems.
While the federal government is barred from creating a national system, it continues to fund state projects. A big boost came from the 2009 economic-stimulus legislation, which pumped $250 million into states’ income-tracking programs.
There’s a chance, albeit a slim one, that the federal government could get into the game directly. Sen. Ron Wyden, Democrat of Oregon, plans to reintroduce the “Student Right to Know Before You Go Act” — a bill, last floated in 2013, that would create a dashboard allowing students to search earnings and student-debt data by degree program. The bill would roll back some of the national restrictions enacted in 2008.
But for the bill to pass, Mr. Wyden would need to assuage concerns about the database’s privacy and security, and about how the data would be used by the government. A Senate Democratic aide says she is “not too optimistic” about the bill’s prospects, especially in the face of fears of data breaches and concerns that the data could be tied to financial-aid funding. The aide notes that the bill has supporters and skeptics on both sides of the aisle, and that Republican control of Congress does not necessarily damage or raise the bill’s prospects.
So states have taken up the mantle themselves. Fourteen have ponied up money, and 19 others are in the process of allocating funds for their own data systems. Minnesota’s Legislature recently appropriated about $1.8 million to acquire additional data and maintain its current system.
Despite holes in the data and concerns about its value, the systems aren’t going away. And if funding is any indicator, more states will jump on board soon.
“Everybody,” says Mr. Carnevale, “is heading down this road, almost without exception.”
Lance Lambert writes about data and trends in higher education. You can follow him on Twitter @NewsLambert, or write to him at lance.lambert@chronicle.com.