Many states around the country are experimenting with programs that tie at least part of their annual appropriations for state colleges and universities to how well those institutions perform various tasks. Right now, 11 states tie some appropriations to measures of public institutions’ performance, and 15 more are likely to follow suit within the next five years, according to state officials and policy analysts. Australia, New Zealand, and Britain also are experimenting with performance-based budgeting, in the belief that it will make their colleges and universities more accountable to the public.
Why this penchant for performance? Is it merely the latest tide of management reform to wash up on the shores of educational institutions -- succeeding the searches for excellence, reinventions of this and that, and re-engineerings of everything else? Paul Light, director of public-policy programs at the Pew Charitable Trusts, has argued that public organizations have undergone too many waves of reform in recent years -- none of which is ever fully implemented or evaluated. Will “performance-based funding,” as it is commonly called, be another of those attempted innovations that spark initial enthusiasm and then fade away? Or will it take root and become a standard way that many states apportion at least part of their higher-education budgets? Given the growing demands that colleges do a better job of accounting for how they spend tax dollars, we’d bet on the latter.
Once it was access that concerned politicians and the public. The United States wanted to extend the benefits of higher education to all. Polls confirm that the public’s belief in the value of college has remained firm, even as faith in other institutions has crumbled. The 82 per cent of American high-school students who say they plan to attend college -- not to mention the untold numbers of non-traditional students clamoring for further education -- means that educators and policy makers have succeeded in showcasing education’s benefits.
But in recent years, at least partly because of competing demands on state resources, politicians’ desire for broad public access to college has been overtaken by their desire to make institutions more accountable to the citizens who pay to support them, just as governors and legislators must be accountable to voters. The challenge, of course, is to find meaningful ways to measure whether a college or university is improving the education that it provides, while simultaneously rooting out waste and administrative bloat. Indeed, the attractiveness of such measures -- commonly called “performance indicators” -- as a concept is equal only to the difficulties of designing, implementing, and assessing them.
Economists usually recommend the use of indicators based on “outputs,” such as the proportion of students who graduate, rather than indicators based on processes, such as trying to quantify how well a professor teaches. Processes are hard to define because of their abstractness and variability, and emphasizing them may lead to less efficiency. For example, if a state wanted to emphasize the process of effective learning, it might push for classes with 15 students rather than 40. But most universities do not have the resources to offer small general-education classes.
Focusing on outputs, however, can be dangerous, too. Using fewer professors to teach more classes might increase productivity, but would it improve learning? And what breakthroughs would we miss because those professors had no time to do research?
Furthermore, how should we determine the value of other, intangible outcomes, such as learning to think, imagine, persevere, see connections, and live a moral life -- all of which colleges and universities seek to teach their students?
Despite the measurement challenges, though, performance funding is one wave of reform that is unlikely to recede. It’s a crucial part of the management revolution that has swept through America’s manufacturing and service sectors, the health-care industry, and now government and education.
Tennessee was the first state to link appropriations for public colleges and universities to performance, in 1979, when the state’s higher-education commission began the program as a way to improve undergraduate education. From the beginning, institutions’ participation has been voluntary. Initially, the focus was on process: Institutions measured their progress primarily against state guidelines. Now, however, the program focuses on results. Roughly 5 per cent of the state’s budget for higher education is earmarked as incentive bonuses for institutions that meet or exceed state- and self-defined goals, such as improved student performance on various tests, and student and alumni satisfaction with their education.
At the University of Tennessee at Martin, for example, departments have systematically thought through -- some for the first time -- precisely what they were trying to accomplish with their students and how to assess those accomplishments. As a result, some departments have changed their requirements or the order in which students take courses; other departments have added “capstone” courses to integrate what students have learned in other classes. Throughout the university, administrators and faculty members use assessment data to revise the curriculum and other activities, such as counseling services.
No one in Tennessee claims that the program is perfect. It is difficult to find tests that thoroughly assess general education, for example. Some departments and institutions have been slow to use information from past assessments to improve their programs. In tight budget years, some institutions grumble that the money set aside for the performance bonuses could be better used to finance basic operations. Over the years, though, the steps that institutions have taken to improve their performance have boosted the confidence of the public and its elected officials in the state’s colleges and universities.
In contrast, South Carolina’s General Assembly didn’t adopt performance indicators for higher education until 1996. This year, $4.6-million was linked to performance goals, and next year the amount will rise to $6-million, if the South Carolina Senate concurs with a recent House vote. Because the state’s Commission on Higher Education has set different standards for two-year colleges, regional universities, and research universities, the likelihood is greater that the institutions will be able to meet their goals. And staff members on the commission are working with administrators and faculty members to revise the goals and methods of measuring progress, based on experience with the program thus far.
Elsewhere, Missouri’s master plan for higher education now includes 24 goals, some of which are used to allocate performance-based awards amounting to roughly 2 per cent of the state’s higher-education appropriations. Campuses also can receive money for programs that they themselves design. Crowder College, for example, is developing tools to assess student learning in such subjective areas as ethical decision making.
In Florida, legislators and educators have attempted for several years to use performance measures in the budget process, both to evaluate system-wide performance and to link financial incentives to specific goals. State policy makers are working with higher-education administrators and employers to improve the quality of the information on which the incentives are based.
In one action, the Legislature recently allotted $24-million for performance-based bonuses for community colleges that provide increased remedial education, award more associate’s degrees, and graduate students more quickly. This is serious money, even in a state as big as Florida. State officials are still developing ways to measure graduates’ success in the labor market. But anecdotal evidence suggests that students’ advisers are doing a better job of counseling, and that students are completing their degrees in less time. The state’s next task is to revamp the incentive measures -- for example, to give institutions credit for meeting intermediate targets. The Florida Commission on Government Accountability has recommended that the Legislature use a completely performance-based budget for the state’s community colleges within five years.
What can we learn from states that have already used performance-based appropriations?
On the positive side, most systems rely on at least some outputs that are easy to measure, such as retention and graduation rates or faculty teaching loads. Other common measures include how long it takes students to earn their degrees, the transfer rates of students from two-year colleges to four-year institutions, and graduates’ job placements, scores on professional licensing tests such as bar exams, and surveys of their satisfaction with their education.
However, those measures don’t meet with unanimous approval. Some two-year institutions don’t like using job placement as an indication of their performance during an economic downturn, for example. And some four-year colleges object to the transfer and time-to-degree measures; the latter could result in reduced revenues for some institutions if students earned fewer credit hours.
Who initiates -- and who buys in -- can make or break a system of appropriations that relies on measuring performance. Participants in Tennessee’s long-lived program, for instance, believe that its voluntary, non-competitive nature has led to strong support from university presidents and administrators. Another secret of Tennessee’s success is that the program has rewarded success rather than penalized failure. Arkansas’s experience was very different: An experiment linking appropriations to performance lasted only two years. State officials failed to win local college presidents’ support for a program that included financial sanctions as well as rewards.
Timing is also a key: Elected officials may want immediate implementation and results, but colleges and universities need enough time to make the process work. Experts who have studied performance-based financing, including Joseph C. Burke, a former chancellor of the State University of New York, and Andreea Serban, a research associate at the Rockefeller Institute of Government at the State University of New York, recommend that policy makers set broad priorities and goals when they mandate such appropriations. Then they must work with a representative group of external stakeholders and college officials to develop ways to measure progress toward the goals.
Unfortunately, most universities still resent the whole idea of performance-based appropriations -- at least when it comes to any area in which they don’t perform well. And in an era when it can take six people to approve getting a copier fixed, state universities are reluctant to add another layer of bureaucratic accountability.
But the public’s demand for accountability is not going away. And information-driven, market-oriented reforms such as performance-based budgets are part of the answer to that demand. Linking performance to appropriations gives policy makers and customers a clearer sense of how the public’s investment in education is being used. Fortunately, when performance systems are designed and used effectively, faculty members and administrators can use them to strengthen and prove the value of their programs -- and sharpen their competitive edge.
Anthony P. Carnevale is vice-president for public leadership at the Educational Testing Service. Neal C. Johnson is senior research partner at the Educational Testing Service. Anne Ruffner Edwards is director of state-and-local-policy programs at the University of North Carolina at Chapel Hill.
