Panel to Give Colleges ‘Gentle Shove’ Toward Testing

The chairman of the federal Commission on the Future of Higher Education says it will not propose mandatory testing of college students but will urge colleges and accrediting agencies to voluntarily adopt assessments of student learning, and will recommend two tests.

“There is no movement, no intent, no expectation of a mandated test at the federal level,” the chairman, Charles Miller, wrote in an e-mail message sent to his fellow commission members in March.

With college costs on the rise, Mr. Miller predicts that institutions will come under increasing pressure to prove their worth to taxpayers and students’ parents, and many colleges will turn to testing. With time, a test will emerge as the gold standard, “not by mandate, but by a normal process of early movers and best practices.”

“I believe the tipping point has already been reached. We need to give it a gentle shove,” he wrote in the e-mail message with the subject line “Standardized Anxiety.”

The message refutes rumors, which began circulating soon after the commission’s creation in September by Secretary of Education Margaret Spellings, that the federal advisory board was weighing a No Child Left Behind-style testing regime for colleges. The commission, which is made up of 19 business and higher-education leaders, is charged with devising a comprehensive plan for the future of higher education by August 1.

But the commission’s testing recommendations will still be controversial. Critics of college-level testing say that it is best left to professors, that tests should be tied to institutional curricula, and that no test can measure the effects colleges have on students’ social and emotional development.

“You can’t quantify life transformation,” says Mark F. Smith, director of government relations at the American Association of University Professors.

Unknown Investment Returns

The call for increased accountability in higher education is not new. For years state and federal policy makers have been pressing colleges and universities to demonstrate their worth. As tuitions have risen, outpacing inflation, many state legislatures and accrediting agencies have tried to create ways to measure the improvements college educations make in students’ knowledge and skills. Some of those efforts at accountability already use testing to measure students’ growth.

Yet there remains a nagging sense among many policy makers and some members of the commission that there is no “bottom line” way of knowing what colleges contribute to students’ intellectual development. As one commissioner, Richard K. Vedder, a professor of economics at Ohio University, puts it, “Harvard students are brilliant going in, and they’re brilliant coming out. The question is: How much value did Harvard add?”

The commissioners’ concern stems, in part, from recent studies suggesting that college graduates perform poorly on basic-skills tests such as the National Assessment of Adult Literacy. That survey, last administered by the Department of Education in 2003, found that less than a third of college graduates could read complex texts and make complicated inferences.

Adding to the alarm are reports from business leaders on the commission that college graduates are unprepared for 21st-century jobs. The executives note that U.S. companies spend billions of dollars to give their workers remedial education each year.

Richard Stephens, a commissioner and senior vice president for human resources and administration for the Boeing Company, said testing would allow companies to evaluate job candidates based on how well their alma maters prepare students for the work force.

“We receive two million job applications each year,” he said. “I want to translate where they went to school into value in the marketplace.”

The push for increased accountability is also coming from the Education Department, which provides one-third of higher-education funds. In announcing the commission’s formation, Ms. Spellings said it was “time to examine how we can get the most out of our national investment.”

Commissioners say the message from the department, policy makers, and the public is that higher education can no longer rest on its laurels.

“For years higher education has said that we do something very special that only we can understand,” said Robert M. Zemsky, chief executive of the Learning Alliance for Higher Education at the University of Pennsylvania and a member of the commission. “We can’t do that anymore. An increasing number of people are becoming concerned that it’s all smoke and mirrors.”

Advocating Accountability

The commission’s biggest advocate of accountability is Mr. Miller, former chairman of the University of Texas System Board of Regents and chief architect of the university’s accountability system, which includes testing to measure student learning.

Mr. Miller, a nationally known investor who once managed millions in public funds, says he believes colleges owe it to the people who invest in them to be sure they are using the money effectively.

Mr. Miller worried many college officials when he told reporters at a December meeting in Nashville that the commission might propose tying an institution’s eligibility for federal student aid to standardized testing. “We’re looking for leverage points,” he said. “Funding is one, accreditation is another.”

While most higher-education associations remained silent, lobbyists for independent colleges and universities went on the defensive, warning that standardized testing would homogenize higher education.

“We need to play to the diversity of higher education and not come up with a Prussian, centralized model of how to go forward,” David L. Warren, president of the National Association of Independent Colleges and Universities, said at a commission meeting held in February in San Diego. At least two commissioners wrote to Mr. Miller to object to standardized testing, with one, Charlene R. Nunley, president of Montgomery College, in Maryland, saying she was worried about conflicts between the national commission’s recommendations and state efforts.

Recognizing the potential firestorm, Mr. Miller has taken care to clarify his position recently, stressing that he sees markets, not mandates, as the path to testing.

His theory, he explains in an interview, is that some colleges will resort to tests to justify their rising tuitions. Once those colleges start publishing test results, their competitors will be pressured to do so as well, driving the cost of testing down. Since policy makers, employers, and the public will want to be able to compare colleges, institutions will gravitate naturally toward a single test, which will then dominate the market.

To give colleges the “gentle shove” the chairman envisions, the commission may urge accreditation agencies to place more emphasis on measuring the effects colleges have on students, or ask the Education Department to require accreditors to do so. Accrediting agencies have been involved in student assessment since the 1980s, when the Education Department directed them to include measures of student learning. But the agencies now often let colleges set their own standards for student assessment.

That approach is much more palatable than mandatory testing to commission members with higher-education backgrounds. If a test proved valid, says David Ward, president of the American Council on Education, “some legitimacy would develop through the marketplace.”

“And that’s much better legitimacy than insisting everyone take it,” he adds.

Mr. Miller stresses, however, that it will be increasingly difficult for higher education to justify requests for increased state and federal aid without providing more data about student learning.

“To make the case for more money,” he says, “you have to show that you’ve done the best you can with what you have.”

Asked if he is still considering tying testing to federal support, Mr. Miller says that he would recommend it “only if colleges didn’t develop (testing) voluntarily ... if there was no progress, or resistance.”

While the commission will not require the use of a single test, it is likely to endorse two: the Collegiate Learning Assessment, or CLA, and the Measure of Academic Proficiency and Progress, both of which Mr. Miller highlighted in a memorandum on accountability sent to fellow commission members in January.

Mr. Miller is particularly impressed with the CLA, which he described at the commission’s Nashville meeting as “a middle-of-the-fairway, perfect drive.” The test, which was developed by the Council for Aid to Education and the RAND Corporation, has been used in the Texas accountability system and has been administered to 30,000 students since 2002. This year it is being used by 121 institutions — including 35 members of the Council of Independent Colleges, who have signed up voluntarily — and is being adapted for use by community colleges.

The test’s creators say it provides a truer gauge of an institution’s quality than its average SAT scores, endowment levels, student-satisfaction scores, or self-reports. However, they stress that the CLA should not be used as a diagnostic tool by itself, but as part of a portfolio of assessments.

The goal of the test, says Stephen P. Klein, a senior research scientist with the RAND Corporation, is to measure the skills “that colleges say are important to them in their mission statements": critical thinking, analytic reasoning, and writing.

He adds that the test eliminates fears about “teaching to the test” because it does not measure content knowledge, but broad-based skills that all students should possess.

Richard H. Ekman, president of the Council of Independent Colleges, says that the CLA allows colleges to show what they contributed to student learning because it evaluates students’ achievement based on where they began and where they ended up. He suggests that if the commission wants to encourage the use of the test, it should ask the Education Department to subsidize the $6,000 cost of the one-year measurement for smaller colleges.

“If you could eliminate the financing problem, that would increase use,” he says.

While the CLA was designed to measure institutional performance, not individual student performance, its creators say it could be modified for use as an exit exam. Students already receive their test results, although colleges receive aggregated results unless given permission by students to receive individual scores.

Will Students Care?

The second test, the MAPP, is the new version of the Educational Testing Service’s Academic Profile, which was retired in January. The test, which is multiple-choice, is also designed for institutional-improvement purposes, and allows colleges to compare themselves with peer institutions of their choosing.

Assessment experts say the emergence of these two tests makes it hard to argue that testing can’t work on the college level.

“It’s almost impossible to kill this thing by saying that what we teach in college is so unmeasurable that there is no way you can assess it,” says Patrick M. Callan, president of the National Center for Public Policy and Higher Education.

Mr. Callan’s group conducted a five-state pilot project that sought to measure the performance of colleges and universities within a state by comparing data on professional-licensing and graduate-admissions tests, the results of the National Assessment of Adult Literacy, and students’ scores on the Collegiate Learning Assessment and another test designed for two-year institutions. The commission is likely to recommend an expansion of the national center’s project in the commission’s final report.

Not everyone is taken with the new tests, however. In an e-mail message to Mr. Miller, Mr. Zemsky, of the University of Pennsylvania, said tests like the CLA are flawed because students do not have a stake in how well they do.

Mr. Zemsky says colleges do not need a new test — they simply need to make better use of the results from the many standardized exams already taken by their students. He proposes that benchmark scores, such as the median for all students who take the Graduate Record Examination, be published, so consumers could compare competing institutions based on the relationship between their SAT scores and the scores on exams for graduate schools.

“We’re not short on data, we’re short on data that has been made public in a consistent way,” he says in an interview.

Mr. Miller acknowledges that concerns about motivation are valid, but he says the evidence from the CLA suggests that students are indeed motivated to do well.

“It’s not a fatal flaw,” he says.

‘A Wake-Up Call’

Critics of such tests, like Leon Botstein, president of Bard College, say the answer to assessment lies not in standardization, but in “rigorous self-policing and peer review” by college faculty members. He calls the controversy over testing “a wake-up call for institutions, to make sure they’re doing their job.”

Indeed, some commissioners question whether colleges will ever embrace testing without outside pressure. “If you leave it up to the faculty, things will never get done,” says Arthur J. Rothkopf, president emeritus of Lafayette College and senior vice president at the U.S. Chamber of Commerce. “Faculty are great on the status quo.”

Meanwhile state-college lobbyists are trying to get out in front of the commission by offering their own ideas about how testing fits within a broader sense of accountability. They say the debate over accountability offers a prime opportunity for colleges to define their priorities and design measurements that reflect them.

“Public universities realize this is coming,” said Travis J. Reindl, director of state policy analysis at the American Association of State Colleges and Universities, which is working with the National Association of State Universities and Land-Grant Colleges to develop position papers on testing. “It is better to get out in front, manage it, and shape it than to be a victim of it.”

http://chronicle.com Section: Government & Politics Volume 52, Issue 31, Page A33