‘U.S. News’ Sizes Up Online-Degree Programs, Without Specifying Which Is No. 1

U.S. News & World Report has published its first-ever guide to online degree programs—but distance-education leaders looking to trumpet their high rankings may find it more difficult to brag about how they placed than do their colleagues at residential institutions.

Unlike the magazine’s annual rankings of residential colleges, which cause consternation among many administrators for reducing the value of each program into a single headline-friendly number, the new guide does not provide lists based on overall program quality; no university can claim it hosts the top online bachelor’s or online master’s program. Instead, U.S. News produced “honor rolls” highlighting colleges that consistently performed well across the ranking criteria.

Eric Brooks, a U.S. News data research analyst, said the breakdown of the rankings into several categories was intentional; his team chose its categories based on areas with enough responses to make fair comparisons.

“We’re only ranking things that we felt the response rates justified ranking this year,” he said.

The rankings, which will be published today, represent a new chapter in the 28-year history of the U.S. News guide. The expansion was brought on by the rapid growth of online learning. More than six million students are now taking at least one course online, according to a recent survey of more than 2,500 academic leaders by the Babson Survey Research Group and the College Board.

U.S. News ranked colleges with bachelor’s programs according to their performance in three categories: student services, student engagement, and faculty credentials. For programs at the master’s level, U.S. News added a fourth category, admissions selectivity, to produce rankings of five different disciplines: business, nursing, education, engineering, and computer information technology.

To ensure that the inaugural rankings were reliable, Mr. Brooks said, U.S. News developed its ranking methodology after the survey data was collected. Doing so, he said, allowed researchers to be fair to institutions that interpreted questions differently.

Some distance-learning experts criticized that technique, however, arguing that the methodology should have been established before surveys were distributed.

Russell Poulin, deputy director of research and analysis for the WICHE Cooperative for Educational Technologies, which promotes online education as part of the Western Interstate Commission for Higher Education, said that approach allowed U.S. News to ask the wrong questions, resulting in an incomplete picture of distance-learning programs.

“It sort of makes me feel like I don’t know who won the baseball game, but I’ll give you the batting average and the number of steals and I’ll tell you who won,” he said. Mr. Poulin and other critics said any useful rankings of online programs should include information on outcomes like retention rates, employment prospects, and debt load—statistics, Mr. Brooks said, that few universities provided for this first edition of the U.S. News rankings. He noted that the surveys will evolve in future years as U.S. News learns to better tailor its questions to the unique characteristics of online programs.

W. Andrew McCollough, associate provost for information technology, e-learning, and distance education at the University of Florida, said he was “delighted” to discover that his institution’s bachelor’s program was among the four chosen for honor-roll inclusion. He noted that U.S. News would have to customize its questions in the future, since he found some of them didn’t apply to online programs. He attributed that mismatch to the wide age distribution and other diverse demographic characteristics of the online student body.

The homogeneity that exists in many residential programs “just doesn’t exist in the distance-learning environment,” he said. Despite the survey’s flaws, Mr. McCollough said, the effort to add to the body of information about online programs is helpful for prospective students.

Turnout for the surveys varied, from a 50 percent response rate among nursing programs to a 75 percent response rate among engineering programs. At for-profit institutions—which sometimes have a reputation for guarding their data closely—cooperation was mixed, said Mr. Brooks. Some, like the American Public University System, chose to participate. But Kaplan University, one of the largest providers of online education, decided to wait until the first rankings were published before deciding whether to join in, a spokesperson for the institution said.

Though this year’s rankings do not make definitive statements about program quality, Mr. Brooks said the research team was cautious for a reason and hopes the new guide can help students make informed decisions about the quality of online degrees.

“We’d rather not produce something in its first year that’s headline-grabbing for the wrong reasons,” he said.

‘Honor Roll’ From ‘U.S. News’ of Online Graduate Programs in Business

The numbers represent the rankings in each category, but there is no overall ranking.