Government

Where 3 Accountability Measures Meet, a Hazardous Intersection

April 09, 2015

Last week the U.S. Department of Education released a list of 544 colleges facing an extra level of financial scrutiny known as "heightened cash monitoring," after initially refusing to make the list public.

The department's release of the list brought into focus yet another arcane-sounding accountability measure. (Colleges can face one of two increasingly strict levels of monitoring, both of which include a delay in colleges’ student-aid reimbursements.) Two other measures, colleges' financial-responsibility scores and cohort-default rates, have been around for years. All three metrics have the potential to help policy makers and the public make better decisions about colleges.

But how much can we learn from this tangle of accountability measures? The Venn diagram below shows just how little overlap there is among colleges facing sanctions or potential sanctions for violating the department's standards under these metrics: Only 26 institutions ran into trouble in all three categories.

Six of them are private nonprofit and the other 20 are for-profit institutions. The majority of these colleges, such as Taylor Business Institute, in Chicago, and the Florida School of Traditional Midwifery, are very small vocational colleges. However, two institutions are better-known. Allen University, a nonprofit historically black university in South Carolina, has been placed on warning by its accreditor for financial issues as it struggles with enrollment. Everest College, once a part of Corinthian Colleges Inc., has been partially purchased by ECMC Group and will make a transition to nonprofit status.


Proof: An occasional series in which higher-education insiders work out new arguments using data. If you’re interested in contributing, email jeff.young@chronicle.com.


These accountability measures may sound trivial, but violating any one of them can have serious consequences. Last year, Corinthian was subjected to the less-severe of the two levels of heightened cash monitoring, and the giant for-profit educator eventually collapsed. Colleges with high default rates can lose access to federal financial aid. And running afoul of any of these standards can attract negative news-media attention that prospective students and their families will see.

Checking the Alignment

To see how well these three accountability metrics line up (or don’t) with each other, I constructed a dataset of colleges that met any of the following three criteria:

  • Are publicly known to be subjected to either level of heightened cash monitoring in April 2015.

  • Had a financial-responsibility score at or below 1.4 at least once between the 2010-11 and 2012-13 academic years, subjecting the college to additional oversight. The financial-responsibility score has three components that are designed to reflect the short-term and long-term financial stability of private colleges. However, colleges have raised concerns about the validity of the scores, and I showed in a blog post last month that they are essentially uncorrelated with Moody’s credit ratings.

  • Had a cohort default rate over 30 percent in any of the last three years, representing the percentage of students who left college from 2009 to  2011 with federal student loan debt and who subsequently defaulted within three years. While colleges can game this measure somewhat by encouraging students to go into forbearance, a default rate over 30 percent in three consecutive years can result in the loss of federal financial-aid eligibility.

My analysis identified 1,150 private nonprofit and for-profit colleges that faced sanctions or potential sanctions under one of those three metrics in the last three years. Public colleges were excluded because they do not receive financial-responsibility scores and very few have default rates high enough to face potential sanctions.

Of these 1,150 colleges, 438 were subjected to heightened cash monitoring, 403 had a cohort default rate above 30 percent, and 635 faced additional oversight for their financial responsibility scores.

There is some overlap among the three groups of institutions, but not much, as it turns out.

Three hundred and six colleges had been subjected to restrictions under heightened cash monitoring for having low financial-responsibility scores. But only 171 of those institutions had financial responsibility scores meriting sanctions in the last three years.

But colleges can also be placed on heightened cash monitoring for financial-responsibility reasons without receiving a score — basically, they automatically failed the financial-responsibility test for not meeting other performance criteria, so there was no need to calculate a score. However, the Department of Education has not released details on which colleges automatically failed the test (and why they failed) without a score being calculated.

The overlaps between the other combinations of measures are much smaller. Just 76 of the 438 colleges facing heightened cash monitoring had a cohort default rate above 30 percent at least once in the past three years, although an additional 58 colleges most likely did not participate in the federal student loan program. As research suggests that colleges are more likely to stop offering federal loans if their default rates approach 30 percent (in order to protect Pell Grant eligibility), some of these colleges would probably have high default rates if they continued to offer federal loans.

Finally, only 76 colleges are facing potential sanctions for low financial-responsibility scores and high cohort default rates—less than 20 percent of each group of institutions.This suggests that these two previously existing metrics are capturing different portions of a college’s performance.

The Takeaway

What can be learned from the release of the list of colleges under heightened cash monitoring? The biggest takeaway is that there is surprisingly little overlap between colleges subject to additional oversight due to heightened cash monitoring and those subject to oversight or sanctions for their cohort default rates or financial-responsibility scores. Data limitations play a role in the lack of overlap, as not all colleges are assigned financial-responsibility scores or choose to participate in federal student loan programs. But even among colleges with complete data on all three measures, the overlap is far from complete. This suggests few colleges are uniformly bad across all three metrics.

If students and their families are considering a college subject to heightened cash monitoring, they should look at why the college is subject to additional oversight. If it is because the college posted a low financial responsibility score one year, that is probably not a major sign the college is in trouble. But if accreditation is the stated reason, that should be a much larger concern. And if a college is facing additional oversight for poor scores on two or all three metrics, students and policy makers alike should proceed with caution before making large investments.

It is critical for the Department of Education to release additional data about how and why colleges are subjected to heightened cash monitoring, if that measure is expected to be a meaningful accountability tool. Doing so would make the list more valuable to stakeholders and could reduce the risk of unduly harsh sanctions on colleges that barely met the threshold for heightened cash monitoring while focusing sanctions on the absolute worst performers.

Robert Kelchen is an assistant professor of higher education at Seton Hall University. He blogs at Kelchen on Education and is on Twitter at @rkelchen. All opinions are his own.