The government should heed the lessons from ratings in other education and care markets in developing its plans for rewarding teaching excellence, in addition to research excellence, according to a new paper by the Higher Education Policy Institute, or HEPI.
Nick Hillman, HEPI’s director, said: “Ministers want to see the best possible teaching and learning in higher education. But assessing the quality is easier said than done. There is no agreed way of assessing what students learn. There is no off-the-shelf solution available from another country. And there will be perverse consequences if we get it wrong. But we can learn from experience in other sectors.”
This is an article from University World News, an online publication that covers global higher education. It is presented here under an agreement with The Chronicle.
At the end of last year the government set out its plans for a Teaching Excellence Framework, or TEF, to complement the existing Research Excellence Framework, or REF, in a Green Paper, with a view to implementing it 2017-18.
The idea is to ensure that potential students – and their parents – are able to make informed choices about where to study. Whereas the REF specifically focuses on research and rankings tend to be biased towards it, the TEF will enable them to find out about the quality of teaching and learning at institutions they are considering as an option.
Under the plan, higher education institutions which have met or exceeded Quality Assurance Agency expectations in their last review will be granted a level-one award, with the option to apply for higher-level awards from year two onwards.
Assessment will involve a review of metrics and wider evidence by a panel. Depending on the level awarded, institutions will be able to increase their tuition fees above the cap at differentiated rates but not exceeding a real terms increase. The objectives are threefold: to recognise, and promote, teaching excellence and to inform choice.
Hillman said ratings have existed for many years for nurseries, schools and hospitals and examining them provides provide positive and negative lessons for the new TEF.
“If we consider them closely, we can find out how to ensure the TEF does not become too big, too bossy or too bureaucratic – and that it hits the right target.”
The paper, “Designing a Teaching Excellence Framework: Lessons from other sectors”, examines the relative success of Ofsted, the schools’ inspectorate, where ratings have expanded from schools to the wider education market, and compares it to the more intermittent history of ratings in health and social care.
Key factors
The paper’s author, Louisa Darian – who led the work on higher education at Which? and now works for Wonkhe – concludes from the comparison of ratings that certain factors will be important, but also a challenge, for the TEF.
These include stability in the organisation delivering the ratings. Instability in the organisations delivering ratings presents challenges for regulators, providers and consumers alike. It will pose a challenge for the TEF, which comes alongside significant changes to the regulatory landscape, the paper says.
The use of a wide range of evidence, including good outcomes data and visits is important. In other markets regulators draw on a wide range of data, in addition to good outcomes data. The lack of good outcomes data in higher education will make wider sources more important, but will also prove more difficult in the absence of a visit, the paper adds.
Another concern is that, while there has been an increasing emphasis on involving experts and practitioners in delivering ratings and the TEF will draw on the expert views of a panel, this expertise could become diluted with the possible move to subject-level ratings.
In order to support choice, ratings in other sectors have covered all providers, and have been made available at more granular levels – for example, at the level of a hospital department. Subject-level ratings will be important for the TEF, the paper says.
Ratings often sit alongside wider comparable information, and their inclusion in league tables has been important. The TEF will need to be sufficiently weighted against other league table variables, such as degree outcomes and research excellence, the paper adds.
Data limitations
The paper found limitations in existing plans for using outcomes data in the TEF.
Under the TEF proposals, a good review outcome will act as a gateway to the TEF in year one. However, Quality Assurance Agency review outcomes relate to the quality of an institution’s processes, rather than actual quality or standards. It is at least theoretically possible that a provider with good processes delivers poor teaching, the paper says.
The TEF will also include an assessment of employment outcomes, based on the Destination of Leavers from Higher Education, or DLHE, survey and perhaps even government revenue collection agency, HM Revenue and Customs or HMRC, data.
However, the DLHE suffers from small sample sizes in places, and there are concerns about gaming. HMRC data will provide better information on earnings, which will be helpful for those students who are motivated by earning potential, but it is not a measure of quality. Better occupational employment outcomes data, such as the number of students in graduate-level jobs, is needed to inform the TEF.
The government does not currently plan to include degree outcomes in the TEF, in recognition of the incomparability of degree standards, and will encourage a move to Grade Point Average to recognise students’ achievements.
However, the proportion of ‘good’ degrees (2:1s or above) awarded by providers is likely to continue to be used within league tables, given the weight placed on the measure by employers and students. According to one survey 81% of employers use 2:1 and above as a cut-off as part of recruitment practices. This will have implications for the TEF, the paper says.
Informed choices
The report identifies two points for the government and sector to consider. It says there is a need for the TEF to be more deeply integrated with the quality assurance, and external examining, system. This will help minimise burden, and support the collection of a richer set of evidence at subject level.
Second, it calls for the TEF to be postponed to allow the findings from the technical consultation to feed into it. This would also allow sufficient time to consider how the TEF will fit with the quality assurance, and wider regulatory, system.
Darian said a well-designed TEF has an important role to play in supporting students to make informed choices.
“In order to do so, institutions will need to apply for higher-level awards, and ratings will need to be based on rich evidence and available at subject level.
“While moving to an Ofsted-style scheme would be difficult in higher education, government and the sector should consider how to further integrate the TEF into the quality assurance system. The government should also postpone the first year of the TEF to ensure that it delivers meaningful information to students.”
