The Research Excellence Framework, whose results appeared in the normally news-free days before Christmas, will be a decisive influence on UK university funding for years to come.
But as with any ranking of universities, the REF findings had barely been published before new versions started to emerge. At the last count, at least five institutional tables have been published plus numerous subject rankings.
Given the number of REF rankings already in existence, why add another? Each table has its own methodology, producing sometimes wildly different results. Media FHE has evened out the effects by creating a new REF top 50 "table of tables", drawing on rankings from the Times Higher Education, Research Fortnight and The Times as inputs. We hope this provides an accessible and balanced look at all possible approaches to the REF results.
The Media FHE results confirm the dominance of the “Golden Triangle” of Oxford, Cambridge and London. Oxbridge, plus Imperial and University College London, have opened up a decisive gap on Bristol and Edinburgh in joint fifth place, and on Manchester, the top university in the North of England in 11th place. King’s College London is in seventh place and can also look forward to increased research funding. Specialist London institutions such as the Institute of Cancer Research (top in the GPA listing), the London School of Economics and the London School of Hygiene and Tropical Medicine also show up well here. Universities from outside of London in among the top rankers include Edinburgh, Warwick and Southampton. Below these, there is a large upper middle group headed by Manchester and including a very tightly packed collection of 14 institutions topped by Sheffield. Tucked underneath leading the next layer down are Bath, Reading, Loughborough and Strathclyde.
The original REF approach uses research excellence, impact and the quality of the research environment to assess excellence. It is reflected here in each institution’s Grade Point Average (GPA), essentially its average score for all the units of assessment (ie subjects) it entered, as published by THE. The Research Fortnight approach is designed to translate these results into money. It works by adjusting the results for the total number of staff submitted and the number of 3* and 4* subjects, these being the ones that will attract funding council cash. Its findings are probably the best guide to the funding that will flow from this REF.
The THE compares institutions overall and in a separate table corrects for the percentage of academic staff they submitted to the REF, producing an "intensity" ranking. This compensates for academic game-playing and yields some intriguing surprises, such as a strong showing for King’s College, London. The Times examined the percentage of staff in 3* and 4* departments, again allowing for the estimated proportion of staff submitted for the REF.
In both the Research Fortnight analysis and the Media FHE synthesis, the University of Oxford emerges as the UK’s top institution, despite coming fourth on the raw GPA analysis. Some rankings have suggested that Oxford is losing ground to Cambridge and to Imperial College London. But these results mean that the country’s oldest university is now set for some years of well-resourced research.
The exact formula which translates the REF results into sums of money is yet to be determined. If austerity continues to rule UK public spending, even the winners here may not be completely happy with the end result.
A further twist is that although the REF was carried out on a UK-wide basis, the devolved administrations can implement it as they see fit. We list eight Scots universities in the top 50 published here, a creditable result that suggests tricky spending choices for the Scottish government. Both of the universities of Northern Ireland also feature, again a positive result. In particular, Queen’s University Belfast does well on this analysis, up from 42 on GPA alone to 23 here. There might be less pleasure at these results in Wales, since only two of its many institutions figure here. They are Cardiff and Swansea, both losing ground in this methodology, especially because of their poor showing in The Times analysis.
These varying approaches will each have their supporters, but problems remain. One is the use of a single system to assess the future funding of expensive research in science and medicine alongside cheaper subjects such as the humanities. It is certainly possible that the Funding Council’s current review of the REF, including its new impact component, will lead to changes in the way this REF is used and future assessments are carried out. The cost and scale of the REF are bound to come under scrutiny at a time of pressure on public spending.
Despite these issues, the REF has one definite use, in UK university rankings. Most rankings systems use it, with the exception of The Guardian. This new data will cause a one-off change in the research rankings component of these systems when their compilers get to work on the next edition of their tables. In the longer term, rankers may decide that the REF is too slow-moving for their needs, and start to build some other measure of research, such as citations, into their metrics.
Martin Ince is chair of the Advisory Board for the QS World University Rankings. He was adviser on REF impact for Goldsmiths (University of London)
Media FHE REF Table of Tables methodology:
The table is compiled by allocating scores for the top 50 institutions in the Times Higher Education GPA Overall and Intensity league tables and the Research Fortnight league table, plus research subjects listed by institution in The Times top 50 table for all subjects. For the THE and Research Fortnight tables, scores correspond to rankings (50 for first place, 49 for second, and so on). One bonus point is then added for each subject listed in The Times table. Institutions that did not appear in the top 50 of either of the THE or the Research Fortnight tables were not counted. Media FHE makes no claims for statistical rigour and acknowledges the methodological limitations.
Top of the Media FHE table: Oxford University