If you are a registered HEi-know user, please log in to continue.

Unregistered Visitors

You must be a registered HEi-know user to access Briefing Reports, stories and other information and services. Please click on the link below to find out more about HEi-know.

Find out more

REF2014 – results, comments, debates and media

University research in the UK has improved significantly and is leading the world, according to the Higher Education Funding Council for England (Hefce) as it announced the results of the 2014 Research Excellence Framework.

Nearly one third of work submitted – 30% - was judged to be world-leading (4*) and a further 46% was rated internationally excellent (3*).

For submitted work alone - weighted at 65% in the formula to assess excellence – the council said the proportion judged 4* had jumped from 14% in 2008 to 22%. There was a similar rise in work judged 3* which went up from 37% to 50%.

Of the 1,911 submissions, including 52,061 academic staff, 191,150 research outputs and 6,975 impact case studies, 20% were rated 2* recognised internationally and 1* recognised nationally.

David Sweeney, HEFCE’s head of research, education and knowledge exchange, announcing the results in London. He attributed the rise to increased investment in UK research over the period and the country’s ability to attract the best brains from abroad.

“Investment in our own people and the attraction of people from abroad to build up our research has improved UK research over the last seven years by any comparator you look at. The UK is becoming the best place in the world to do research,” he said.

He dismissed claims of “grade inflation”, saying the REF2014 results were in line with other measures of UK research performance, such as studies on the number of journal citations.

HEFCE also pointed to the trend towards increased collaboration across universities here and abroad, and work done by inter-disciplinary teams.  Nearly a quarter – 24.5% - of journal articles submitted had authors from more than one UK university, and 40.5% had at least one author from outside the UK, it said.

In a similar pattern to 2008, the highest rated research was concentrated in a minority of large, research-intensive institutions. The Russell Group said its 24 universities were producing research “on a grand scale”. Though they represent only 15% of institutions they had produced 68% of 4* research and 68% of the 4* impact judgements.

The results will reinforce the concentration of research funding on a small number of institutions, said Professor Michael Gunn, the Vice-Chancellor of Staffordshire University and chairman of new universities think tank Million+.

“In the last five years government priorities and polices has resulted in the funding councils redefining excellence which has led to the hyper concentration of taxpayer-funded investment in fewer universities.  Ministers and the funding councils need to deliver a more balanced funding formula that invests in all universities and provides a broader research base across the country,” he said.

North-South divide?

The gap between research output in London and the south-east and the north of England appears to be widening, with several northern universities slipping down the league tables produced by various organisations from the data. Research Fortnight claimed the north was “taking a hammering” based on its league table which showed Manchester, Liverpool, Sheffield, Birmingham and Leeds slipping slightly.

The BBC claimed that the Oxbridge grip on research in the UK had been broken for the first time by the rise of London’s elite universities.

Professor David Price, Vice-Provost for research at University College London, said that the old order, which has seen Oxford and Cambridge claiming the top two positions, had been disrupted for the first time in history. 

"UCL breaking in to that is great," he said. "We have done excellently across the board. We have by far and away the best research power in biomedicines also in STEM. We also won top spot in social sciences which will be a surprise to some. London has about 25 per cent of the HE in the country, that's historic, and quite a lot of it is good. Part of problem might be that London attracts international staff more than some other cities."

Excellent work is going on in the northern universities, but some had submitted fewer academics to the exercise than others in the SE, affecting their position in tables measuring volume as well as research ratings, said HEFCE.

For the first time, HEFCE assessed not just the content and results of the research but its social, economic and cultural impact which counted for 20% of the assessment. Impact had been expected to boost the position of new universities, but HEFCE said the distribution of excellence across the sector was “much the same as last time” it was measured in 2008.

There was some excellent work going on in the new universities and some had made considerable steps forward but others did not see research as a key priority, said Sweeney. 

Three quarters of universities had at least 10% of their work graded 4*, and the top quarter had at least 30%. Three-quarters had at least 49% graded 3* and above.

Several organisations used the data to construct league tables of universities. Research Fortnight put Oxford at the top with UCL second, pushing Cambridge into third place followed by Edinburgh, Manchester and Imperial.  The Times Higher Education rankings puts the Institute of Cancer Research first followed by Imperial College, LSE, Oxford, Cambridge, and Cardiff. Meanwhile WonkHE produced its own table based on GPA including a measure for "intensity". This also places the Institute of Cancer Research top, followed by Cambridge, Imperial, UCL and Oxford.

Critics accused some universities of manipulating the figures by submitting only their best researchers to the exercise. Based on figures from the Higher Education Statistics Agency (HESA) for the number of staff submitted by each institution as a proportion of those who were eligible to take part, the University of Loughborough produced a very different table.

It put Imperial College London top with 83.28% of eligible – rather than submitted – staff rated 3* or 4*. Next came Cambridge with 82.96% , Bristol with 76.11%, Oxford with 75.70% and Southampton with 75.64%.  Loughborough itself came 17th with 65.47% of all eligible staff in the top two categories. Professor Robert Allison, Loughborough’s Vice-Chancellor, said 88% of staff had been submitted, including some PhD students.

“We took an inclusive, collegiate approach because our mission is a balance between research, teaching, enterprise and sport. Our world-class academics are not hidden away. Students are paying £9,000 a year and they deserve a 9 in 10 chance of being taught by someone recognised for excellence by the research community,” he said.

According to Sweeney, there was no more selectivity of staff to submit this time than there was in 2008.

“The same number of people who were eligible to be submitted were entered. So we have not seen any more selection,” he said.

Playing the numbers game

Despite warnings from HEFCE about the misuse of grade point average – the scores achieved divided by the number of academics submitted – universities were marketing themselves on the scores as soon as the embargo on the data was lifted. 

“HEFCE’s overall quality profile for subjects makes a nonsense of the whole exercise,” said one vice chancellor.  “Across the world Cambridge is known for the excellence of its physics department yet Strathclyde came top in the HEFCE data because it had 40% of its staff rated 4* compared to 38% at Cambridge. Then you look closer and see that Strathclyde submitted only 27 researchers and Cambridge 153.”

The HEFCE subject tables also show Bedfordshire doing better than Oxford for English based on 4* ratings for the 45% of its nine staff submitted compared to 42% of Oxford’s 88 staff.

In an HEi-think article on the Media FHE website, Professor Nick Talbot, the Deputy Vice-Chancellor for research and knowledge transfer at the University of Exeter, said some institutions had adopted highly selective strategies, returning only a small proportion of staff for evaluation.

“Of course, we all look for ways of showing ourselves in the best possible light, in spite of these considerable differences in REF strategies, but dispassionate observers will be somewhat bewildered by the data.  Is it credible that a less prominent university ranks higher for English than Cambridge, for instance, even though it returned only three of its researchers?  That is how some league tables will show the data and how some in the media may interpret the outcome, “ he said.

Among the movers, the University of Worcester was the biggest riser in the Research Fortnight table, climbing 32 places in the rankings after submitting 112 academics compared to 25 in 2008. Professor David Green, its Vice Chancellor, said Worcester had done very well on the impact category. 

“Research carried out in areas from tackling the obesity epidemic to how to stimulate children's imaginations through art and illustration is having a growing impact for the good. The huge increases in the quality, quantity and range of the research work carried out at the University, despite shoestring government funding over the past 12 years, are a tribute to all the individual researchers involved,” he said.

The THE named Cardiff Metropolitan University as the biggest riser based on grade point average. It moved 62 places from 103rd to 41st in the THE rankings.  SOAS in London was the biggest faller in its table, dropping to 31st in 2008 to 61st.

Other notable risers include the University of Exeter, which moved up four places in the Research Fortnight table, Cardiff, moving up 16 places in the THE table, and Bristol -- up three places in both the THE and WonkHE rankings and rising by one place in the Research Fortnight table.

Some universities focussed on their best results in certain subject areas. Sheffield Hallam University, where 65% of research submitted was rated as 4* and 3*, pointed to its art and design research ranked first among 23 modern universities and in the top ten of all UK universities.

Professor Paul Harrison, Pro-Vice Chancellor for Research at Sheffield Hallam, said: "These fantastic results are yet further evidence that world-class research exists across the UK's universities, and not least in modern institutions like Sheffield Hallam University."

Funding decisions to come

The results will be used by HEFCE to distribute quality related funding (QR) and by the research councils to make funding decisions. QR funding stands at £1.6 billion this year and has yet to be announced for next year.

HEFCE said it would be allocating funds on the basis of excellence – currently to 3* and 4* work -  raising questions about how the increased amount of work that would be eligible under the 2014REF could be funded from a limited pot of money.

Professor Stephen Decent, the Pro Vice-Chancellor for research at Lancaster University, said the REF gives the sector the opportunity to lobby for increased public funding.

“ Lancaster has risen to 15th for world leading 4* research that puts us amongst much larger universities showing the REF can successfully measure quality and not size,” he said.

However, the sector waited with anticipation on what the results would mean financially, he said.

“If there are further public spending cuts, I would anticipate an increased ratio between the funding allocated for 4* and 3*,” he said.

Reducing the amount of funding for 3* research would be a blow to emerging departments building up their teams and further increase the concentration of research in a minority of institutions.

The University of Derby more than doubled the number of staff submitted and increased its proportion of research judged to be 2* and above from 66.6% in 2008 to 74.5%. Professor Paul Weller, its research excellence framework manager, said the disciplinary footprint had doubled from four to 10 without any public investment and now several departments would be eligible for quality related funding.

But any further moves in the direction of concentrating funding on the top 4* category would render the REF descriptors inconsistent, he said.

“Work recognised nationally and internationally is no longer funded and if they decide internationally excellent is not worth supporting either there will be a danger to the entire system. Take the analogy of the Premier League in football, the big clubs couldn’t exist on their own.”

HEFCE said research funding was not a finite pool because universities could attract research funding from many sources.

“Of course, we want to build the case for public investment and I think this exercise does that and we will be putting the case to government that we have very valuable work going on and if we invest more we will see a bigger return in terms of impact on society and the economy,” said Sweeney. “We have found excellence in many, many universities across the country and the policy is to reward excellence wherever it is found. I hope that policy will continue but everyone needs to make heir own judgements,” he added.

The four funding councils will use the information in different ways, but all will be “making the case for further investment” said Madeleine Atkins, HEFCE Chief Executive.

“The results confirm that UK universities are in the top research rank of an international community. From an already strong position in the last exercise in 2008 our universities are delivering more ground-breaking work of the highest quality,” she said.

Sir David Greenaway, Vice-Chancellor of Nottingham University – which came seventh in the THE rankings and eighth in the Research Fortnight table - said he hoped that the return on investment demonstrated in the results would persuade the Treasury to look again at the resources that could be devoted to university research.

“The results are consistent with international studies such as the one carried out by Universitas 21 on national systems, which showed the UK performing extremely well on a low level of resources,” he said.

While universities digested the results, HEFCE said it had already put in train programmes to assess the exercise, and will be evaluating the way that universities prepared for the new impact category and its cost. Stephen Hill, HEFCE’s head of research policy, said it was calling for feedback.

“In many ways work starts today in terms of understanding this whole exercise, what it means and how we can improve it for the future. All this work will be published in March and will form the basis of evidence on which we start discussions about the future,” he said.

In addition to evaluating the current exercise HEFCE has set up a group chaired by Professor James Wilson at Oxford University to look at the question of how metrics might be used.

“We are also looking at ways we can benchmark internationally, playing a part in the global search for excellence,” said Hill.