If you are a registered HEi-know user, please log in to continue.

Unregistered Visitors

You must be a registered HEi-know user to access Briefing Reports, stories and other information and services. Please click on the link below to find out more about HEi-know.

Find out more
Moving the HE landscape’s quality contours … again

The government's announcement of a major review of the National Student Survey signals a worrying shift in the HE regulatory landscape, warns Jon Scott, higher education consultant and former Pro Vice-Chancellor (student experience) at the University of Leicester.

Government plans mark a seismic shift in higher education policy

Statements from ministers this week have made it clear that higher education in England is facing significant reforms, re-setting its focus towards helping to plug the UK's skills gaps and rebuilding the economy. Fariba Soetan, Policy Lead for Research and Innovation at the National Centre for Universities and Business, argues that the proposed changes bring a welcome focus on graduate outcomes and supporting the careers of young people.

UK universities affirm 'deep commitment' to high quality TNE

Universities UK and GuildHE have commissioned the Quality Assurance Agency to develop a new approach to reviewing and enhancing the quality of UK TNE. QAA will consult on a new review method later this year and will launch a programme of in-country enhancement activity in 2021.

Cassandra calling out higher education

After a week of largely disappointing news for UK higher education, Nicola Owen, Deputy Chief Executive (Operations) at Lancaster University, fears that gloomy forecasts for the future of the sector may prove to be uncomfortably accurate.

Whatuni award winners announced

Loughborough University has been named University of the Year for the second time in three years in the latest Whatuni Student Choice Awards .

Welcome to the higher education rollercoaster

UK higher education had more than its fair share of ups and downs over the past week. Charlie Ball, Head of Higher Education Intelligence at Prospects, charts the highs and lows.

Universities must re-think recruitment strategies as rules change on unconditional offers

As the Office for Students places a moratorium on ‘conditional unconditional offers’, Jon Scott, HE consultant and former Pro Vice-Chancellor (student experience) at the University of Leicester, reviews the context of the decision and considers its implications.

HEi-know Good Practice Briefing: UK universities describe "amazing" shift to online delivery

Universities across the UK have rapidly moved their learning, teaching and assessment online in response to the COVID-19 pandemic. The unprecedented overhaul of traditional teaching practices has presented a major challenge to institutions, staff and students. In this Good Practice Briefing, HEi-know shows how some universities have responded to the situation.

World events highlight stark inequalities in HE

Sutton Trust associate director of media and communications Hilary Cornwell and research and policy assistant Maariyah Dawood comment on equality and widening access issues that have emerged in a week of higher education news.

HEi-think: Implementing Stern - how will REF2021 shape up?

As the Higher Education Funding Council for England launches its long-awaited consultation on the second Research Excellence Framework planned for 2021, Professor Nick Talbot, Deputy Vice-Chancellor for Research and Impact at the University of Exeter, examines some of the key issues it raises.


We live in interesting times in the Higher Education sector. We have been rocked by the implications of Brexit and, most recently, the proposed introduction of the Teaching Excellence Framework, and the enormous consequences of the Higher Education and Research Bill, which will lead to important changes to governance and regulation of the sector and, indeed, the autonomy of universities.

This is coupled with the likely introduction of UK Research and Innovation (UKRI), the new single body that will oversee the research base of the country.  These are profound changes, the effects of which will be felt for many years to come. We therefore might look to some comforting constants at a time like this, such as our old friend the Research Excellence Framework (REF).

But that too is about to change.  The next REF, which will happen in 2021, will be guided by the recommendations of the Stern Review (see HEi-know Briefing Report 310), published in July, and now HEFCE has published its long-awaited consultation (see HEi-know Briefing Report 329) on how best to implement Stern’s recommendations.

This is not a straightforward exercise, and the 44 questions in HEFCE’s consultation paper will require careful thought and some creativity from the sector.  There is a lot at stake– the REF will determine the distribution of £1.32 billion of mainstream quality-related (QR) funding.  In fact, given the welcome news in the Autumn Statement, this might increase substantially, perhaps restoring the purchasing power of QR that has been eroded by inflation since its last substantial uplift. 

So what are the major questions in the HEFCE Consultation that will exercise the minds of leaders in the sector? The Stern Review was clear that the REF was important and valuable, should be driven primarily by expert peer review, but that the administrative burden of REF2014 (which was estimated, in a hotly disputed study, to cost £246 million) was too high. 

Stern therefore said that every effort should be made to reduce the cost and burden of the REF. All of the review recommendations, and HEFCE’s desire to implement them, must therefore be measured against how they could contribute to reducing administrative cost and this, more than anything else I think, will be hard to achieve.

The HEFCE consultation earnestly presents the dilemmas thrown up by the Stern review and is a thoughtful and well crafted piece of work.  It will, however, require an equivalent effort by the sector before the March 17th deadline to produce a REF that has the best elements of Stern (of which there are many), but that can actually be implemented fairly across the diverse university landscape, as a single basket of measures.

One key element of Stern was a desire to de-couple the REF evaluation of subjects from any assessment of individual academic performance.  This is admirable in principle, and indeed was widely welcomed, but as the consultation document makes increasingly clear, it is quite hard to actually implement, in part because of some of the other Stern recommendations, which will undoubtedly require mapping of outputs and impacts back to individuals.   

Stern challenges

The biggest changes recommended by Stern, for instance, were that 100 per cent of research-active staff should be submitted to REF2021 and that publications submitted to the process should no longer be portable. That means that research papers or authored books could not be taken with a staff member to their new employer when they moved university during an assessment period, but must be evaluated where they were produced.  Connected to this was the view that there should be flexibility in the number of outputs that could be returned per member of staff in a unit of assessment (UoA). Stern initially recommended that the number of assessed publications should be set at 2 per FTE in all panels, with between 0 and 6 being acceptable limits.  Both of these recommendations therefore will require mapping of outputs back to individuals in some way.

How then have HEFCE proposed that these key recommendations be implemented, and what areas might prove problematic (and perhaps even default back to REF2014 methodology)?

The issue of 100 per cent return of academic staff is very sensible and, I think, very widely supported.  Here, HEFCE proposes that ‘research-active’ staff should be identified using the Higher Education Statistical Agency (HESA) classification, that records academic employment function as ‘Research’ or ‘Teaching and Research’, with a measure of independence used to filter out research assistants as non-eligible.  There is also a suggestion that staff are mapped to UoAs using HESA cost centres (a discipline classification).

This is perhaps not too far removed from the current situation, but still needs some thought. Many universities are now deeply interdisciplinary in their activities and, indeed, their structures.  Many do not even have the traditional department structure, so beloved by the REF.  This makes coding staff to different UoAs difficult and (I hate to bring it up) administratively burdensome.  There are mathematicians studying climate change, social scientists studying mining, psychologists studying business and so on. Universities in 2016 are inherently interdisciplinary, which is part of the reason we are so successful. 

It may be that this is surmountable by more cross-referral between REF panels, but I think that universities must have the freedom to return staff where they fit most appropriately. This is good for interdisciplinarity and for rewarding the best research wherever it is found.  There must, of course be safeguards against large-scale movement of research inactive staff into UoAs conveniently not returned to REF by an unscrupulous university to disguise a lack of research intensity.  This can be audited via the annual staff return made to HESA, as long as it is made clearer in REF returns.  

Outputs conundrum

Given the increase in staff likely to be submitted to REF2021, what about setting the number of outputs to 2 per FTE, with a minimum of 1 per FTE as the entry standard?  Introducing a minimum entry level of 1 output per FTE seems very sensible, but will the ratio of 2 per FTE be sufficient to discriminate between submissions containing a large amount of high quality work?  If REF2014 results are anything to go by, then the answer in many STEM disciplines is probably no. Top performing units could have near to 100 per cent of their outputs in the top 4* category, with little discriminatory power between the best units.

So what is the answer? Remaining with 2 per FTE and introducing a 4** or 5* category? Let’s hope not, and thankfully HEFCE have not recommended it.  My view would be that 2 outputs per FTE is probably acceptable for many humanities and social science disciplines, but in STEM and Medicine the ratio should remain at 4 per FTE, as in previous exercises.  I would, however, favour a random sampling methodology to ensure that workloads remained acceptable on those panels, coupled to greater use of metrics, which work reasonably well in these disciplines.  I would link any desire to increase from 2 outputs per FTE to a requirement to accept sampling methodologies. 

The flexibility in numbers of outputs will, of course, also require mapping of outputs back to individual academics, which breaks one of the Stern principles.  However, there is probably no other way to achieve this, but there must be safeguards against unintended consequences. Some disciplines have extensive co-authoring for example, and individual names could appear many times in a list of publications from a UoA.  The attribution of the outputs must, therefore, clearly be shown to ensure we do not suddenly limit co-authorship and introduce a disincentive to collaboration.  

The other major conundrum I think will be in the thorny issue of portability of outputs.  This is being recommended for the best of reasons– to limit the transfer market in academic staff close to REF census deadlines, along with their publications.  This undoubtedly makes managing universities a challenge at times, but the dynamism in the sector and the power of the individual academic are also key features of the UK HE sector that have, arguably, been important to its success.  So some flexibility is probably required.  The problem is that this necessary flexibility may equate to administrative cost in tracking staff and their publications and correctly attributing them to different universities.  

HEFCE propose that the acceptance point for a publication should count as the means of identifying which university should claim the work.  This may be relatively straightforward in STEM disciplines, but it will be much harder where books and monographs are concerned, especially considering their (often) very long gestation periods.  Indeed, one of the biggest costs we foresee in implementing Stern would be in tracking publications for staff no longer on the payroll, so this will require some creative thinking.  The mandatory use of Open Research and Contributor ID (ORCID) will help in publication tracking, as will institutional green open access repositories, but it will still be costly.

I am of course cherry picking the biggest and, in my own personal view, the most challenging aspects of implementing REF2021.  There are many other recommendations in the consultation that are very sensible, however, and actually quite easy to implement, such as the institutional environment statement (great), reduction in UoA-level narratives, case study ratios, and weightings.

In other areas I take stronger issue.  I am not convinced, for example, that we need a deeper, broader definition of impact.  The REF2014 definition seems to me to be widely understood and accepted across the sector.  Introducing ‘academic impact’ seems to me a retrograde step, as the other components of REF measure that anyway. The definition of ‘wider impact’ was what I understood impact to be in any case, so seems to be sufficient to me. Why complicate something that worked well in REF2014?

So, please enjoy the 44 questions during your Christmas break and beyond and try to be creative before March 17th.  I think we all need to be with so much funding to win (or lose).

etiamos / 123RF