Login

close

Login

If you are a registered HEi-know user, please log in to continue.


Unregistered Visitors

You must be a registered HEi-know user to access Briefing Reports, stories and other information and services. Please click on the link below to find out more about HEi-know.

Find out more
British Council launches MOOC to help overseas students study in the UK

A new online course - the first of its kind - has been developed by the British Council to help international students prepare to study at a university in the UK.

Access to HE has a bright future after 25 years of achievement

Val Yates, Director of Access and Inclusion at the University of Worcester, raises the curtain on an annual access and continuing education event, now in its 25 th year, taking place at her institution this week.

Universities with top social media reviews do well in NSS and TEF

Universities that are rated highly on Facebook and other online review sites tend to do better in formal measures of learning and teaching, new research suggests.

HEi-think: High quality teaching does most to boost value for money perceptions

Key findings of the latest Student Academic Experience Survey from Advance HE and the Higher Education Policy Institute are outlined and examined by Jonathan Neves, Advance HE Head of Insights and author of a report on the survey results.

Rating boost for half of institutions that reapplied to TEF 2018

Half of the institutions that reapplied to the Teaching Excellence and Student Outcomes Framework in 2018 saw their rating improve.

CUC publishes new code on senior university staff pay

The Committee of University Chairs has published a new voluntary code for setting the pay of senior university staff.

New universities leap ahead in latest Guardian rankings

New universities are challenging the historical hierarchy in UK higher education  as a significant number leaped ahead of Russell Group institutions in the latest Guardian league tables .

HEi-think: REF 2021 shapes up sensibly without any “bonkers” rules

Professor Nick Talbot, Deputy Vice-Chancellor for Research and Impact at the University of Exeter, examines the new rules for the 2021 Research Excellence Framework and finds that, save for one or two concerns, they represent a sensible approach.

 

It is hard to see beyond the gloom of last week’s Budget and the prospect of another decade of stagnant economic growth. One thing is very clear, however. The UK university sector has never been so badly needed.

If we are to create the knowledge economy of the future– the digital revolution that the Chancellor talked about, with autonomous vehicles and new industries built upon artificial intelligence –then we will definitely need world class universities. We will need them to carry out cutting edge discovery science, innovation and commercialisation of ideas, and to train the highly motivated graduate workforce of tomorrow.

Well the good news for the Chancellor– and heaven knows he needs some –is that we are up for that challenge. The latest data commissioned by BEIS shows that the UK, with 0.9 per cent of the world’s population, 2.7 per cent of its R+D spend and 4.1 per cent of the global researchers, generates 6.3 per cent of the world’s research papers, 9.9 per cent of the downloads, 10.7 per cent of citations, and 15.2 per cent of highly cited articles.  Indeed, one of the constants in all the Brexit declarations to date, by both Leavers and Remainers, has been to stress the strength of the UK university sector. The Leavers use this to illustrate how the UK will prosper outside the EU, while Remainers stress how much harder the HE sector is going to find things outside of the funding structures and collaborative networks facilitated by EU membership.

So, in short, universities are going to matter a lot to the UK’s fortunes. As, of course, they always have, but now with a lot more government attention. To help them stay focused on their key tasks, universities will henceforth be evaluated by ‘excellence frameworks’ in all three main areas– Research, Teaching and Knowledge Exchange.  Whether this REF, TEF, KEF triple evaluation is actually necessary or not, it is happening, and we therefore need to know the rules– indeed, hopefully we can even help to craft them, so they are actually effective tools.

Of all the evaluations, the REF is the most mature, having evolved from the Research Assessment Exercises stretching back to 1986, and last week we saw the much anticipated publication of the rules for REF2021. To our relief, much of what is proposed had been previously aired by HEFCE with the sector and shaped by extensive consultation. Indeed, to use Steven Hill’s own phrase, his team worked hard not to come up with any rules that were bonkers. I think in the main they were pretty successful. 

The new REF rules

So what did we find out? Well, first of all we know that everyone has to be returned in REF2021.  The REF will define Category A staff as those academics with ‘significant responsibility’ for research. Significant responsibility can be defined by universities as having time in a contract allocated to research, or PhD supervision responsibilities, for example, and a ‘menu’ of ways in which ‘significant responsibility’ can be defined will be provided to help institutions. But the onus will be on universities to justify why a staff member is not being returned to the REF, rather than the other way around. In other words, the default will be that research-intensive universities will submit all their staff, as Stern originally suggested, and more teaching-focused institutions will need to define their research-active staff very carefully and be prepared to evidence the selection.  This is clearly a good thing overall, which should show in a more transparent manner, where the most significant volume of research is being carried out.  The evaluation itself will produce a quality profile based on assessment of outputs (publications, predominantly), impact case studies and the research environment.  There will, however, be a greater emphasis on impact, now scoring for 25 per cent of the quality profile, instead of outputs, scoring 60 per cent now instead of 65 per cent in REF2014.

The number of outputs required will be 2.5 per FTE, with a minimum of 1 and maximum of 5 returned for any individual. This reduction in number of outputs, will enable REF panels to evaluate approximately the same number of outputs as the last exercise, albeit produced by a much larger volume of academic staff, given that most institutions were selective who they returned in the last exercise.  The link between an individual academic and the evaluation of a subject area is also not as strong as previously (although not completely broken as Stern suggested), because of the variation in number of outputs required.  This means a department can showcase its very best work, without worrying quite so much about attribution to individuals.

We also see the expected changes to portability of outputs, which can now be submitted by both the university employing the staff member on the census date and the original institution where they were employed when a research output was “demonstrably generated”, which will normally be determined by the date when it was first made public.  This is a reasonable compromise that allows mobility of staff to be rewarded, while also rewarding institutions that supported the original work.  We shall see how it works in practice and, as I have said previously on Media FHE, I doubt it will prove material in any funding calculations.

The other consequence of the new rules on portability are that outputs can be submitted from staff who have moved into different sectors, retired or even died. So even death is no escape from the REF….although I presume that a department with a large proportion of dead staff would score less well in the environment section.

The extra emphasis on impact will not, however, require an increase in the number of impact case studies. Each submission will require one impact case study, plus 1 further case study per 15 FTE. This ratio is maintained up to 105 staff, and thereafter only 1 per 50 staff is required, so that very large units do not have to find such a large number of impact case studies as in REF2014.  So the overall number of impact case studies evaluated will be broadly equivalent, based on increased staff submissions, or perhaps even less than in REF2014, and previously used case studies can also be updated, if new impact has emerged.

Finally, on open access, papers will need to be deposited within 3 months of acceptance after April 2018, but “deposit exception” will also be in place so that outputs will remain eligible as long as they are deposited up to 3 months after online publication, extending the current policy to allow full compliance to be achieved in a pragmatic way.

So, what are we to make of the REF2021 announcements?  Well, most importantly there were no surprises and nothing that is at odds with the spirit of the Stern review, but HEFCE have listened to the sector’s concerns about the feasibility of some of the earlier suggestions which have been quietly dropped, or kicked into the long grass (institutional impact case studies, for instance). All very sensible.

Some concerns

There remain, however, at least a couple of concerns on my part. Will we really get the resolving power necessary to distinguish between submissions from our very best universities, when only 2.5 publications per FTE (from a six year census period) are being assessed?  If scoring is similar to REF2014, then some Units of Assessments (UoAs) will have more than 80 per cent 4* outputs on such a measure, and some UoAs were already close to, or at 100 per cent intensity last time. How will sub-panels, and indeed main panels, ensure they really can resolve between our very best departments, especially in subject like Physics, Chemistry and Biological Sciences where publication rates are so high?  A challenge I think, that might require some different scoring criteria. 

HEFCE, or soon to be Research England, may say it doesn’t really matter and be content to give similar amounts of QR to the best 10 institutions in some UoAs. It is the sector after all that obsesses about league tables, rather than the funding body, as David Sweeney has said on occasion.  But, I think the power to resolve between the very best performing institutions has been a hallmark of the recent exercises and one that have given them a lot of credibility internationally, so some care is needed.

The other concern has to be the game-playing around 100 per cent submissions that will now take place across the sector and how that can be safeguarded against. I hope we do not see a lot of unusual activity in HR departments, busily changing contracts of employment for academic staff. That would be so against the spirit of what is being proposed, but also a logical action for a large research-intensive university that isn’t actually as research-intensive as it likes to say in its prospectus and on its website. 

Finally, it is worth saying that this will be the 8th iteration of Research Assessment since 1986 and we are, finally, getting to a pretty fair, defensible and equitable process.  Most academics like to complain about the REF– but only in the way they complain about car parking, workload models, administrators, and research funding –that is, they don’t really object to the integrity of the process, they just don’t like rules very much, or not getting funded of course. 

I worry that it has taken over thirty years to achieve a pretty good design for REF and that we have embarked on one round of TEF so far, which some leading institutions shunned, and have now started planning the KEF. If all three of these assessments are actually necessary– and I am not really sure the KEF is actually required to be frank –then I would implore Research England and all those involved through UKRI, to learn the lessons of REF (I actually remember the lunacy of RAE 1986 and 1989!) and engage long and hard with the sector, so they can avoid doing anything that is well….bonkers.

 

 

filmfoto / 123RF
Back