How academics are assessed: Rewarding how we do research and not just where it gets published

All academic researchers currently want to see their research published in the ‘best’ possible journals, which means those with the highest impact factors (IFs). An IF basically translates to readership, so a journal with a higher number will be read by more people. However, this does not necessarily mean that an individual paper published in a journal with an IF of 5 (for example) is going to be read and cited at all: the current overwhelming international use of IFs as metrics to assess the ‘quality’ of both literature and individual academics is increasingly being viewed as flawed, and not fit for purpose. 

Why is this?

Well, citation distributions within journals can be highly skewed, as some papers are read and used while others are not. This has led many academics and policy makers to prefer the use of individual article metrics or scores when assessing the impact of outputs. Similarly, the properties of journal IFs are very field-specific: the outputs from some research disciplines just tend to get published in ‘better’ journals that other disciplines!

We also need to think about the kinds of papers that appear in academic journals. One might publish an invited review article in a journal with a high IF: currently, this output would be given the same weight in academic assessment as primary, peer-reviewed research appearing in the same outlet. Is this fair?

Finally, and perhaps most concerning, there is the suspicion in some quarters that journal IFs can be manipulated (‘gamed’) by editorial policy, while the underlying data used to calculate these values are neither transparent nor openly available to the public.

As Jeremy Farrar, Director of the Wellcome Trust, wrote in a recent opinion piece: ‘it’s time to reimagine how we do research as the current emphasis on excellence in the research system is stifling diverse thinking and positive behaviours’. Certainly, this extends to article IFs, as within our current publishing ecosystem most researchers would do pretty much anything just to get their work into the ‘leading journals’, those with the highest values as assessed by IF. 

Is there an alternative?

Yes. There are strong, positive movements within the academic community towards fairer and more research-centric methods of assessment. 

The San Francisco Declaration on Research Assessment (DORA) is the highest-profile example of an academic-driven policy proposal in this area (https://sfdora.org/read/). Originally framed at the 2012 Annual Meeting of the American Society for Cell Biology in San Francisco, DORA has evolved to become a worldwide initiative covering all scholarly disciplines and all key stakeholders including funders, publishers, professional societies, institutions, and researchers.

DORA is a movement that essentially aims to ‘advance practical and robust approaches to research assessment globally and across all scholarly disciplines’. Its broad aims are to: 

• increase awareness of the need to develop credible alternatives to the inappropriate uses of metrics in research assessment;
• research and promote tools and processes that facilitate best practice in research assessment, and;
• extend reach and impact across scholarly disciplines and in new areas of the world.

The global community of academic researchers are obviously a hugely diverse group of people. One thing that almost all, if not all, would agree on is the need for more ‘research-centric’ methods of assessment that move away from the current overarching, unquestioning focus on journal IFs. For this to actually happen however, funders, national governments and research organisations (as well as the academic publishing industry) will need to be brought on board. We’ve all heard the English expression ‘it’s going to be like herding cats’!

At Charlesworth Author Services, we know and understand the academic community because our expert editing and language polishing services are driven by experienced, active researchers, all of whom have a minimum PhD-level qualification. This sets us apart from our competitors. 

Our team provides a range of author services, including expert English language editing, cover letter review and editing, abstract writing, pre-peer review, journal selection, and consultancy: all designed and proven to significantly enhance your chances of being successfully published in your preferred journal.

Why not get in touch with a member of our Charlesworth Author Services team to discuss how we can help you? To find out more click here.

Our academic writing and publishing training workshops, online course materials and blog articles can provide you with a toolkit to help you navigate academic writing and publishing, and maximise your potential as a researcher. More details can be found here.

Maximise your publication success with Charlesworth Author Services.