Misleading research metrics

In an article entitled ‘Capitalism is ruining science’, Meagan Day (for www.jacobinmag.com) points out that universities existed before capitalism and pursued not profit but truth and knowledge

But no longer

The modern university has become increasingly subservient to the imperatives of capitalism i.e. competition, profit maximisation and increasing labour productivity

In academia, this manifests itself as ‘publish or perish, funding or famine’

Without public investment, universities are compelled to play by private sector rules i.e. to operate like businesses, focus on their bottom line and constantly evaluate their inputs and outputs

Hence, according to researchers Marc Edwards and Siddhartha Roy, they have introduced new performance metrics which govern almost everything researchers do, including:

  • Publication counts

  • Citations

  • Journal impact factors

  • Total research dollars

  • Total patents

These metrics now dominate decision-making in faculty hiring, promotion and tenure, and awards – so academic scientists are increasingly driven to get their research published and cited – scientific output as measured by cited work has doubled every nine years since WW2, they say

But quantity does not equate to quality:

  • Rewards for publication volumes have resulted in scientific papers becoming shorter and less comprehensive, ‘boasting poor methods and an increase in false discovery rates’

  • The growing emphasis on work citations has resulted in reference lists becoming bloated to meet career needs due to peer reviewers requesting their own work be cited as a condition of publication

Meanwhile, because increased grant funding also includes more professional opportunities, scientists spend an outsize amount of time writing grant proposals and overselling the positive results of research to catch the attention of funders – and lose opportunities for careful contemplation and deep exploration, which are vital if they are to uncover complex truths

Sadly, the combination of perverse incentives and decreased funding increases pressures which can lead to unethical behaviour – and if a critical mass of scientists become untrustworthy, a tipping point may be reached where scientists are thought to be corrupt, and public trust is lost

Peter Higgs, the British theoretical physicist who, in 1964, predicted the existence of the Higgs boson particle, said he would never have been able to make his breakthrough in the current academic environment:

  • “It’s difficult to imagine how I would ever have enough peace and quiet in the present climate to do what I did in 1964

  • Today, I wouldn’t get an academic job – it’s as simple as that – I don’t think I would be regarded as productive enough

  • I became an embarrassment to the physics department at Edinburgh University when they did research assessment exercises – they would send around a message saying ‘please give us a list of your recent publications’

  • I would send back a statement – ‘None’

  • I was kept around, despite this, solely in the hope that I would win the Nobel Prize which would be a boon to the university

The noble purpose of any science academy is to provide the resources and encouragement for people to carry out rigorous experiments that will enhance collective knowledge about the world we live in

At present, those aspirations suffer when (US) austerity-minded administrations stem the tide of federal funding and institutions change their business models to suit

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.