By Siouxsie Wiles 12/01/2018 3


Shocking revelations around a clinical trial of a new tuberculosis vaccine are just the tip of the iceberg. Maintaining public trust in science depends on open science.

Ten years ago, Dr Ben Goldacre published Bad Science, a book described by The Economist as “a fine lesson in how to skewer the enemies of reason and the peddlers of cant and half-truths”. In his book, Goldacre slams the bull-shitters who misuse science, taking aim at detoxing and ‘brain gyms’, as well as ‘magical water’ homeopaths and pill-pushing fake PhD-holding nutritionists. Scientists like me shouted with glee.

Goldacre followed that with Bad Pharma in 2012, subtitled ‘How medicine is broken, and how we can fix it’. The book revealed how the pharmaceutical industry manipulates drug trials, buries data it doesn’t like, and misleads regulators, doctors, and patients. This time, scientists like me signed petitions and joined Goldacre’s call for all medical trials to be registered so we’d know if drug companies were trying to pull any dodgy shit by stopping their trials early, or never publishing their findings.

For the past few years, I’ve been waiting for the last book in the trilogy, Bad Academy – How science is broken, and how we can fix it’*, exposing the rot within academia and calling on scientists like me to get our house in order. You might have heard whispers of this rot, sometimes referred to as the reproducibility crisis.  In a nutshell, several studies have found that many findings from many fields of science cannot be reproduced. There are lots of reasons why this is the case, ranging from researchers falsifying data, or misusing statistical tests, to researchers carrying out experiments badly or using reagents that weren’t what the researchers thought they were (1).

As far an I know, Goldacre hasn’t written Bad Academy*, not yet anyway, but this week prestigious medical journal the BMJ published an explosive investigation by its associate editor Dr Deborah Cohen (2). It’s shocking, gut-wrenching stuff. It relates to an experimental vaccine, MVA85A, which was recently tested in almost 3,000 infants in South Africa, to see if it could protect the babies from the infectious lung disease tuberculosis (TB). Unfortunately, MVA85A failed despite being shown to be safe and effective in studies in mice, guinea pigs, rabbits and macaque monkeys.

Or was it? Cohen has uncovered evidence that the researchers cherry-picked data from the animal studies to get further funding and approval to trial the vaccine in humans, effectively sweeping any data that showed the vaccine didn’t work under the proverbial rug. One of those studies apparently showed that when MVA85A was given to macaques along with the current vaccine (known as BCG), five of the six animals had to be euthanised because they became so ill with TB. Shocked? And if you are wondering which institution the researchers pulling this shit are from, it’s the University of Oxford, ranked number one in the Times Higher Education World University Rankings.

I really can’t do the BMJ investigation justice. Please, go and read the piece for yourself. It’s a tale involving a whistleblower who paid a steep price for raising his concerns, internal investigations that cleared the researchers involved of any wrongdoing, and the perverse incentives that allow bad and fraudulent behaviour to flourish in academia.

Over the next few days you may hear a whole bunch of esteemed academics talking about how complicated it is to do vaccine experiments in animals, and how animal’s immune systems are not the same as humans, so we shouldn’t be surprised when vaccines that work in animals don’t work in humans. I imagine they are really worried about the damage the BMJ revelations will do to people’s confidence in vaccination. I know I am. Mumps cases in New Zealand are the highest they’ve been in decades, and some people still believe the dangerous bullshit disgraced former gastroenterologist Andrew Wakefield spouts about the MMR vaccine and autism. And those academics are right, it is hard to do animal experiments, and there are differences between laboratory animals and humans. But that’s not the point. This is a story about hiding research data. We need to be talking about that.

You might also hear the case referred to as an isolated incident. It’s not. It’s the tip of the iceberg. At every academic institution I’ve ever worked at, I’ve seen varying examples of the kinds of stuff that have led to the reproducibility crisis, as well as shit I would call actual research misconduct. I once even tried complaining officially, but after a two-year investigation, which I am not allowed to talk about, the person involved got off. I notice they’ve since been promoted to Professor.

Enough is enough. It’s time for academics to stand up, own this shit, and make real efforts to fix it. Because there are ways to fix it. Aside from making everyone do conjoined science and humanities degrees, properly training all research scientists in ethics, experimental design and analysis, and removing the perverse incentives that allow bad science and misconduct to thrive, what we also need is transparency. Just like Goldacre’s calls for all medical trials to be registered, similar moves are needed for animal trials. Hell, they are needed for all types of experiments. A growing number of scientists are reporting their methods and data online and in real time, rather than only publishing their most exciting results behind a paywall in some academic journal. It’s called open science, and you can find out more about it in the video below. Unfortunately, open science is nowhere near being the accepted way to carry out scientific research. This has to change. Now. Maintaining public trust in science depends on it.

*Just to be clear, there is no actual trilogy, and I just made up that title….

References:
1. Ioannidis JPA (2005) Why Most Published Research Findings Are False. PLoS Med 2(8): e124. https://doi.org/10.1371/journal.pmed.0020124
2. Cohen D (2018). Oxford vaccine study highlights pick and mix approach to preclinical researchBMJ, 360 doi: https://doi.org/10.1136/bmj.j5845


3 Responses to “It’s time for academics to stand up against bad science”

  • Tuberculosis is a terrible disease; BCG is the only vaccine and it’s not very good. Therefore I can understand why researchers would be keen to find some improvement. Had they been successful we wouldn’t be having this discussion, yet there’s nothing new about results from BCG trials being ambiguous.

    Ben Goldacre runs the evidence-based medicine data lab at Oxford. Maybe he will comment. Maybe my hair will go grey while I’m waiting.

    https://www.rt.com/uk/415593-oxford-vaccine-tuberculosis-babies/
    11 January 2018
    … Professor Beverley had made allegations regarding the trial against vaccine developer Professor Helen McShane. She, however, was cleared of any academic misconduct, with the investigating panel going “so far as to add that on the basis of the vaccine’s proven safety in humans and positive phase 1 and phase 2A trials, it would have been unethical not to have proceeded with the phase 2 trials in infants.”

    University Registrar Professor Ewan McKendrick said: “The time has come to stop the repeated repackaging of criticisms and allegations which independent expert analysis has demonstrated to be without foundation.”

    Professor Mike Turner of the Wellcome Trust, which funded the study, said clinical trials were carried out “to the highest standard.”

    “The decision to test this candidate vaccine was correct and based on robust, positive data from smaller trials in humans that showed that the candidate vaccine was safe and that it might be effective,” he added, according to the Daily Mail.

    “Human trials do not always generate the same results as animal testing, which is why results in animal models are typically only one of a set of considerations in determining whether to move research forward.”

  • Maybe you could write the third book of the trilogy and let Ben Goldacre write the fourth?

    This is a reminder that “we’re making progress” and “success before the heat death of the universe” are not the same thing. For all the things are getting better in the world, we still have systematic errors being made, and blatantly obvious solutions not being applied. “one death at a time” is not fast enough.

  • Excellent article Siouxsie – it gave me goosebumps. I recently tried to access a journal article where the research had been funded by NZ government agencies but could not get past the abstract. Why are scientists still choosing to use a closed access journal? I am off to visit PLOS.