Fact or fallacy, a survey of immunisation statements in the print media

By Grant Jacobs 19/01/2011 24

Most of us know anecdotally that print media on occasion present immunisation information incorrectly, but you can’t put a finger on how often and when without hard numbers.

A recent research article examining New Zealand newspapers puts numbers to the errors.

Helen Petousis-Harris led a team surveying the immunisation statements in articles printed in four national New Zealand newspapers over 6 years (2002-2007).

They find that the proportion of unsubstantiated statements about the meningococcal immunisation programme increased with increased public interest and fell again as interest waned.

Overall they attribute this to a reliance on ‘he said, she said’ journalism, with the fallacies this brings, writing (pages 525-6)

[…] For new vaccines, prior to introduction, the coverage is less emotive and more scientifically substantiated, focusing on the potential of the vaccine to prevent disease. […]

Following the routine use of a vaccine the proportion of unsubstantiated reporting increases, including fallacious arguments. These arguments are used by both supportive and anti-immunisation spokespeople, although more commonly by those opposed to immunisation. Most arguments proposed by anti-immunisation lobbyists have been demonstrated to be fallacious.4,20-22 Apparent here is the failure of the press at times to verify both the material and the credibility of their sources. […]

and conclude

Most media ‘balance’ given to immunisation relies on ‘he said, she said’ arguments using quotes from opposing spokespersons with a failure to verify the scientific validity of both the material and the source.

It’s tempting to think that what is happening is that as public interest increases some journalists and editors increasingly ’present the other side’–which is not a necessarily appropriate thing to do–rather more thoroughly assess and critique the arguments presented by those interviewed.


A problem with ‘he said, she said’ is that in matters of fact it is not people’s opinions that matter, but the strength of the evidence backing the statement that matters.

Considerable space in the article is devoted to discussing errors of fact and logic, concisely introducing the errors examined.

I feel it’s a pity that this article is not open-access. The discussion deserves to be more widely read by journalists, editors and the wider sphere of science writers. (Including science bloggers.)

In addition to introducing the types of errors, examples are given, with sources cited. These ‘worked examples’ would be an excellent aid to help those writing the articles gain a ‘feel’ for what to be alert to.

From the 360 articles with vaccine-related information, overall they observed that 21% (76 articles) contained ’some factually unsubstantiated information about vaccine efficacy or safety.’

Articles about new vaccines, or vaccines not in routine use, were observed to be accurate, rare cases excepted. By contrast, one-third of articles about the MMR vaccine and one-quarter of articles about the New Zealand-specific meningococcal B vaccine contained erroneous information.*

Their survey is based on a database of articles derived from a commercial media watch service containing words associated with vaccines and vaccine-preventable disease.

They consider both errors of fact and errors of logic under a taxonomy of errors, which are identified, marked** and the number of lines of each category in the article counted. Errors in statements both favouring vaccination and against vaccination are considered as errors; what is being assessed is accuracy of the statement, not the viewpoint of the statement.

About the cartoon

Perhaps I shouldn’t pollute a serious article with a cartoon like that, but I like a little fun… This joke runs deeper than the surface: each of the twitter accounts in the cartoon can be found on twitter. (Yes, I had a hunch and just had to check for myself.) hanneloreEC’s next tweet after the one in the cartoon reads ‘I’ve got 500 surgical masks, twenty gallons of hand sanitizer, and a flare gun. As ready for swine flu as I’ll ever be.’ Now that’s being prepared.)

If you want to check these twitter accounts for yourself, enter http://twitter.com/#!/ followed by the twitter name, skeeve37, hanneloreec, etc.


* I don’t know if these differences are statistically significant.

** You might argue that they ought to have counted the number of statements, rather than the number of lines in scoring the articles. It seems to me that this would depend on your objective. If the idea is to assess how much ‘space’ is spend in different states (correct or fallacious in various ways) in the article then counting lines seems reasonable, assuming all lines are of the same length as they typically are in print newspaper articles. This would cover if the writer spend more space emphasising a particular aspect compared to others, something counting the number of statements might overlook.


Petousis-Harris, H., Goodyear-Smith, F., Kameshwar, K., & Turner, N. (2010). Fact or fallacy? Immunisation arguments in the New Zealand print media Australian and New Zealand Journal of Public Health, 34 (5), 521-526 DOI: 10.1111/j.1753-6405.2010.00601.x

Other articles in Code for life:

Media thought: Ask what is known, not the expert’s opinion

Sources for medical information for non-medics and non-scientists

Vaccine promotion — the medium matters too

Immunisation, then and now

Wakefield autism studies slammed as fraud by BMJ

Rubella, not a benign disease if experienced during early pregnancy

24 Responses to “Fact or fallacy, a survey of immunisation statements in the print media”

  • It’s just like xkcd to do that, eh?

    While I’m writing, I’m ruminating that I should have emboldened the paragraph immediately above the cartoon to highlight it’s message.

  • Well perhaps now is a golden opportunity for you email this summary and reference to all NZ newspapers and other news media practioners

  • John,

    What needs to be read—in my opinion!—is the discussion and examples of errors in the article itself, which my brief summary doesn’t really touch on at all.

    My summary really only touches on the overall improvement needed, not the individual errors needed to be recognised and challenged in order to make that improvement. It’d be similar to giving someone a goal without an outline of the means to achieve it.

    One simple thing to do would be to write a brief letter to the editor of each newspaper (i.e. to be published) encouraging them to read the research article, perhaps including a quote of an error from their paper as a teaser. (The research paper has a table of each type of error with examples from the newspapers examined. e.g. – the NZ Herald is cited as committing false attribution in how they cite “Risk analyst Ron Law”; the Dominion Post is cited as presenting an irrelevant conclusion in writing “She says that diseases such as scarlet fever have also declined hugely during the past 110 years, despite the absence of a vaccination program.” and so on.)

  • The paper claims “All articles from 2002 to 2007 were assessed for errors of fact and logic. Fact was defined as that which was
    supported by the most current evidence-based medical literature. Errors of logic were assessed using a classical taxonomy broadly based in Aristotle’s classifications.”

    In other words, the paper introduces a bias in its definition… it assumes that “evidence-based medical literature” de jour is fact when in fact much of it suffers from publication bias and massaged results. It is patently clear from the natural decline of meningococcal disease and deaths that was well in hand PRIOR to the rollout of the MeNZB vaccine that the vaccine could never have been the cause of the ongoing decline in the disease. A fact unpublished in the scientific literature is that in the years 2006-2009 the ONLY deaths in children who were entitled to be vaccinated with MeNZB were children who received the vaccine. NONE of the deaths were of children whose parents rejected the vaccine for whatever reason.

    Could someone please explain how a classical taxonomy
    broadly based in Aristotle’s classifications could have been used to conclude the comments made about me personally?

    The paper, in Table 3: Examples of errors of logic about vaccination appearing in the New Zealand print media 2002–2007, says, “False attribution “Risk analyst Ron Law, a critic of the vaccination program, said the program had been a failure.” Health reporter, NZ Herald, 2006
    The term “risk analyst” implies expertise on the subject when in fact there is none giving the statement more credibility.”

    I have been a practicing risk and policy analyst/adviser for over a decade… In 2000 the Ministry of Health appointed me to an expert working group to advise the DG of Health on the management and reporting of medical injury in the health system; the appointment was recognition of my risk analysis experience and skills.

    The claim that the decline of scarlet fever is largely due to
    the development of antibiotics is also false… they should have provided a reference to support their fallacy… much of the decline had occurred PRIOR to the introduction of antibiotics… here’s a link to one of many paper proving that.

    The paper is factually flawed.

  • The media in general are guilty of accepting anything an “expert” says without fact-checking it, and the Science Media Centre are as guilty as any of that. I refer you to my comments on the incorrect statement about cumulativeness of bisphenol-A that was made by Professor Shaw.

  • Rosalind, Professor Shaw may well be in error as you say or he may be right or his comments may have been misinterpreted – I don’t know. What I do know is that he is one of the foremost experts in this area in New Zealand and a seasoned science communicator which is why we regularly approach him for commentary. In this case I believe he was commenting on a piece of research, so that’s actually a layer of context that journalists receive that they wouldn’t otherwise. When it gets really valuable is when half a dozen or more scientists from across the country, or across the planet (there are also SMCs in the UK, Australia, Japan and Canada) weigh in with their views. They won’t always agree but then a journalist really gets a feel for what the scientific community makes of a new piece of research or a scientific development. That’s what we aim for at the SMC but sometimes, scientists aren’t (available or willing) to turn around comment in a time frame that suits the media…

    Ian Shaw I’m sure would be interested in hearing from you regarding his comments, you can find his details on the Canterbury University website – why not drop him a line?

    Peter Griffin, Science Media Centre manager

  • Rosalind, which part of Professor Shaw’s reported comments are you concerned about? Below is an article in the ODT last week…

    Then there is this…


    can you point out where he’s wrong in fact?

    Published on Otago Daily Times Online News (http://www.odt.co.nz)

    Plastic not so fantastic

    Questions about the safety of some plastic bottles are finding few answers in this country, Chris Barton, of the NZ Herald, reports.
    Prof Ian Shaw doesn’t use cling film to wrap his sandwiches.


    See: http://www.odt.co.nz/print/89155

    [ Ron – if the article is on-line, there’s no need to cut’n’paste the whole thing. It’s kinder on readers to just post the link. (Readers not interested in it won’t have to wade their way past it.) Besides, I prefer to keep on the better side of copyright.

    Chris Barton and Tracy Barnett’s original article that the Ron points to is derived from is here: http://www.nzherald.co.nz/health/news/article.cfm?c_id=204&objectid=10619136 ]

  • Ian Shaw’s comment:
    “Small doses [of bisphenol-A] over a long time can initiate female hormone responses.” is absolutely wrong in fact, and I have explained why at length in the comments on an earlier article on bisphenol-A on Sciblogs. I’m not going to go through it all again. I refer you back to that article.

    Ian Shaw has also published two papers espousing his theory that methyl bromide (MeBr) causes amyotrophic lateral sclerosis (ALS). MeBr has been used widely around the world, including very careless use causing multiple cases of mild clinical toxicosis in some countries such as Turkey, and there is absolutely no international evidence for his theory. The fact that a tiny number of Nelson MeBr workers developed ALS is meaningless because correlation does NOT prove causation. ALS occurs in clusters in many places where MeBr is not used. Early last year he announced to the media that he had just discovered that MeBr reacts with glutathione. In fact this has been known for decades and has been in standard toxicology textbooks for years. He went on to say that he had a PhD student working on whether MeBr reacts with erythrocytes, and said that the results would be “crucial” to his theory of its toxicology. Again, it has been known for decades that MeBr reacts with erythrocytes. Furthermore , whether or not a xenobiotic reacts with erythrocytes does not dictate whether or not it is toxic. Offhand, I could name a number of deadly poisons that react with erythrocytes and some substances of low toxicity that do. A PhD is supposed to be original research, but finding that MeBr reacts with erythrocytes is not original research.

    As a Diplomate of the American Board of Toxicology (the premier qualification in toxicology) myself, who undertook her postdoctoral residency in toxicology with a distinguished toxicologist who is a former president of the Society of Toxicology, I often read Ian Shaws’s opinions next to those of Australian toxicologists in SMC’s “Experts Respond” and cringe with embarrassment, on behalf of New Zealand, at how poorly his commentaries compare to theirs. To a toxicologist such as myself, they come across as knowledgeable and he does not.

  • Rosalind, can I assume you are referring to this blog?

    One of the fascinating thing with science is that even when one discusses issues with two experts such as Diplomates of the American Board of Toxicology (the premier qualification in toxicology) there are often two or more opinions expressed.

    Being an expert does not provide a mortgage on fact, fact or opinion.

    I am aware that even water with a short half life due to evaporation in the framing of many modern houses causes long-term damage… not because of accumulation, but simply because damp is maintained sufficiently for fungi etc to thrive. Longterm use of some hormones can turn mechanisms on/off longterm…

    Professor Shaw has obviously come across as knowledgeable to someone of higher standing… After all he’s been promoted to a professorship by someone.

    Professor Shaw’s profile on the

    I have a BSc (Hons) in Biochemistry from the University of Bath and a PhD in Biochemistry (Toxicology) from the University of Birmingham. My jobs have included the first lectureship in Toxicology in the UK at University College London where I worked on the toxicity of anticancer drugs. Then I moved to Boehringer Ingelheim (a pharmaceuticals company) to continue my anticancer drug work, but in a more clinical setting, then to the UK’s Central Veterinary Laboratory as Head of Toxicology Section where I was responsible for food analysis and research on toxic chemicals in food. Following this I was Chair in Toxicology at the University of Central Lancashire (UCLan) where I was also Head of the School of Biological Sciences and Head of the Centre for Toxicology. My work at UCLan focussed on pesticides, their measurement in food and their effects on consumers. During this time I chaired the UK government’s Pesticide Residues Committee. I moved to New Zealand in 2000 to take up leadership of the Food Safety Group at the Institute of Environmental Science & Research (ESR). From there I moved to the University of Canterbury as Pro Vice-Chancellor (Science) and in February 2009 reverted to the academic life as Professor of Toxicology in the Department of Chemistry. My research interests remain the mechanisms and impact of environmental chemicals on human health, but with a particular interest in chemicals that mimic hormones (endocrine disrupting chemicals – EDCs). I am also passionate about science communication and have made TV programmes (e.g. Is it safe to eat? TVNZ, 2008), been on radio many times and write regularly for newspapers and magazines.

    Are you saying he’s unqualified to be viewed as an expert????

  • Rosalind, here is a quote from someone I presume has views similar to yours… I wonder if he missed maths 101, or if he was miss quoted. My point is that no one should accept uncritically what they read.

    Dr Warren Foster, Director and Professor of CIHR/Ontario Women’s Health Council and Director of the Reproductive Biology Division in the Department of Obstetrics & Gynecology at McMaster University, Ontario, said:

    “BPA is a very weak estrogen relative to the body’s own estrogen called estradiol. Indeed some standard tests for estrogenicity fail to detect any estrogenic action of this chemical. It is estimated to be approximately 1/10,000th the potency of estradiol or in simple terms if one person makes 10,000 dollars/year someone else making 1/10,000th as much would receive 10.00 dollars/year.

    At present there is no credible evidence to suggest that contemporary exposures are linked with increased risk of breast cancer.

    “Relative to other chemicals to which people are exposed such as the phthalates, BPA presents minimal health concern.”


  • I’m saying that on the basis of Prof Shaw’s comments on various issues in SMC’s “Experts Respond”, and his publications on MeBr and ALS, he comes across as credulous and lacking in critical acumen, and liable to be reach values-based rather than science-based conclusions. That may not be how he really thinks, but that’s how he comes across. In a couple of cases in which papers were reviewed by multiple experts in “Experts Respond”, the other ‘experts’ identified important limitations and caveats about research findings which Prof Shaw completely missed. As a result, he enthusiastically embraced conclusions that other ‘experts’ had strong, very well-grounded reservations about accepting. When I see an individual repeatedly miss important details that everyone else spots, I start to wonder why. I wonder about someone who claims to be a toxicologist yet never addresses the all-critical question of dose. I wonder about someone who doesn’t seem to know that correlation does not prove causation, and can’t seem to research the international evidence (or lack thereof) before claiming it does. I wonder about someone who announces a “new discovery” to the media when the finding has been in textbooks since the seventies, since doing a good literature search is such a basic research skill. It doesn’t add up to a good impression.
    As for what is safe to eat, I’d rely on John Reeve of NZFSA if I were you. He is a good toxicologist.

  • (Shameless plug alert)

    Journalists are inherently untrustworthy. Should anyone be interested in my thoughts on that they can be found here:





    [Ed – Yes, that is quite a shameless plug. I’ll let you have it – just.]

Site Meter