By Ken Perrott 27/04/2017

One thing that gets me going (as readers here probably know) is the way scientific citations are cherry-picked and dragged in to support biased positions.

It’s a common form of confirmation bias in the fluoridation debate. And I get even angrier when the perpetrators cherry-picking citations will then claim they “have science on their side.”

Parroting citations and (often unconnected) claims are not what science is about!

So I was very pleased to see this article Why Citing a Scientific Study Does Not Finish An Argument by Jonny Anomaly and Brian Boutwell at Quillette. They point out that throwing down a gauntlet like “actually, studies show . . . “ often kills the discussion:

“It’s hard to know what to say when people cite scientific studies to prove their point. Sometimes we know the study and its relative merits. But most of the time we just don’t know enough to confirm or refute the statement that the study is supposed to support. We are floating in a sea of information, and all we can do is flounder around for the nearest buoy to support a view that’s vaguely related to the conversation.”

I think this is why anti-fluoride propagandists rarely get challenged when they come out with their misinformed claims that fluoridation causes IQ loss, ADHD, hypothyroidism, etc. Their discussion partners are often not familiar enough with the scientific literature to challenge the claims. Of course 99 times out of 100 the propagandist is also completely unfamiliar with the literature and is simply parroting a claim they saw in one of their “nature news” newsletters, or similar. And surely throwing out citations one has never read is a clear example of the arrogance of ignorance.

Credit: Johannes Jansson / Wikimmeida.

The fact is: “All of us lack the time to understand more than a small fraction of scientific research.” But when the discussion partner is familiar with the cited studies the propagandist quickly lurches into a Gish Gallop – or deletes the online discussion and bans the person. Those of us who have entered into this debate with the motivation of clarifying the science will know what I am talking about.

We should be wary of arguments relying on citations even in cases where the proponent has read the literature. Citing an individual study is really meaningless:

“Of course, that’s not always how science works, or how knowledge is spread. A single study is rarely anything more than suggestive, and often it takes many replications under a variety of circumstances to provide strong justification for a conclusion. And yet, poorly supported studies often make their way into newspapers and conversations as if they are iron clad truths.”

That’s another thing that angers me – even fairly reputable magazines will report individual studies as if we should take the results as “gospel truth,” without even considering the quality of the research – let alone any supporting research.

The problem of correlations

Often such poorly supported studies rely on correlations – and the way commenters and the media cite such correlations as “evidence” is another bugbear of mine. The authors rip into this problem:

“Correlations are everywhere, and given enough data from enough studies, we will find correlations that are surprising and interesting. But . . .  causation is difficult to infer, and some correlations are flukes that don’t admit of a common cause, or that can’t be consistently replicated.

“We are pattern-seeking creatures, and correlations are patterns that cry out for explanation. But sometimes our political views infect our prior beliefs, and these beliefs lead us to look for patterns until we find them. Given enough tests and time, we will find them.”

I am amazed at how studies relying on the poor use of correlations often make it into scientific journals. I have written about one example in ADHD linked to elevation not fluoridation.” There is a similar situation for the recent paper of Hirzy et al. (2016) which I discuss in Anti-fluoride authors indulge in data manipulation and statistical porkies. Although I understand that particular paper was rejected by several scientific journals before it ended up in Fluoride which accepts anything that is anti-fluoridation.

Frankly, I think more papers like this should be challenged and that journals have an ethical responsibility to publish critiques of such papers. Unfortunately, I think I am being a bit idealistic here as many editors have their own biases.

When it comes down to it I think even with the scientific literature it is a case of reader beware. One should never take citations at face value – especially when used to confirm a biased argument. Rather than accepting such arguments we should follow them up, read the cited paper – and other papers in the research area of we have time. We should approach all such claims using citations critically and sensibly.

This is in line with the conclusion the authors make to their article:

“We’re not advising you to commit social suicide by interrupting every conversation with a demand for more evidence. But we do think the phrase “studies show…” should be met with cautious skepticism, especially when the study supports the politically-motivated preconceptions of the person who’s talking.”


Feature image credit: Why Citing a Scientific Study Does Not Finish An Argument.

0 Responses to “Citing scientific studies and the arrogance of ignorance”

  • Ken – please swap disciplines and bring your rational mindset to the discourse of water quality in NZ. There are so many parallels with the Fluoride debate; mis-citations of studies (lacking important contextual information that should have prevented their wide extrapolation) and over-reliance upon correlation rather than causation by advocate “scientists” (indirect relationships, mirroring effects of some other environmental driver that is not in turn addressed by so-called solutions like reducing nitrogen, nitrogen, nitrogen).

  • Good work Ken. One such example of misinformation is Broadbent 2014. There was no control group in that particular study yet it is used by Gluckman and Skegg to say that there is no affect on IQ from water fluoridation.

    Thank you for providing a platform to expose Broadbent and his coauthors.

    • Pleased you approve of my article, Kane. However, your comment does suggest you are confused or misinformed and are committing some of the mistakes I referred to in my article as endemic amongst anti-fluoride activists.

      You refer to “Gluckman and Skegg” as if this is a citation. But there is no such article.

      I believe you are really referring to Eason, C., & Elwood, JM. Seymour, Thomson, WM. Wilson, N. Prendergast, K. (2014). Health effects of water fluoridation: A review of the scientific evidence.

      Please, note – neither Gluckman or Skegg coauthored this report. It is only referred to in the way you have (“Gluckman and Skeg”) by anti-fluoride propagandists in their articles condemning it. This suggests to me that you have not read the actual report but rely completely on the FAN and Fluoride Free NZ propaganda related to it. My reference to “surely throwing out citations one has never read is a clear example of the arrogance of ignorance” is surely relevant here – and is a very common problem with anti-fluoride propagandists.

      Your reference to Broadbent at al. (2014) also indicates confusion – again probably because you have not read the actual paper and are relying on biased critiques of it from anti-fluoride propagandists. This paper used data from the Dunedin Multidisciplinary Health and Development Study – which was not a naive experiment set up with “controls.” That would be completely unethical. Because many variables have been measured in this study it is possible to tweak out the effects of different variables using sophisticated statistical analyses. This is what Broadbent et al. (2014) did.

      Yes, I know motivated crtics like Hirzy et al(2016) have claimed the differences in fluoride intake were too small to detect an effect of fluoride on IQ – but they are completely silent about the fact these differences were large enough to detect differences in oral health due to fluoride.

      Yes, of course, the sample size is a limitation of Broadbent et al (2014) – as it is in all the studies relied on by Connett in his campaign aginst community water fluoridation. Researchers continually face this problem and that is why they cite confidence intervals in the resulting data. I referred to this in an early version of the paper critiquing Hirzy et al. (2016) submitted to the journal Fluoride:

      “Aggeborn & Öhman (2016) made that criticism of Broadbent et al., (2016) and all other fluoride-IQ studies. This Aggeborn & Öhman(2016) study had a much larger sample size than any of the other studies – over 81,000 observations compared with around 1000 or less for the commonly cited studies. It was also made on continually varying fluoride concentrations using the natural fluoride levels in Swedish drinking waters, rather than the less effective approach of simply comparing two villages or fluoridated and unfluoridated regions. The confidence intervals were much smaller than those of Broadbent et al., and other cited studies. This makes their conclusion that there was no effect of fluoride on cognitive measurements much more definitive. Incidentally, their study also indicated no effect of fluoride on the diagnosis of ADHD or muscular and skeleton diseases.”

      The study of Barberio (2016) also had a large sample size – over 2,500 observations. This reported no statistically significant relationship of cognitive deficits to water fluoride.

      The Aggeborn & Öhman (2016) and Barberio (2016) studies will be hard to beat and anti-fluoride propagandists have so far remained silent on them. (Incidentally, as you would expect Fluoride will not accept my critique of Hirzy et al. It is the first paper of mine to be rejected {in a long research career] and I am quite proud Fluoride did reject it – considering the poor quality and bias of that journal. However, it is very likely to be published elsewhere so, in the end, such censorship does not work).

      You can read my critique of Hirzy et al at Researchgate – CRITIQUE OF A RISK ANALYSIS AIMED AT ESTABLISHING A SAFE DAILY DOSE OF FLUORIDE FOR CHILDREN and I will upload a final version when it is published in a different, more respectable, journal.

      Finally, Kane, as you are the person continually challenging others to “debate” the fluoride issue let me offer you the possibility of a real online discussion of the fluoride-IQ subject that seems to concern you – or any other similar subject. I suggest a good faith discussion of the sort I had with Paul Connett in 2013/2014. There have been several developments in the research and the critiques since then so another in-depth, good faith, scientific discussion would be timely.

      What do you say?

      • Ken, just quickly. Was your critique published in a peer reviewed journal? If not I’m sorry but I’ll have to disregard.

        • Kane, the publication status of my critique was explained in my comment.

          Now about my suggested online discussion – are you up for it?

          Come on – don’t be shy.

  • Ken, there is a very good chance that your “paper” will not be accepted as peer reviewed standard. I’m sorry but I will have to disregard you “research…”

    All the best with your hobby.

    • So Kane – Now about my suggested online discussion – are you up for it?

      Why do you keep running away from this question? – a simple yes or no is all that is required.

      Mind you – if you aren’t up for it what about one of your mates – Stan Litras, Lynn HJordan (Penelope Paisley), Mary Byrne, Mark Atkins?

      Or what about Bill Hirzy or Paul Connett?

      Come on – you are pals with all of these – you could organise something?

  • Ken, your work is not peer reviewed. I’m sorry but everyone heard on science day that “if it’s not peer reviewed then it’s not worth anything”.

    All the best with your hobby.

    • So, Kane, is the fact you have no peer-reviewed work the reason you and your mates in FAN and FFNZ are unwilling to participate in a good faith online discussion.”

    • You are actually wrong, Kane – I think I have four peer-reviewed papers on fluoride. You could check on google scholar,5

      But I do not see your lack of peer reviewed papers on fluoride as a reason not to have a discussion with you – or any of the others I listed. After all, we will be discussing others’ work, not our own. It is simply a matter of critically and intelligently considering the published science.

      So are you up for it? Or if not, what about getting one of your mates to take up the offer?

      Good faith scientific discussion of fluoride and IQ would be very interesting at the moment.

  • Prof Connett has been there and done that Ken. You’ve got very little integrity so I think we will leave this here.

    All the best.

    • And so Kane backs away.

      No surprise there.

      But none of his mates, including Paul Connett, are prepared for such a good faith online discussion of the science.

      We can draw the obvious conclusion.

  • Well that’s hardly surprising, is it? Another humiliating exit from Kane.

    He must be increasingly frustrated that Connett is no longer willing debate with Ken, especially now that he finds himself scurrying off when afforded the same opportunity.

    When I consider Kane’s longstanding opposition to fluoridation (undoubtedly inherited from his mother) it reminds me of the work of University of Adelaide researcher Dr Jason Armfield. He refers to the extremist arguments that find fertile ground among disenfranchised, psychologically disturbed, and alienated individuals. Armfield suggests that a subculture has now developed around and for such people who are believed to find psychological gratification in imagining themselves heroically in the possession of such secret and ‘subversive’ information.

    Kane certainly strikes me as one of these people. Someone who may or may not be conflicted (as opposed to say Ainsley Fitzgibbon or Pete Evens) Kane is quintessential ‘true believer’, someone that works tirelessly for what he believes to be good and true – someone that sees himself waging a heroic war – against an ever encroaching authority.

    It’s a darn pity that Kane’s war is little more than a deluded witchhunt reminiscent of 50’s era McCarthyism. For counted among his following are a seemingly disenfranchised lot; gun toting anti government libertarians and hysterical mothers spraying bottles filled with vinegar into the air.

    If there were ever a suitable exhibit for why we need more science communication in our high schools it would Kane Titchener and the anti – fluoride movement.