Cherry-picking and ring-fencing the scientific literature

By Ken Perrott 17/03/2014 12


studytracker-ba

Don’t believe the advertising hype.

Anti-fluoride activists are promoting a “Study Tracker” which they see as a powerful weapon in their fight against fluoride. It is simply a search facility for their own database of articles – really Paul Connett’s database at Fluoride Alert.

So they can now search Connett’s database – big deal.

But why would anyone use this “study tracker?”  You certainly wouldn’t if you were at all interested in finding out what the current status of knowledge is with a particular area of fluoride research. You would use Google Scholar or one of the more specialised databases fo scientific papers. For example here are the results of a few searches I did using both the “Study Tracker” and the more easily accessible and useful Google Scholar.

Dental fluorosis prevalence

First – lets look at the prevalence of dental fluorosis in humans. The “Study Tracker” does not allow you to use your own search words so I just used theirs – “dental fluorosis,” prevalence, and “human study” (adding the word fluoride) for Google Scholar as well. (That inability to use one’s own search terms is already  handicapping a user).

Results – 16 references from the “Study Tracker” and 4440 references from Google Scholar (click on the images to go to the actual search results.

Dental fluorosis prevalence

16 references

Fluoride-Action-Network---S

4440 references

dental-fluorosis--fluoride-

Arterial calcification and heart disease

I wrote about the dishonest use of a research paper on this topic in Fluoride and heart disease – another myth. So using the words allowed by Connett’s “Study Tracker” – heart, arterial calcification, human study – I found 9 references using the “Study Tracker” and 5850 references using Google Scholar. (Again, click on the images to see the details).

Heart arterial calcification

references

Fluoride-Action-Network---S

5850 references

dental-fluorosis--fluoride-

So why use Connett’s “Study Tracker” when it produces such poor results and Google Scholar is more accessible? Here’s a couple of reasons I can think of:

  1. The cherry-picked database provided by Connett and his family. Activists make a lot of the 1500 references Connett has collected in his database – but that is infinitesimally small compared with the total available literature resulting from fluoride research. But if you were only interested in confirming your own bias, finding papers that you can use to support an anti-fluoride narrative, this “Study Tracker” removes the necessity to do your own cherry-picking.
  2. Unfamiliarity with the concept of literature searching. If you are not used to searching the scientific literature the “Study Tracker” may be the only method you have heard about.
  3. Desire to impress. Again, many activists want to give a “sciency” veneer to the material they produce. How better than to actually quote a scientific reference with journal details and a link?  In my experience most people with this motive don’t bother reading the paper to check that it says what they want it too. Most probably don’t even read the abstract! How else could organisations like the Fluoride Action Network of NZ quote a paper describing a proposed clinical procedure for determining risk of heart disease using a radioactive F isotope as “evidence” for fluoridation causing heart disease (see  Fluoride and heart disease – another myth)!
  4. Some people are happier wearing blinkers. After all, if your whole intention is confirmation bias (and anyway papers providing evidence that doesn’t support that bias are written by “industry shills) why should you even allow your eyes to see anything else? Much safer, and certainly less stressful, to stick with a cherry-picked database than look at all the literature available.

I think cherry-picking and confirmation bias a problems for all of us when searching the scientific literature. These are human failings. But this shouldn’t be encouraged.

The research accessible in published literature is immense and complicated. It should be approached intelligently and critically. One can’t do that using blinkers like the “Study Tracker”.

Unfortunately political activists are very often highly motivated in their use of scientific literature – so it is easy to see why this “Study Tracker” will appeal to most anti-fluoridation activists. But this approach is in direct conflict with the scientific ethos which at least encourages processes aimed at an objective approach.

Franky I see this “Study Tracker” as just another tool encouraging an opportunist use of scientific knowledge, encouraging confirmation bias and dishonesty in using this knowledge, and encouraging people to restrict their sources when looking for scientific support for their claims.

It is a form of ghettoisation of scientific knowledge. Users will only be encouraging their own ignorance, rather than knowledge.

Similar articles


12 Responses to “Cherry-picking and ring-fencing the scientific literature”

  • I sent a letter to the local Mayor and Council. This comment was included:

    “An important part of my teaching is encouraging postgraduate students to critically use the scientific literature. I particularly warn my students to beware of the biases of activist organisations such as FAN. Professional and academic sources are, as a general rule, stronger and more reliable.

    The thing is, when campaigners such as these use the scientific literature, they do so selectively, looking for the tiniest note that supports their position. The question their research seeks to answer is, “What can I find, that can be used (twisted) to help our case?” In contrast, when the health authorities use the literature, they look at all the information. Their question is, “On the balance of all of the information, what is the best for our communities?” Additionally the professionals have the knowledge and skills to make a judgement, amateurs do not! The council’s responsibility is to follow the guidance of the Ministry of Health.”

    Keep up the good work.

    Thanks

  • Maurice says [quote]Additionally the professionals have the knowledge and skills to make a judgement, amateurs do not![unquote]

    Possibly unintentionally, this is a horrendous slide from ‘campaigners’ to ‘amateurs’. Campaigners are people with a strong opinion who try to change other people’s thinking. An amateur is a person who engages in an activity without payment. In days gone past, many of the most talented sportspeople were amateurs. Volunteer firefighters are amateurs, etc. Please don’t try to load terms with negative connotations!

  • I beg to differ Stephen and I support Maurice’s interpretation.

    Without payment is only a part of the definition – otherwise doctors, lawyer and other professionals would have to significantly redefine themselves when doing pro-bono work.

    I’d venture to suggest that a number of the well qualified pro-flouride advocates are working pro-bono as well, and would not appreciate being downgraded from professionals to amateurs on the basis that they are unpaid for their efforts.

    So an unpaid volunteer is not, by definition, an amateur. The lead criteria for amateurism is that the activity is a “pass-time” rather than a professional vocation, or the participant lacks the capacity to be defined as a professional. Both these criteria fit well with Maurice’s use.

    See the link for a more succinct definition.

    http://www.thefreedictionary.com/Amatuer

  • My point is that even the dictionary definition of ‘amateur’ has become conflated with negative connotations, when it really should only mean unpaid. The flipside is that “professional” is often wrongly loaded with positive connotations, implying high levels of competence, when it really just means paid. It does indicate a MINIMAL degree of competence, which COULD be missing from an “amateur”, since total incompetence would presumably lead to cessation of payment by employer/client. In my area, I often see professionals only competent to this minimal level, whereas some amateurs care about what they are doing and do far better work.

  • The point being that definitions should be kept pure, or else you can abuse them by, for example, implying that someone who does something without pay must therefore be incompetent. Such is the stuff of rhetoric.

  • The distinction between qualified and unqualified may be more along the lines that Maurice intended. This is not the same distinction as between professional and amateur. Certainly an unqualified doctor would be too much of a risk to take seriously. One the other hand, in other areas of science, some relatively unqualified people are more talented and competent than some realtively highly qualified people. Life is never so simple. Qualifications are a function of ambition as much as anything else, but ambition isn’t everyone’s first priority in life. At the end of the day, judge people on their merits, and not on thier qualifications or employment status.

  • In academia, ‘amateur’ usually refers to someone without formal training in a subject, i.e. someone who is self-taught.

    A few amateurs quite good. Usually that is very rare because of the amount of work needed to get to a comparable level to those who have formally training and research experience.

    • So how do we define Paul Connett – is he a professional or an amateur? (Some claim he is the world expert on fluoridation!!).

      He has PhD and worked in an academic college. However, he is retired, no longer employed academically. He is not being paid by any scientific insitute, but then again he is being paid (as are other members of his family – his wife and son) by a political activist organisation.

      His academic qualifications do not relate to fluoride or fluoridation as he has not done any original research in this area and has no real scientific papers in it (He does have 2 letters to the editor and participated in a review paper).

      Quite a few retired people do get involved in political issues on retirement – sort of a hobby and a way of retaining status. They may rely on academic titles to obtain or give the impression of credibility. But their views may be of no more credibility than your butcher or milkman on the specific topic.

      Are the just amateurs who are dishonestly playing on former jobs and qualifications to impress gullible supporters?

  • Yes, I agree Grant (you do need a coffee! Just kidding!)

    Trained and qualified are basically synonymous

  • Judging purely from your comments above, he would be classified as a retired qualified professional in one area, who is now an unqualified professional in another area. It is highly relevant that he is being paid by a political rather than a scientific organisation. However, none of these facts relate directly to the credibility of his public views on fluoridation. They are consistent with his views being anything from spot on to utter crap. That will have to be judged on its merits.

  • Stephen –

    “Trained and qualified are basically synonymous”

    Just for clarity, I wrote my comment before you one that refers to this (my writing crossed over your comment, as it were; the order on the page on blogs can mislead in that way!)