Cherry-picking data is an old technique used by those who wish to raise doubt about a scientific consensus. On one hand it isolates the cherry-picked data from their context and the rest of the data. On the other hand it presents a “sciency” authority to the argument by pretending to be evidence-based.
I have written about cherry-picking in several articles discussing the fluoridation issue. But the current surge of activity by anti-fluoridation activists attempting to raise doubt with the upcoming parliamentary discussion of new legislation on fluoridation is producing a fresh wave of cherry-picked arguments.
One example is this letter to the Christchurch Press. I thought it worthwhile to check out the quoted figures to see if the arguments were justified. The figures were taken from the Ministry of Health’s (MoH) 2014 data for the dental health of New Zealand children. Unfortunately, while the actual numbers used are correct, the data has since been withdrawn because of errors in the spreadsheet. So I will use the data for earlier years, 2005 – 2013, in my analysis.
The overall picture
The overall picture shown by the Ministry of Health data is that community water fluoridation does reduce tooth decay. Of course, that is why the anti-fluoride campaigner rarely discusses the overall picture – instead, they cherry-pick data to confirm their bias.
In the figure above I have separated the data by ethnicity because of the big differences between Māori and Pacifica and other ethnic groups. In particular, the dental health of Māori and Pacifica children is poorer. This is an important factor which needs to be taken into account when comparing data from different regions. I discussed this further in my article, ‘Anti-fluoridation cherry-pickers at it again.’
Data for Canterbury
It is likely that at least some of the 2014 spreadsheet mistakes were in the Canterbury data – but still the claim that there is no real difference between data for fluoridated and non-fluoridated areas could well be true, at least for some years. The figure below displays the data for 5-year-old children. Choose your year and you will get the answer you want to confirm your bias. Children from fluoridated areas seem to have poorer teeth in 2008 and 2010 and better teeth in 2012 and 2013.
The plots in the above figure indicate how unreliable such comparisons are for Canterbury because the fluoridated data is all over the place. This is because of the very low number of children in the fluoridated area: 22 – 70 over the years, 42 on average. There were on average 4720 children in the non-fluoridated areas. Children from the fluoridated area usually comprised less than 1% of the total.
The data for Canterbury does not deny the effectiveness of fluoridation, as the letter writer claims. They just show that no conclusion can be drawn from this cherry-picked data. At least I cherry-picked the data from 2005-2013 which enabled me to see how unreliable they were. The letter writer just cherry-picked one year! What will they do if the corrected spreadsheet for 2014 no longer supports their bias – switch to 2010 instead?
Comparing Canterbury and Waikato data
Here we have a different problem. The letter writer has simply cherry-picked these figures because they confirm her bias. She has not taken into account the important influence that ethnic composition has. Any intelligent analysis of this comparison must consider this aspect.
The table below is the ethnic composition of the 5-year-olds from MoH data (averaged over 2005-2013):
So, whereas only 13.4% of Canterbury 5-year-olds are Māori or Pacifica, 33.6% of Waikato 5-year-olds are Māori or Pacifica. This is an important difference – especially as the dental health of Māori and Pacifica is poorer than others as demonstrated in the first figure.
Any analysis that does not take this difference into account will be misleading.
As well as ethnic distribution between regions there is also the influence of ethnic distribution between the fluoridated and non-fluoridated areas. This was a factor I discussed in my article, ‘Anti-fluoridation cherry-pickers at it again’.
What this means is that the mean value for fluoridated Waikato areas is decreased by the higher presence of Māori and Pacifica than in the non-fluoridated Waikato areas. This higher proportion of Māori and Pacific in the Waikato region also affects the comparison of the two regions made by the letter writer.
Rather than comparing oranges with apples, let’s compare Canterbury and Waikato for the same ethnic group – Others (not including Māori and Pacifica). As the figure below shows, removal of the effect of Māori and Pacifica from the Canterbury data increase the caries-free percentage – but it is still slightly less than the equivalent data for the fluoridated Waikato areas.
So much for children from non-fluoridated Canterbury areas having better teeth than children from fluoridated Waikato areas.
Auckland and Counties/Manakau
Some anti-fluoride campaigners are pulling the same trick – asserting the dental health of non-fluoridated Canterbury children is better than for the fluoridated Auckland and Counties/Manakau children.
Below is a table showing a comparison of the ethnic composition of the three regions for the 2013 5-year-old MoH data:
|% Māori + Pacifica|
See the problem? It is just completely naive – or worse, dishonest – to compare data between regions like this without taking ethnic composition into account.
But that is not going to stop determined activists who will just cherry-pick whatever fits their bias. I think the naive presentation of data in this way is no more justified by the declaration “Do the math” than misrepresentation of the science is justified by the declaration “Do the research!”
Note: I am well aware that the MoH data have other problems. A truly scientific analysis would also take into account factors like the degree of misallocation of children due to different fluoridation status of home and school, dental treatments such as fluoride varnishes differently used in different regions, missing data, different proportion of attendance according to region and ethnicity, etc. I am not the person to make such a thorough analysis. My sole purpose here is to show how such raw data can be misused for confirmation bias and “sciency” support of mistaken political agendas.
Featured image: CC flickr