With the publication of the new Cochrane Fluoridation Review (Water fluoridation for the prevention of dental caries) we have, once again, both fluoridation supporters and opponents claiming it as evidence for their contradictory positions. They surely both cannot be right.
The reality is that any such review is going to have its own complexities and limitations which allow committed believers on either side to confirm their biases. Unfortunately, this confirmation bias is promoted by inadequate reporting relying on “sound bites” from the executive summary. Real understanding of the review and its results requires more thoughtful analysis, a reading of the full review and not just media reports and a bit of thinking about its limitations.
The post is the first of three articles looking a bit more deeply at the Cochrane fluoridation review. Here I discuss the strict criteria used by the review team for selecting the studies they considered, the limitations this has caused for their findings and the misinterpretation of the review results this has produced.
97% of fluoridation research ignored
This seems amazing – why ignore so much of the research? We can understand the need to filter poor research or poorly reported claims. But 97%?
Yet, that is what the review reports – and summarises in their Figure 1:
The high exclusion rate was caused by the review teams decision to only consider studies which conformed to strict criteria:
“For caries data, we included only prospective studies with a concurrent control, comparing at least two populations, one receiving fluoridated water and the other non-fluoridated water, with at least two points in time evaluated. Groups had to be comparable in terms of fluoridated water at baseline. For studies assessing the initiation of water fluoridation the groups had to be from nonfluoridated areas at baseline, with one group subsequently having fluoride added to the water. For studies assessing the cessation of water fluoridation, groups had to be from fluoridated areas at baseline, with one group subsequently having fluoride removed from the water.
For the purposes of this review, water with a fluoride concentration of 0.4 parts per million (ppm) or less (arbitrary cut-off defined a priori) was classified as non-fluoridated.”
This criteria requiring measurements at several time periods and the inclusion of data from before the commencement of fluoridation was probably the main reason for excluding studies. In the review’s Table “Characteristics of excluded studies” the most often mentioned reason was “Evaluated caries in a single time point cross-sectional study.”
OK, you can sort of see the logic behind these strict criteria:
“The cross-sectional studies, whilst able to provide information on whether water fluoridation is associated with a reduction in disparities, are not able to address the question of whether water fluoridation results in a reduction in disparities in caries levels.”
But this inevitably resulted on consideration of only a small part of the available research:
“155 studies (162 publications) met the inclusion criteria for the review. However, only 107 studies (15 caries studies; 92 studies reporting data on either all fluorosis severities or fluorosis of aesthetic concern) presented sufficient data for inclusion in the quantitative syntheses.”
Inability to comment does not mean no effect
Exclusion of so many important studies meant the review was unable to come to any conclusions about important aspects like the effect of community water fluoridation (CWF) on socioeconomic difference in tooth decay, the effect of stopping CWF programmes on later tooth decay and the effectiveness of CWF in reducing adult tooth decay. Yet, in the review’s discussion they did make note of research which did draw some conclusions in these areas – research they refused to consider. (And it is rather ironic that one of the review’s authors, Helen V. Worthington, has co-authored several papers which conclude that CWF does reduce socioeconomic differences in dental health).
Of course, anti-fluoride propagandists have chosen to misrepresent the review – reporting its inability to draw conclusion on these questions as evidence that CWF does not influence socioeconomic differences, is not effective for adults and tooth decay does not increase when CWF is stopped! (See Misrepresentation of the new Cochrane fluoridation review). Clear misrepresentation – but helped by the combination of exclusion of most research and the vague language used in the review summary.
And you do sort of wonder at ignoring so much evidence when considering issues related to community health. Was Cochrane throwing away the baby with the bathwater?
Confirmation fluoridation is effective
The strict exclusion criteria enabled the review team to winnow studies down to a small number which could be analysed quantitatively. They were able to confirm from analysis of 9 studies that CWF:
“resulted in a 35% reduction in decayed, missing or filled baby teeth, and 26% reduction in decayed, missing and filled permanent teeth.”
But the strict exclusion criteria, specifically rejection of cross-sectional studies, is still a fly in the ointment. Recent studies of situations where fluoridation has been in operation for a long time did not fall within the strict selection criteria because pre-fluoridation data would not realistically be available in most cases. The review consequently did not consider properly the recent evidence – 71% of the research considered occurred before 1975!
The review, therefore, raised the issue of how applicable their findings are to the current situation in developed countries because of improved dental care and use of fluoridated toothpaste. A reasonable proviso which could have been discussed properly using the research they had excluded. But again a proviso which enables misrepresentation by anti-fluoride propagandists who imply that their findings are irrelevant to our current situation.
The review authors acknowledge that exclusion of such data presents a problem for their conclusions:
“In the past 20 years, the majority of research evaluating the effectiveness of water fluoridation for the prevention of dental caries has been undertaken using cross-sectional studies with concurrent control, with improved statistical handling of confounding factors (Rugg-Gunn 2012). We acknowledge that there may be concerns regarding the exclusion of these studies from the current review. A previous review of these cross-sectional studies has shown a smaller measured effect in studies post-1990 than was seen in earlier studies, although the effect remains significant. It is suggested that this reduction in size of effect may be due to the diffusion effect (Rugg-Gunn 2012); this is likely to only occur in areas where a high proportion of the population already receive fluoridated water.”
Of course, the review team was correct to raise the question of the possible reduced efficacy of CWF in modern developed societies. But doesn’t that suggest they should not have used such restrictive criteria in selecting studies to consider? And isn’t it irresponsible to leave the impression that CWF is no longer effective when they excluded the studies which could have provided better answers?
The Cochrane fluoridation review suffers from the fact that only 3% of available studies were considered. The restrictive selection criteria enable quantitative estimates showing CWF is effective for children but excluded the possibility of answering questions related to the effectiveness for adults, the ability of CWF to reduce socioeconomic differences in oral health, the effect of stopping fluoridation on later tooth decay and whether improved availability of dental treatments and use of fluoridated toothpaste has reduced the efficacy of CWF in modern developed societies.
The language of the review report itself encourages misinterpretation – and this is even worse in their blog post about the review – “Little contemporary evidence to evaluate effectiveness of fluoride in the water.” Here they repeatedly refer to lack of evidence but only explain this is due to their exclusion of such evidence in a few places. What is the uninformed reader, who does not bother to read the full document, make of points in the summary such as:
- “There is insufficient evidence to determine the effect of water fluoridation on disparities in caries levels across socio-economic status
- There is insufficient evidence to determine the effect of water fluoridation on caries levels in adults
- There is insufficient evidence to determine the effect of removing water fluoridation programmes from areas where they already exist”
Finally, anti-fluoride propagandists are motivated enough to misrepresent the findings in any fluoridation review or other documents. The very restricted selection criteria used by the Cochrane review and the language of its summary and news reporting of the review is a bit of a godsend to such propagandists.
Expect to see a lot of cherry-picked quotes from the Cochrane review. Twisted to turn the lack of evidence of effects (due to the exclusion of studies) into evidence for no effect.
My next article on the Cochrane review will deal with its discussion of “bias” and poor quality in the studies it considered.