It’s probably familiar to many of us … you are debating some issue when someone with an opposing comes out with “but I read a study that showed … ” and then goes on to explain how the so called study justifies their position. You then track down the study (assuming they can tell you where to find it) and it does nothing of the sort.
This has happened to me twice in the past week.
First, someone trying to justify homeopathy by attacking conventional medicine comes out with the suggestion that a study has shown that during a doctor’s strike in New York, mortality dropped, which demonstrates that without doctors prescribing their “poisons” less people died.
Tracking down the paper I quickly discovered
1) Five doctor’s strikes had been studied
2) None were in New York
3) Only two out of five showed a decrease in mortality
4) The authors suggested that this drop in mortality was most likely due to no elective surgeries taking place during the strike (i.e. so no relation to the prescription of drugs)
Then in another debate about same sex marriage I had someone tell me of a study they “remembered” that showed that same sex marriages only lasted one and a half years on average, with eight partners during that time (not sure if that is supposed to be each, or a combined score).
I did a bit of reading on line, as the person pushing this idea couldn’t remember the source, and discovered it is a major distortion of research into HIV transmission. A good summary can be found here, however key points include
1) The study was carried out before gay marriage was legal in the Netherlands
2) Monogamous gay men were excluded from the study
3) Did not involve lesbians
This is why I am so wary when people use the statement “But a study has shown …”
If you do want to use a study to support whatever position you are arguing could I please suggest the following:
1) Make sure the study actually exists (and ideally you should be able to provide any interested person with a reference to it)
2) Read it and make sure it does support your position
3) Make sure it is from a reputable source (reputable scientific journals – yes, magazines – no)
4) Make sure you understand the difference between causation and correlation
5) Make sure that there aren’t ten other reputable studies refuting your position (if this is the case then some care has to be taken to work out why there is disagreement, and which source(s) are likely to be more reliable)
6) If you can’t fulfill steps 1 to 5 above then don’t use it with me in a debate, especially if, like so many people, you can’t even get past step 1!