how not to do science: the scole experiment

By Alison Campbell 23/03/2010

I listen to quite a lot of podcasts. Lately I’ve been listening to more than usual. I’ve had the flu (I’m assuming that’s what it was, since colds tend not to come with fever, chills, & sore joints) & listening to stuff was easier than reading. Anyway, I digress.

One of my current favourite podcasts is Brian Dunning’s – excellent primers in critical thinking, nicely presented, & not too long. One of these concerned the ‘Scole experiment’ – supposedly an excercise in which scientists tested the claims of mediums (people claiming to communicate with the spirit world) and  – gasp! – found the claims justified. I’ve been interested in claims about the paranormal ever since reading (& re-reading, multiple times) Martin Gardner’s book Science: good, bad & bogus.

The Scole experiment – named for the village in England where it was carried out – was a series of seances led by 6 mediums and investigated by 15 members of the Society for Psychical Research. Involving large numbers of investigators & psychics, It supposedly provided evidence of the existence of ‘spirits’: lights moving about in a darkened, closed room; photographic images appearing on film that had previously been sealed into secure containers, physical contact with invisible entities; tables lifting off the ground, & voices coming out of nowhere. Nor, it’s claimed, was there any evidence of fraud.

So, evidence that there is an afterlife, & us sceptical sorts should start to revise our worldview? Not so fast, says Dunning.  When you come to look closely, the Scole experiment turns out to be a very good example of how not to do a scientific investigation.

I know that if I was going to undertake an investigation of psychic claims, about the first thing on my list would be to put in place various controls & restrictions, thus minimising the opportunity for any possibility of fraud. Things like cameras, motion sensors, venues thoroughly checked beforehand. Amazingly, Dunning tells us that this was not what happened in the Scole study. Here, the mediums set all the rules, thus effectively ruling out the possibility of this being a serious scientific examination of the proceedings. In fact, it appears that the investigators did everything that the mediums asked of them – Dunning describes them as acting as an audience, rather than researchers. Thus:

  • the psychics were effectively free to move around during each seance (no hand-holding), and thus the investigators weren’t able to exclude the possibility of the phenomena they witnessed being generated by the performers themselves.
  • they banned any use of still or video cameras – & this included infra-red & night-vision equipment. (One has to wonder why this was the case – if the seance was genuine, this technology would surely be no threat..)
  • the box into which unexposed films were locked was supplied, not by the investigators, but by the psychics. What’s more, one of the investigators wrote that he was able to easily open the box in the dark… Films placed in boxes supplied by the ‘researchers’ never developed any images, a fairly suggestive finding.
  • the seances were carried out in a room provided by the mediums, not the researchers.
  • and – despite the fact that the Scole performances have been hailed as proof positive of an afterlife – there’s been no follow-up at all.

This complete lack of any serious controls and experimental protocols means that we can’t take the study’s findings seriously. And it does bring to mind a couple of comments from Gardner’s book. One has to do with the apparent hypersensitivity of psychic phenomena to any sort of critical examination. The other was Gardner’s statement (I think originally made by prestidigitator & debunker James Randi) that scientists are perhaps the easiest audience to fool  if you’re a magician or psychic – because they don’t expect anyone to be setting out to deliberately pull the wool over their eyes…

I do enjoy my regular Skeptoid fixes 🙂

0 Responses to “how not to do science: the scole experiment”

  • Your thought is right but none has been able to prove it as an absolute fake. Every thing in the universe can not explain with scientific theories like some of the traditional herbal healing techniques are very effective but their theoritical facts are hoex in scientific terms.

    • Please provide citations in support of the statement that “some of the traditional herbal healing techniques are very effective”. The recent Nobel prize in medicine is not evidence in support as a huge amount of scientific research went into screening 2000 different ‘TCM’ materials to find one that contained the relevant active ingredient, followed by yet more testing to show that it works.

      Also, I don’t think that one can have ‘theoretical facts’ in science.

  • I would not accept the results of any scientific study of paranormal phenomena unless it was carried out by physicists. The claim being made, after all, is that physics is broken, and that there are forces commonly at work in the universe other than those recognised by our current physics. Obviously physicists are the appropriate people to investigate such claims.

    • Well, yes & no. There’ve been studies done of supposed paranormal claims where the scientists doing the studying have been comprehensively fooled – Uri Geller would be an example. Martin Gardner commented in his book “Science: good, bad, & bogus” that scientists are relatively easy to fool because they aren’t expecting someone to try to bamboozle them.