’I was a Skeptic, but…’

By Darcy Cowan 24/04/2011

I wish people wouldn’t say this, it’s usually followed by some lame reason why we should trust their anecdotal experience over empirical data. Sure the word skeptic (or sceptic if you prefer) has a certain colloquial definition and to a large extent words are defined by the way they are used, I mean no-one uses the word “gay” to refer to being happy anymore.

Even so this usage is getting on my nerves. When I use the word “skeptic” to refer to myself I mean someone who evaluates the available evidence and comes to a reasonable conclusion. Implicit in my definition is also  an understanding of human foibles with regard to cognitive biases and a deep seated inability to view our own experiences impartially. Refer to my previous post for more in this vein.

On a whim I thought I’d look up what on-line dictionaries had to say about the word, I found some variation of the following to be popular:

1. One who instinctively or habitually doubts, questions, or disagrees with assertions or generally accepted conclusions.”

That doesn’t seem any better to me. So, what’s my problem?

Well for a start those alluded to in the title of this post are not applying skepticism they are merely doubtful. And when evaluating claims they are not using the methods of science they are using the unreliable guide that is personal experience. Thus, while their protestations of skepticism and subsequent conversion sound impressive, they are (to my ears) merely the hollow echo of true inquiry.

Harsh enough for you? well perhaps. I don’t expect all who use the word skeptic to apply to it the same definition that I do, but it still chafes.

The dictionary definition given above is also lacking in nuance, it appears more suited to define a contrarian than skeptic. What my favourite skeptical interviewer DJ Grothe refers to as “knee-jerk skepticism”. A skeptic isn’t someone who just says “no”, a skeptic is someone who asks “how do we know?”.

The reason my hypothetical skeptical convert gets on my wick so much is when answering the “how do we know?” question they assume that they can draw general conclusions from their informal experiment where n=1. This ties into the “don’t knock it ’til you’ve tried it” line of argument. NO. Trying it myself is not the way to determine the validity of a claim. This falls under the category of anecdote, and anecdotes are not good quality evidence. At best they should be the start of investigation – not the end.

When evaluating a claim we should look at two things in particular, yes we should determine the direct evidence for the claim i.e. is there evidence to show that it acts as claimed? but we should also attempt to see how the specific claim fits into the wider scientific ecosystem – the prior probability if you will.

Often in day to day claims this is of little practical importance and so it becomes overlooked when it is relevant. A new gadget or medication is often based on previous iterations of the same technology or medical practice and represents an incremental improvement or merely an additional option in the sphere or possibilities. However some claims are sufficiently far from mainstream understanding that we should take a step back and consider the likelihood that the claim is possible, irrespective of the evidence presented for the claim itself.

In the case of say, homeopathy or power balance bands our current understanding of the science should make us extremely wary of efficacy even before the specific claims are considered. To be clear here though, plausibility should be used as only part of the process, there are many things that work without us knowing how they work but the further outside of current knowledge something is the stronger the evidence we should require before we accept it. Certainly for many “alternative” therapies that strong evidence simply does not exist, as I presented for Amber teething beads there is no reason to think they should work from a physical or medical point of view so our standard of evidence should be higher than the earnest assurances of people in mothering forums, or even our own experience – as noted above.

But this is exactly the sort of pseudo-evidence that we are wired to find most convincing. Throughout most of our history the ability to evaluate randomised trials, statistics and p-values would not have aided our survival one whit. Therefore it’s not surprising that most of us are bad at it.*

Yes, it’s hard. Yes, it requires work, and yes you will probably get it wrong most of the time.** But it’s worth it. So give it a try – be skeptical, like you mean it.


* Arguably all of us, it requires practice and even the “experts” can get it wrong.

* I certainly do.

Enhanced by Zemanta

Filed under: Sciblogs, skepticism Tagged: Medicine, sceptic, Science, Science in Society, Scientific method, Skeptical Inquiry, skepticism

0 Responses to “’I was a Skeptic, but…’”

  • Nice one,

    The thing that really really gets to me about knee-jerk skepticism of skepticism (meta-skepticism?) is the claim one should be open minded. Almost always followed up with because I saw a ghost/this guy I know was cured by homeopathy/I surf way better when I wear this bangle, so I know it’s true.

    Being open minded means not jumping to the first conclusion that congeals in your brain after hearing some spooky story, but thinking a little about how else that evidence might have come about – something the skeptic-skeptics don’t often do

  • Gees, I hate the phrase “I was a skeptic, but” with a vengeance as you can guarantee it will be followed by something innane.

  • Exactly right david, generally it’s just a big argument from ignorance or appeal to their own authority. I don’t know what it was therefore it was a ghost and I can’t be fooled so I’m right. or something.

    the whole “open minded” thing gets me going too. in popular parlance seems to mean accepting any rubbish so long as it goes against what scientists might say.

  • I forget where the quote originated from

    “it’s good to be open minded but not so open that it allows any old rubbish to fall in.

  • Terry Pratchett’s spin is good too:

    “The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it.”

  • The thing that really annoys me is when people consistently use run-on sentences in a work they plan for other people to read. I was interested in this and probably would have actually read the whole post if the run-on sentences weren’t so painful. When you learn how to proof-read, I’ll try again.

  • If you open your mind to much your brain might fall out, by Tim Minchin. I don’t know if it was originally his quote or not though.

    I’ve been hit with the “you’re being close-minded” thing before, last time was in a conversation where I was asking for some explanation of an osteopath being able to deduce someone’s emotional state over the previous month based on the texture of their liver. Drives me batty. I notice though, that most of those who use it, don’t actually realise that it works equally well against whatever argument they are currently making. On the basis of whatever personal experience they had, there is usually a refusal to accept the possibility that they might be wrong. Which drives me batty as well.

  • Tim Minchin is awesome.

    Don’t you know that being open minded only works if you are simultaneously rejecting science?

  • The problem I have with the “you’re so closed-minded” comment is that anyone can use it. It’s the exact same as Pascal’s Wager. You can apply it to any belief you wish, no matter how cracked it is. You can claim anything you want, say, that sticking chocolate ice-cream under your armpits will cause you to live 20 years longer, and if someone says no, you can use this “oh you’re so closed minded” retort, or variations on the theme like “there are more things on heaven and earth” etc. It’s cheap, it’s facile and it tries to move the burden of proof over to the skeptic.

    Great article by the way. Plausibility on its own is not enough. We should always seek the best possible evidence.

  • I think you’ve got it about right, it always seems to be up to the skeptic to prove that something isn’t so rather than on the claimant to prove that it is.

    • Yes, it’s funny I seldom see real sceptics declare that they are sceptics over and over.

      Anyway, I found this funny:
      “Spirit chasers needed specialist equipment, including digital cameras, night-capable video cameras, voice recorders and a K2-meter which picked up electrical currents – and will go fully prepared to Pukekohe.”
      “So we try to connect [with the paranormal] through that, and ask them questions, like why they are there? If [owners] are scared or whatever, the medium can ask if they want their house cleansed and she’ll have gear with her and go around the house … with sage and water and bless the house.”

      1. Do they REALLY need that equipment? Possibly to convince themselves and their “clients” they are doing real sciency stuff.
      2. Sounds like they don’t need to investigate; they’ve already decided what the answer is.