Validity in experiment

By Marcus Wilson 20/11/2012

Last Friday I was at the Waikato Science Teachers’ one day conference in Cambridge. There was a wide range of different material talked about, which made for an interesting day. One of the questions which was tackled (led by Simon Taylor)  was ‘When is an experiment valid?’ Or, what is ‘validity’ all about when it comes to science. Some thoughts from the audience were ‘When it does what it is supposed to’, ‘When it agrees with theory’, ‘When it is controlled’, ‘When it is repeatable’, ‘When it measures what it is supposed to’.

All of these I think are reasonable responses, depending upon the situation. f you are illustrating a physical principle to a class then you certainly want your experiment to behave – we all know that physics experiments are too often characterized by the fact that they don’t work – especially when they have a big audience.

But an experiment can never ‘not work’. It does what it does. The fact that it didn’t do what you wanted or expected could be for a variety of reasons – bad experimental design, poor control, statistical variation, or maybe because of some ‘new’ phenomenon. After all, where would science be if every experiment agreed with current theory? Major strides forward in science have usually been triggered through experiments that didn’t do what the experimenters were expecting.

I think a good definition of ‘valid’ would be ‘that the experiment measures what you think it is measuring.’ That, from memory, was basically Simon’s point. If we achieve that, then it doesn’t matter whether we find a new phenomenon, validate an existing theory, or just make a mundane measurement of electric current in the lab. We’ve done some good experimental physics.

Sorry – short entry this one – baby wants attention.


0 Responses to “Validity in experiment”

  • I disagree that and experiment can’t “not work”, perhaps this fits under your definition of experimental design but if I run a chemistry experiment and use the wrong reagent the experiment won’t work. I may or may not learn something from this (assuming I figure out that the wrong thing was used).

  • “I disagree that an experiment can’t not work.” Have to think carefully about the triple negative in this one. Well, yes, I suppose if you make a mistake in the experiment then it doesn’t do what you wanted it to do. But then that’s also a case of “does it measure what you intend it to measure?” isn’t it? With my physics students, I often struggle to convince them to write down every result in their lab notebook – even those they think are suspicious. Often there’s a lot of experimenter-bias in what gets recorded and what never gets written down, which is not good practice.

    Plus, though it’s a long time ago now, I am still irked by my secondary school biology teacher who told me that my woodlice results were wrong. I put six woodlice into a choice-chamber – half was dark, half was light. Five of them (I remember this well) sat in the light half, one in the dark half. I’m sorry, Mr Smith (a pseudonym used to protect the guilty), that’s what they did. I’m sorry they hadn’t read the text books, or your lesson plan. I’m happy to accept that the method might have been poor, badly controlled, etc, but telling me that my results were wrong only encouraged me to drop biology at the earliest opportunity.

    So, to summarize. I would maintain that if something goes wrong, the problem is in the method, not the results.

  • Yes Grant, – that’s what I meant. Data are data. They might mean nothing, but they are the data.