You can change the ideas, but not the data

By Grant Jacobs 24/03/2010

Not too long ago I attended Lawrence Krauss speaking in Dunedin. Others here have written reviews of Lawrence Krauss’ talks in Dunedin (David Winter at The Atavism) and Auckland (Fabiana Kubke at Building Blogs of Science).

prof-lawrence-krauss-200pxAfter the talk, having not much else to do, I sat for for some time watching Lawrence respond to younger audience members who’d gathered around him. Many (most?) of these seemed to be religious people who choose not to speak up during the questions (too timid, perhaps?), but who wanted to argue with him afterwards.

He put a lot of effort into a good-humoured responses to these people and gave them a considerable amount of his time.

I was unable to follow the dialogue, as those asking questions had their backs to me, but one thing Lawrence said I thought was worth repeating here, something not said in his talk, words to the effect that:

You can change the ideas, but not the data

Often when I read of people trying to dismiss some science they proceed by trying to dismiss the data just as one of those talking to Lawrence Krauss must have, to evoke his response. I recall Lawrence Krauss following this by admonishing cheerfully, wagging his finger in the air,  ’the data are the data.’

Since then I have read a related comment made by Daedalus2u to discussion following an article, Plausibility in Science-Based Medicine, by Stephen Novella on Science-Based Medicine. Let me quote Daedalus2u’s comment in full (see ‘10 Mar 2010 at 6:41 pm’ in the comments to the article):

YSWE, I think you have the idea of prior plausibility not quite right. An idea does not have low prior plausibility if it does not agree with prior explanations, it has low prior plausibility if it does not agree with prior data.

Many (most?) scientists make this confusion too. That is because they are thinking on the level of the explanations, not on the level of the data that led to those explanations. The explanations may be wrong, the data that led to them is not.

The idea of using antibiotics to treat ulcers was incompatible with the idea that ulcers were due to too much acid. It was not incompatible with any of the data surrounding ulcer treatment.

Correct ideas are never incompatible with the data. If a new explanation is compatible with more data than an old explanation is, then we should adopt the new explanation.

Dr Novella’s invocation of the ideas of conservation of mass and thermodynamics to put a low prior probability on homeopathy represents the degree of testing to which those ideas have been subjected to and the levels of certainty with which they are held. If conservation of mass and thermodynamics were wrong (and they would have to be for homeopathy to be correct), then we would have to abandon just about everything science has done since 1900 and start over. Abandon millions of person-years of scientific effort so that we can imagine homeopathy is correct? Based on badly done clinical trials?

All of the great paradigm shifts in science did not contradict the prior data. A heliocentric solar system didn’t contradict any astronomical observations. Relativity didn’t contradict any Newtonian based data, quantum mechanics didn’t contradict any classical physics data. Relativity contradicted the idea of the luminiferous aether and the idea of absolute reference frames in space, it did not contradict any data.

Harriet Hall in a following post, Diagnosis, Therapy and Evidence, that takes it’s lead from this comment has condensing it to:

The data are always right, but the explanations may be wrong. The idea of treating ulcers with antibiotics was not incompatible with any of the data about ulcers; it was only incompatible with the idea that ulcers were caused by too much acid. Even scientists tend to think on the level of the explanations rather than on the level of the data that led to those explanations.

You can’t dismiss data. (Unless it was fabricated, of course.)

You can try find another idea that fits (all of) the data, showing that it does.

You can dismiss ideas that can be shown not to fit the data.


My hearing isn’t the best (I have a hearing loss) and my quotes of Lawrence Krauss are recalled from memory. The essence of them, however, will be correct.

Other articles on Code for life:

Upcoming popular lectures by Professor Lawrence Krauss

Prince or pauper? Tell Nature what you earn

International teamwork: Anthony Doesburg on collaborations

Molecular biology in museums

137 years of Popular Science back issues, free

0 Responses to “You can change the ideas, but not the data”