"a blog is not the place for scientific reviews"

By Grant Jacobs 16/08/2012 15


writes Prof. Christopher Shaw, currently under fire by blogger ‘Orac’ at Respectful Insolence. (See comment at August 15, 11:50 pm.)

Leaving aside the backstory, which is a separate issue that I don’t wish to debate here, why should science not be reviewed on blogs?

We’ve heard this one before – the NASA ‘arsenic life’ story being one well-known example.

Dr. Shaw wrote:

I’m not trying to duck any serious discussion, but, honestly, a blog is not the place for scientific reviews.

The key bit of my reply there was (see Footnotes for full copy) :

Heard that one before, I’m afraid, and for me it doesn’t ring true.

I’m a scientist. A blog is just a means of communication – just like email. That it’s open doesn’t make what is said “wrong”.

At the time of the ‘arsenic life’ fuss, I wrote:

I agree in the end it will be the formal articles that discuss that issue that will stand for the record, but you can’t realistically ask that scientists and science bloggers not explore the issues in public forums.

There is nothing new going on in scientists writing on blogs, compared to writing on bionet (or its ilk). There might be something new in journalists [or the general public] tapping into this writing, perhaps-?

(Text in square brackets added to original.)

Perhaps this message needs to be repeated every now and then – ?

To be fair to Dr. Shaw the example he choses introduces a new element:

One reason for this is simple: if I provide this blog with the complete ms (still in preparation), then it is not publishable in a main stream journal as it would be considered prior publication.

This, too, has parallels of sorts with the arsenic life story – the results, in ‘lite’ form, are being heard  by the public prior to formal publication and review, in this case via a coroner’s inquest and media reporting of that inquest. The inquest provides a different setting than usual, but it seems to me that in describing the results in a forum open to reporting by the media the underlying work must also be open to criticism just as it would be expected to be in other settings.

As a parallel thought, currently there have been many discussions about biologists submitting papers to arXiv – a preprint server mainly used by the physics and mathematics research community. Most biology journals, at this point in time, consider prior publication elsewhere to deny publication in a journal as Dr. Shaw says. Some biologists are using arXiv, however, and there is a push by some for it to be more widely accepted.

What are your opinions on reviewing science on blogs? Mine is clear, but feel free to differ (but offer reasons if you could).

Should science to be presented to the public settings like these should also be concurrently presented at some public source such as arXiv if it is not already in a journal?

Footnotes

1. While writing this, I spotted this comment by ‘rablib’ at the foot of an Atlantic article on the same general issue:

You’ve overlooked the fact that the world wide web (www.), hyper-text transfer protocol (http) and hyper-text markup language (html) were all invented and written at CERN with the goal of allowing scientists in varied locations  and using varied computing platforms to collaborate on research.  Science is not coming late to the party — it is the original host.

2. My full reply to Dr. Shaw at Respectful Insolence is copied below:

“a blog is not the place for scientific reviews”

Heard that one before, I’m afraid, and for me it doesn’t ring true.

I’m a scientist. A blog is just a means of communication – just like email. That it’s open doesn’t make what is said “wrong”.

You might like to look at your UBC colleague Prof. Rosie Redfield’s blog by way of example:http://rrresearch.fieldofscience.com/ (I’d encourage you to try talking to her.)

Personally, I would encourage you to continue. By limiting yourself to email I feel you’ll be making yourself part of the problem rather it’s solution.

A problem (for us in New Zealand too) is that media reports have tied your and Lee’s testimonies to the vaccine (something you claim you didn’t do) – not necessarily through explicit statements but (also) implied through headlines or simply not explicitly saying otherwise, etc., as is often the case with the media.

People have been trying to get an accurate picture by what means they can. If you limit yourself to email, people can only return to trying to get an accurate picture by what means they can – and the situation you are objecting to. In the meantime there will be no reasonable way to present an accurate picture to people.

(One frustration I have expressed elsewhere is the lack of transcripts of the inquiry, which would have been very helpful to clarify—or correct as the case may be—media reports.)

You write that you don’t want to talk because a manuscript is “still in preparation” – forgive me for asking, and I realise there are deeper issues this being involved in an inquest, but do you think it appropriate to present work that has yet to be tested by peers, as such, at an inquest? As a (tentative) parallel, would you consider what is known as “publication by press release” sound? (There someone talks to media ahead of the paper being available for critical examination.)

A issue to all this is once an idea is placed into the general public’s mind they can be very hard to remove and can cause considerable problems.


Other articles at Code for life:

NZ Skeptics Conference 2012 

Media thought: Ask what is known, not the expert’s opinion 

When the abstract or conclusions aren’t accurate or enough

2013 Public Communicators of Science and Technology Symposium 

Of use of the active voice by scientists


15 Responses to “"a blog is not the place for scientific reviews"”

  • Ah, I remember the ‘Arsenic life’ story. It gave me happy memories of reading Dorothy Sayers’ “Strong Poison”, in which an individual carefully acquires a high tolerance for arsenic, so as to have an apparent alibi when he poisons his victim (he is eventually caught because Peter Wimsey, like any well-educated person, is familiar with “A Shropshire Lad”).

    Another example would be the ‘ERV – XMRV’ saga, in which the blogger ERV caught out a research group who had built their reputations on the claim that a particular human retrovirus was the cause of chronic fatigue system… until they used the same Western Blot image in too many papers and presentations, with different labels.

    A relevant issue here is that some journals do not publish replication studies — even when the attempt to replicate fails, thus refuting an earlier publication. I’m thinking of Bem’s ESP claims, published in the J Pers Soc Psych — the journals effectively shut people out from criticising the original study or presenting negative data, so they *had* to resort to blogs to discuss their critiques.

  • “It gave me happy memories of reading Dorothy Sayers’ “Strong Poison”, in which an individual carefully acquires a high tolerance for arsenic”

    You might also like The Poisoner’s Handbook The author writes a blog, too, Speakeasy Science.

    I’ve written a little on the XMRV saga (as you might know?) – a few of the advocates wandered over my way. The anti-vaccine crowd are tame compared to some in that lot! (I actually read someone relate how one researcher in the field vowed to never work on XMRV again owing to how some of the advocates treated him.) I know the figure you’re talking about; same figure, different legends and interpretations… While there is an element of “science by press release” in the XMRV story, I think that story is a bit different in many respects – but there’s no two ways that blogs were a strong part of it’s “outing” as you were saying.

    Another example of the science by press release sort of thing was an announcement of a fossil discovery in South Africa — I’ve forgotten the details but I’m sure Alison can fill you in! — widely announced in the media before the paper had be published. Blogs jumped on that too; I’ve no idea if the criticism made it to the mainstream media.

    Replication bothers me – it’s quite an issue in some area of genetics / genome analysis, part of my own field.

    Interesting to learn that they were forced to use blogs to discuss the work. Must check that out. (If I find time…) It reminds me a little of the original situation before formal scientific journals started in some respects, with scientists forming their own interest groups and communications networks.

  • Grant, I like to compare certain aspects of science blogging to conferences. It’s a forum for the sharing and discussion of new material, even if it hasn’t been published, accepted or even submitted. The dialogues we have in the question sessions or in the hallway are much like the exchange in a blog’s comments. It is just another arrow in the quiver of science communication.

    Daniel

  • Science themselves changed the label on the gel. ERV the blogger got that wrong and Bruce Alberts shouldn’t have decided he had the psychic abilities to say differently. Both he and John Coffin know Science altered the label for publication to protect the identity of patients and simplify the information for that paper.

    There is no XMRV saga. XMRV is a sequence, and it is not the sequences that have been detected in humans. The science saga will end with the retraction of the negative papers that used inferior unproven methods and the reinstatement of Lombardi et al and Lo et al.

  • Daniel (whose blog is Waiology) – that’s my general view too.

    Liz – He could arrange to have a preprint put into the arXiv preprint archive. It’s more used by those working in physics and mathematics, but a few biologists are putting their papers in there prior to print publication. At this point in time I believe you need to clear this with the journal, but it can be done. One example is Rosie Redfield’s critique of the ‘arsenic life’ paper that I mentioned in the article. (Rosie is also from UBC, where Prof. Shaw works.)

  • Krin,

    You wrote “Science themselves changed the label on the gel. ERV the blogger got that wrong and Bruce Alberts shouldn’t have decided he had the psychic abilities to say differently. Both he and John Coffin know Science altered the label for publication to protect the identity of patients and simplify the information for that paper.”

    I believe there is more to it that this but either way this is off-topic really (see below).

    “There is no XMRV saga. XMRV is a sequence, and it is not the sequences that have been detected in humans. The science saga will end with the retraction of the negative papers that used inferior unproven methods and the reinstatement of Lombardi et al and Lo et al.”

    I think it clear I was writing loosely – make that ‘XMRV causes chronic fatigue syndrome’ if it’s clearer. Either way, you appear to be trying to have this both ways, both that there is no “saga” (drama would have been a better word in hindsight) and there is. Shifting the target to ‘science’ quite accurate as this drama includes the role of advocates—non-scientists—pushing this notion. Either way — again! :-) — this is an aside from how the XMRV-cause-CFS drama/story was brought up in this discussion. Here the point is just that scientists discuss/review the science on blogs (which you’ve implicitly acknowledged).

  • Ha. I must file that image away for future use :-)

    You’re right about the distinction of message and medium.

    I think for some embroiled in controversial topics it may be an excuse to not front up after they’ve presented conclusions to the media or public ahead of presenting it to the scientific community.

    Readers might like to check out this example another reader pointed to via twitter (Title: Neanderthal sex debate highlights benefits of pre-publication — UPDATED.)

  • Thanks, Grant; that Neanderthal/human interbreeding debate is getting very interesting now :-) And I totally take the point that earlier availability of material could have seen that informed debate taking place somewhat earlier.

  • Hi Alison,

    You’ll have been following that better than me! Wish I had time, it sounds a great story. (Readers just need to look at the guy sitting on Alison’s left shoulder to see why she might be following this closely.)

    I imagine the back-n’-forth on blogs has been fun to follow and join in. (Probably like mitochondria Eve on bionet was in my student days.)

    It’s not helpful when the research work is not available while the story is still “live” as it lets the misconceptions / misrepresentations stand uncorrected. Hopefully with that story it’s starting to come right – ?

    The Jasmine Renata inquest one is a bit trickier with the want for transcripts also. (I still find it a bit weird that it’s nominally “open”, but has no transcripts.)

  • HDB:

    Thanks for the links. I should have posted the first one myself I read it when it came out after all. The second is new to me, I’ll try check it out tonight.

    “All that has little to do with Grant’s concerns here, of course.”

    Yes, indeed…

    It is an interesting story if you sit back and read it dispassionately.

Site Meter