How did you learn to critique the scientific literature?

By Grant Jacobs 22/08/2011 6


Larry Moran writing on his blog, Sandwalk, started a post* with these remarks about teaching students critical reading of the scientific literature,

I teach a course called “Scientific Controversies and Misconceptions.” (You won’t be surprised to learn that one of the main topics is evolution vs. creationism.) The most difficult part of the course is teaching students to be skeptical about the scientific literature. The reason why this is difficult is because the main focus is evidence based reasoning and science is an important way to gather reliable evidence. […]

I haven’t figured out a way to teach students to read the scientific literature with the same skeptical perspective as those of us who have been doing it for decades. It seems as though this is a skill that must be learned through experience and can’t be taught.

I feel a little uneasy at his second paragraph. You’d like to think that this process can be learnt by more that just osmosis or trial-and-error. (I do anyway.)

A part of how I learnt this as a student was through small-group literature-based classes in the final-year of my BSc(Hons).  The groups consisted of a series of meetings. At the end of each meeting we were set a small number of papers to read and draw our own conclusions. At the next meeting the class would discuss what conclusions we had drawn, aided by the staff member. These classes had from a handful to a dozen students.Keeping the chemistry analogy going, part of this might be considered directed diffusion as compared to passive osmosis. When the staff member broke down the critical thinking involved you had something closer to formal teaching. This approach obviously depended on the small sizes of the groups – it would be difficult with a larger number of students as it is very interactive.

Part of this is simply to show the structure of a typical research paper and what contributions the different sections make. (Some of this is taught earlier in a degree program, but you get my point.) Another is to reason carefully with what is presented, using an understanding of the techniques used in the research and their potential pitfalls, and so on.

Another contribution were ‘journal clubs’ – where staff and students pick a (usually recent) research paper and present an informal critique of it over pizza, etc.

A contribution I’ve found helps is to have some knowledge of the history of whatever niche you are critically reviewing – not just so that you know what went before, but also so that you can consider why particular lines of enquiry where undertaken to get some idea of understanding why the field (or niche) is in the current state it is. This is a little harder for taught classes to address as there generally isn’t the time to follow one particular niche at the sort of depth this requires.

How did you learn to critique the literature? Do you think any particular approach helps teach this more than others?

Footnotes

I’m writing here of university students learning criticism of the research literature, i.e. for research purposes, not teaching a general understanding of science or general critical thinking as previous discussions on this general topic have been. I’m thinking less of how people might have improved on this after their student days, but you’re welcome to add thoughts on that, too.

For contrast you might consider teaching science to the very young! (If I find time I may tackle this – you’re welcome to get ahead of me.)

* Larry’s article is about  the recent rise (real or apparent) of retractions in top-end scientific journals and scientific fraud, but I want to look at the teaching issue.


Other articles on Code for life:

Should we teach examples of scientists falling for unscientific practices?

A course for all degrees: PHIL 105, Critical Thinking

Professors, lost souls with great oratory power?

Retrospective–The mythology of bioinformatics

Minorities, disabilities and scientists


6 Responses to “How did you learn to critique the scientific literature?”

  • I got taught in a third year undergrad honours paper which involved weekly critiquing of various scientific literature. There was also other third year papers that followed a similar principle.

    Most people who only performed undergrad would have missed the vast majority of this, and even going through to post-grad, time spent on critiquing human clinical/epidemiological work was limited to self motivated work.

  • Journal clubs and classes in which literature is carefully reviewed is surely the experience that Larry is referring to. I found that as an undergraduate I was more likely to just cite material to support the ideas I had already formed, or because that is what my lecturer taught me. In my second year I had a paper where each student was given a paper to review and present during class time. This was followed by a discussion between the class, the presenter and the lecturer. Only after this class did I begin to pay more attention to the intricacies of the research I was reading.
    As a masters student one of the classes I took the supervisors purposefully chose some problematic papers for us to asses. I think this was when I began realising that being published in a peer reviewed journal did not guarantee sound research.

  • […] I and others at sciblogs have written about the value of critical thinking and have ruminated about how it is taught, both in universities and schools. The passage above reminds me of Larry Moran writing similar thoughts about teaching university students critical thinking of the scientific literature. […]

Site Meter