I teach a course called “Scientific Controversies and Misconceptions.” (You won’t be surprised to learn that one of the main topics is evolution vs. creationism.) The most difficult part of the course is teaching students to be skeptical about the scientific literature. The reason why this is difficult is because the main focus is evidence based reasoning and science is an important way to gather reliable evidence. […]
I haven’t figured out a way to teach students to read the scientific literature with the same skeptical perspective as those of us who have been doing it for decades. It seems as though this is a skill that must be learned through experience and can’t be taught.
I feel a little uneasy at his second paragraph. You’d like to think that this process can be learnt by more that just osmosis or trial-and-error. (I do anyway.)
A part of how I learnt this as a student was through small-group literature-based classes in the final-year of my BSc(Hons). The groups consisted of a series of meetings. At the end of each meeting we were set a small number of papers to read and draw our own conclusions. At the next meeting the class would discuss what conclusions we had drawn, aided by the staff member. These classes had from a handful to a dozen students.Keeping the chemistry analogy going, part of this might be considered directed diffusion as compared to passive osmosis. When the staff member broke down the critical thinking involved you had something closer to formal teaching. This approach obviously depended on the small sizes of the groups – it would be difficult with a larger number of students as it is very interactive.
Part of this is simply to show the structure of a typical research paper and what contributions the different sections make. (Some of this is taught earlier in a degree program, but you get my point.) Another is to reason carefully with what is presented, using an understanding of the techniques used in the research and their potential pitfalls, and so on.
Another contribution were ‘journal clubs’ – where staff and students pick a (usually recent) research paper and present an informal critique of it over pizza, etc.
A contribution I’ve found helps is to have some knowledge of the history of whatever niche you are critically reviewing – not just so that you know what went before, but also so that you can consider why particular lines of enquiry where undertaken to get some idea of understanding why the field (or niche) is in the current state it is. This is a little harder for taught classes to address as there generally isn’t the time to follow one particular niche at the sort of depth this requires.
How did you learn to critique the literature? Do you think any particular approach helps teach this more than others?
I’m writing here of university students learning criticism of the research literature, i.e. for research purposes, not teaching a general understanding of science or general critical thinking as previous discussions on this general topic have been. I’m thinking less of how people might have improved on this after their student days, but you’re welcome to add thoughts on that, too.
For contrast you might consider teaching science to the very young! (If I find time I may tackle this – you’re welcome to get ahead of me.)
* Larry’s article is about the recent rise (real or apparent) of retractions in top-end scientific journals and scientific fraud, but I want to look at the teaching issue.
Other articles on Code for life: