Should all research papers have a 'Limitations' section?

By Grant Jacobs 09/10/2012 4


Would it help scientists, peer-reviewers, editors and writers (including reporters) if research papers included a section outlining the weaknesses of the work presented in a research paper?

Last month I covered a paper analysing media reporting of subsequent and related findings, published in PLoS ONE. Towards the end of this paper was a section titled Limitations. In this the author had outlined what they considered were weaknesses or limitations of their work.

Off-hand I can’t recall seeing a paper with such as section, at least not recently.[1]

When putting forward an argument for a case—what a research paper is—researchers should probe and test their argument, rather than ‘just’ offer supporting elements. Insisting a Limitations section be present may aid authors to be more critical of what they present, encouraging them to elaborate explicitly where they think it is weak.

It might assist peer-reviewers in assessing the paper and, perhaps, also be a place for outstanding issues from reviewers to be noted.

For wider coverage, e.g. in the media, the contents of the Limitations section might alert science writers and journalists to limitations of the work and hence things not to say about it.

What might be placed within a Limitations section should ideally already be present in the paper, for example in the Discussion. The main effect may be to simply draw these out more clearly, but you might also hope it might also encourage stronger papers and clearer communication to those less familiar with the technical issues of that particular niche.

What do you think?

Footnotes

1. Perhaps it is common in other fields? I typically read molecular biology and computational biology papers, with the odd foray into other areas.


Other articles on Code for life:

Thoughts on scientific abstracts also a science writing check-list

Outreach sections for research papers?

Teaching students to write scientific papers

On alternatives to academic careers and ’letting go’

What got you into science and what keeps you doing it?

What aspects of biology need to be explained better?


4 Responses to “Should all research papers have a 'Limitations' section?”

  • In many papers I read in med journals and in many I write there is often a paragraph in the discussion which begins something like “The trial/study had several limitations….”
    Whilst v good for authors to think about the limits of the conclusions I don’t think it necessary to state the obvious (eg “this was a small single centre study”) which a half discering reader can see. so limit limitations to the important, non obvious. I wld not insist every study have a limitations section.

  • Feynman would approve.

    There was talk a few weeks back of whether papers should have an outreach section, basically a simplified summary for journalists and PR departments. Which in some senses would be good, in others, terrible.

    If the journalists could be induced to read a limitations section – and I would use it to state the obvious here, it could be a good step to providing journalists and by extension (fingers crossed really really tightly) the public with a better idea of how science works.

  • Mine would be boring reading. Too little time, not enough data, poor code, not enough CPU, needs more negative controls, …

  • John,

    Loose thought: I expect papers reporting medical trials do more that most because they are single-study papers. Molecular biology papers, for example, typically draw together several small studies (experiments) probing one molecular system. For them it can be harder to find these statements as they can end up scattered throughout the paper, sometimes you really need to dig into the Supplementary material to determine what the limitations might be.

Site Meter