How should PhD theses be reviewed to avoid a nonsense thesis? Especially if the work is between two disciplines.
Today a PhD thesis that might be considered an extended anti-vaccine essay was brought to my attention. The red flags start popping up from the very first sentences of the abstract. I got as far as the second sentence and honestly thought “Amateur hour?” The flaws are obvious — to anyone familiar with vaccine science.
But that looks to be the catch. This thesis is not from a Science department but from Humanities.
I’m a big fan of cross-disciplinary work.* A key element is that they really need to include solid expertise in both of the two disciplines spanned.
It’s related to that old line, especially familiar to computer science graduates: garbage in – garbage out.
If you’re going to do work that builds on another specialist area, you really need to ensure that the underlying material you’re building on is sound. Was any attempt made to ensure the science in this thesis was sound? It doesn’t look so.
Fellow blogger and vaccine specialist, Helen Petousis Harris, has looked at the science of this thesis, as well as another fellow blogger, Alison Campbell. To be polite, it doesn’t come out well. I’d like to consider how this thesis has, or has not, been reviewed to raise the question of how reviews might avoid the antithesis of a thesis.
Helen calls it a ‘stealth’ thesis — a thesis done in a different division in the university under a ‘favourable’ supervisor, presumably to escape criticism.
When you read the abstract, it’s making statements about the science of vaccines not just about humanities aspects. A thesis is supposed to critically examine an issue. Critically doesn’t mean ‘be negative’, it means the student must examine in depth their hypothesis: they must critique it. This, in turn, means a thesis can realistically only be done in a department that has expertise in the topic. Humanities departments do not cover vaccine science.
So the first question is why was this being done in a humanities department? At the very least it would want to be jointly supervised with staff from a medical department familiar with vaccines.
Even there, there is (rare) opportunities for abuse, or more ordinarily, sloppiness. There are a tiny number of scientists with odd views about vaccines. (The thesis acknowledgements thank a few examples.)
Universities are given accreditation by their governments to confer particular types of degrees. With that universities are asked to hold to some (hopefully high) standards.
How might universities ensure that selection of Ph.D. advisors and reviewers avoid shoddy work being put out under their name, under their accreditation status? A few common steps are listed below.
Review of the thesis proposal
Thesis proposals are usually reviewed by the department. This should pick up things like that the thesis is cross-disciplinary in a way that need input from another department, that it looks unlikely to produce new understanding (a requirement of a PhD thesis), and so on.
It seems hard to believe this has happened (properly).
Probationary period reviews
Many modern Ph.D. theses are dependent on internal reports over the early period. For example, the University of Otago has a probationary of a year. If these fail, the student either cannot continue or is asked to bring their work up to a better standard.
A main tenant of a thesis is that the student critically examines evidence. This means they much consider all relevant aspects. For example, they cannot ignore vaccine science that does not appeal to them. On this note, an offer to help on the science was abandoned because the student would not take on board the science. This would demonstrates a failure of a aspect of being a Ph.D. student, one that I would like to think would fail a properly-done probationary review.
Typically, these have two or more reviewers. In New Zealand three is typical: one internal examiner (from the department), on external examiner, and one international examiner.
How the examiners are chosen matters. The selection should provide sound critique of the work. The Ph.D. supervisor will, understandably, favour their student succeeding. But they must succeed on sound terms.
There are several steps that should have brought in criticism that initiated action over concerns about this work. I think it’s fair to suggest University of Wollongong has a problem it needs to address in this department.
It makes me wonder if some of these steps want to include staff from outside of the department, perhaps at a faculty level with the idea the input that is more distant to the supervisor and his immediate colleagues is likely to be more partial and more able to speak out.
Perhaps independent adjudicators have there place in some steps?
Another thought was if the Ph.D. examiner’s names should be open. As a practical matter they will be known to the department. Aside from the committees, the examiners travel to the department for the examination.
These are just loose thoughts: I claim no special insight.
We should also remember that all human endeavours are prone human aspects of error, relations, unwillingness to speak up and so on. But these are things we ought to try mitigate against if we want quality.
Added belatedly: Alison Campbell has looked the freedom of speech aspect of this. This does not excuse people from putting forward sound argument.
Please feel free to offer suggestions in the comments.
The broad issues I’ve touched on also apply to over work that (legitimately) span disciplines. Science communication for example wants to be build on sound science. Similarly, history and philosophy of science should not mangle the science.
They also apply to review of grants, and maybe job applications.
Image credit: Wellcome Library, London. (Wellcome Images http://wellcomeimages.org) The history of vaccination seen from an economic point of view: A pharmacy up for sale; an outmoded inoculist selling his premises; Jenner, to the left, pursues a skeleton with a lancet. Coloured etching, c. 1800. Copyrighted work available under Creative Commons Attribution only licence CC BY 4.0.
On an only tangentially-related note, I really wish the Taleban would stop attacking vaccine workers. Aside from the obvious upsetting tragedies, it’s one of the key things preventing polio from being eliminated world-wide. Only Afghanistan and Pakistan still have (endemic) polio.
* My own PhD is an example in a different way. Computational biology spans computer science and (molecular) biology. Today it is considered an established discipline in it’s own right, but at that time it was a fairly unusual mix.
See other Sciblogs posts on the University of Wollongong thesis here.