By John Pickering 28/09/2020

In a few weeks, New Zealanders will make a choice whether we implement into law the End of Life Choice Act 2019.  My scientific expertise includes developing and validating methods to predict future events of ill people including death. There is one section of the Act that concerns me deeply.

Section 5(1)c of the End of Life Choice Act 2019 states that one of the criteria of eligibility for assisted dying is that a person “suffers from a terminal illness that is likely to end the person’s life within 6 months”.

Concern 1: How likely is likely?

What does “likely” mean?  Does it mean a 51% chance of dying or a 99% chance?  The Act does not define it. This means that the decision as to what “likely” means is left to the individual physicians’ involved in the decision whether to grant the person’s request for assisted dying.  It is inevitable, therefore, that should this Bill be enacted that there would be inconsistency in the application of this clause. Some physicians would be more liberal in their interpretation than others. Physicians are human and subject to the subtle pressures and biases that affect decision making. The tone of the voice or the story told may affect whether they rate someone as “likely” or not. The physician’s own prior experience plus their familiarity with the literature around a particular disease will be a large part of the equation.  These too will vary considerably between physicians. Unsurprisingly, the literature suggests the accuracy of physician estimates of when a terminally ill patient will die are poor and varied (reference).

Concern 2: How accurate are physicians?

My day job is to help physicians make better decisions by providing them with objective assessments of risk of current or future events (eg risk of a heart attack or risk of dying). Developing and validating these risk prediction models is difficult. In the context of the End of Life Choice Act, the statistic that is most relevant is called calibration. A prediction model is said to be well-calibrated when it gives a prediction of an event of say, 60%, to 100 people 60 of them go on to have that event. Similarly, if the prediction is 5%, 5 out of 100 with that prediction will have the event. Etc. The figure shows a calibration curve I produced of a model that performed very well. Many models, though, may have very good calibration at low predictions, but poor at high ones. Others may be only averagely good across the range. What it means at a particular part of the range when a model is inaccurate is that it systematically over- or under-estimates the risk. It is rare for a model to be accurate across the whole range of risks.

Example calibration figure where the Observed rate of a myocardial infarction (heart rate) is compared to the prediction made by the MI3 algorithm. This is an example of a particularly good calibration. It is published here.

The other statistic that is relevant is called discrimination. A good prediction model discriminates between those who go on to have the event and those who do not have it by allocating to those who went on to have the event a high probability and to those who didn’t go on to have the event a low probability. In the ideal model, these probabilities would be 100% and 0% and the model would never predict a risk of 100% to people who didn’t have the event or 0% to those that did. Of course, these ideals are never reached in medicine.  Humans are just too complex and we cannot accurately measure every variable that matters. Psychological variables are particularly difficult to quantify, and these are incredibly important to a terminally ill person’s wellbeing and prognosis.

My point is simple – even the best scientific techniques applied to estimate risk of death are not perfect and often far from perfect. Should this bill be enacted, how much more inaccurate will the physicians’ estimates of risk be? How many people who would have survived more than 6 months die prematurely? I can’t predict this. For this reason, I believe the End of Life Choice Act is an experiment. If it had been submitted as a study protocol for review for a research grant or to a health ethics committee at the very very least a measure of the accuracy of the physician risk would need to be part of the protocol. There would also need to have been stopping rules that if the physicians were proved to overestimate risk (as they do for a number of diseases) then the experiment would be halted. Without these safeguards, if for no other reason, I believe the precautionary principle should apply and the experiment that is the End of Life Choice Act 2019 should not go ahead.