The importance of evidence for policy

By Matt Nolan 04/09/2013

Good paper by the Prime Minister’s Chief Science advisor on policy – focused strongly on transparent evidence based policy.  I can’t disagree with this as a framework!

1) identifying problems; 2) measuring their magnitude and seriousness; 3) reviewing policy options; 4) systematically assessing likely consequences of options; and 5) evaluating results of policy interventions.

When the Chief Scientist discusses evidence “nudging” policy, he is prescribing a Bayesian view of how we update our priors based on evidence.  This is all cash.

However, I don’t feel that the first stage is ever covered off particularly well … by anyone really.  In this context I just find the term problem uncomfortable – if research is based on understanding “issues” instead of “problems” we can then use data and theory to identify issues where a “problem” exists.  This is far from semantic – the very nature of how we decide something is a “problem” will influence the entire process of analysing policy interventions, either for the better of the worse!

The “appropriate interpretation” of data requires a transparent theoretical base as a starting point, a view I didn’t quite get out of this paper on reading.

Now do not take this as a criticism of the paper, I strongly recommend it, the description in it, and the conclusions of it – I agree with this post by Peter Griffin indicating the sheer importance of thinking in terms of evidence based policy!  And I would note that Bill Kaye-Blake has been making noises about this for some time!

I just thought I’d add my two-cents about the very hard initial identification step, especially given my view that it is very difficult to get sufficient evidence involved to shift our priors together from disparate starting points for many policy questions.  Describing a relationship between a set of variables observed variables is inherently different than identifying than doing welfare analysis of policy relevant issues given:

  1. That heterogeneous individuals make choices based on values we cannot observe, but implicitly wish to target
  2. We need an understanding of the way relationships may change when policy changes

All in all implying our conclusion must involve tacit (or preferably explicit) assumptions about value, and structure to help inform the role of policy given the incentives and following choice of individuals ;)

And this view is indeed in the thrust of where the advisory paper is going, I just felt spelling it out in the way social scientists/economists interpret it would be useful :) .