Ipsos custodes

By Bill Kaye-Blake 25/06/2013

The Dominion Post had a spread on the latest concern about the US NSA over the weekend, entitled ‘Prism: Why We Should Worry’.

Before I get to my main point, one digression. If we start from the idea that words matter, what do we call Prism? ‘Spying’ evokes the Cold War and country-based activities that don’t or shouldn’t involve ‘civilians’. ‘Intelligence activity’ begs the question: it assumes that knowledge is being produced before that has been proven. ‘Eavesdropping’ makes it sound like a nosy neighbour, picking up snatches of conversations. ‘Hoovering up metadata’ is closer to the truth, and does lead to the easy-to-use ‘HUM’ acronym. Of course, ‘hoover’ as a synonym for ‘vacuuming’ is culturally specific, so might not work, but the resonance with J. Edgar works, too.

Digression done.

One of my favourite types of economic projects is the cost-benefit analysis. We get to go into an organisation, assess what they have been doing, and come up with some advice on what works and what doesn’t. Done well, we challenge people’s notions about what they have been accomplishing and whether it was worthwhile.

It is not unusual to find limited evidence that a programme or part of a programme has been effective. By effective, I’m not imposing an outside criterion; I mean ‘effective’ by the organisation’s own standards of what they are trying to do. Nothing takes more time than explaining to people that this great project with documentation and logic and funding and internal stakeholders has been a bust. One project I looked at had impact reports about all the uptake by external people who had been involved, but conversations with those actual people suggested that involvement and impacts had been minimal. Another project had a clear logic laid out about how people would move through stages from awareness to assessment to adoption, and there was no evidence in their own files that anyone had actually been progressed through the stages. And on and on.

The internal people really believe in their programmes. They believe in the mission and goals as well as the implementation. They are surrounded by other people filtering information to reinforce the idea that the programme works. Anything that hints at success is seized upon as evidence of the great things they are doing. Anything that suggests otherwise is explained away: the external party, the context, the timing, lots of things can explain failure without implicating the programme itself. Even data — such numbers from their own databases — are rejected as ‘not telling the full story’.

And that is what I read in the articles about Prism: people inside the programme convinced of the value of what they do, no different from everyone else. We’ve stopped 50 attacks! We’ve saved countless lives! It works!

That’s the value of an outside assessment, a cost-benefit or cost-effectiveness analysis or other approach. These watchmen are, after all, only human. Like most other humans, they believe in their own virtue and sincerity, and thus in the rightness of their actions.

Quis custodiet ipsos custodes? Economists, that’s who should do it.