Back in the last “lockdown” I linked to various pieces of work by other economists attempting to make sense of, evaluate etc, choices the government was making.   There was Ian Harrison’s work challenging some of the modelling estimates the Prime Minister liked to wave around and some aspects of the “Level 4” restrictions.  There was an early attempt at a cost-benefit analysis by Bryce Wilkinson of the New Zealand Initiative, and another exercise looking at a similar question in a different way by John Gibson at Waikato University.  There was another exercise that I never wrote about, but which is reported and linked to here, by Martin Lally, a consultant economist and former Victoria University academic.

What was striking, even at the time, was that there was no sign that the government had commissioned from officials, or officials had undertaken anyway, any sort of serious cost-benefit analysis of the sorts of intervention they were looking at and imposed.  It always seemed likely at the time that there was nothing of the sort –  the public sector had, after all, been woefully underprepared, sluggish in getting any serious planning underway, and complacent for too long that this was largely someone else’s (PRC’s) problem  Anyway, when the government finally got round to publishing the relevant documents, sure enough there was no serious structured attempt to cost and evaluate alternative policy options.  (It is not, I hasten to add, that any cost-benefit analysis can give one “the” answers, but it provides a disciplined framework to analyse the options, assumpions and sensitivities.)  But there was nothing –  even though the New Zealand authorities had the best part of two months of lead time.

These issues take on a fresh salience with this week’s out-of-the-blue partial lockdown of Auckland, and the government decision later today.  It prompted me to finally go and take a look at an exercise undertaken by an economist at the Productivity Commission in early May, illustrating for the benefit of The Treasury –  who we used to assume were the champions of robust cost-benefit analysis –  how the decision in late April on whether to extend “Level 4” for another five days might have been rigorously analysed in a careful cost-benefit framework, looking only at the marginal costs and benefits of the two options the government had had in front of it.    The author concluded that, with the information available at the time, the extension was probably not justified, but that is less relevant than the fact that an economist at another agency was having to do this for The Treasury after the event.  Apparently neither The Treasury nor ministers had been interested in getting such analysis done when the decisions were being made.

Restrictions –  border restrictions –  have remained in place, but there seems to have been relatively little interest in evaluating the costs and benefits of those choices.  But this week’s restrictions have brought the issue back into focus.     There have been a couple of newspaper articles, notably in today’s Herald: this by Kate MacNamara, and a column by (newly returned from working for the National Party) Matthew Hooton.  MacNamara explicitly ends her piece with the argument

“There will be a time when the best option is to ease border restrictions, abandon lockdowns, and let our health system, including tracking and tracing, do the heavy lifting. We need credible analysis to help us know if that time is now.”

I’d say “perhaps” to the first sentence –  and it remains troubling that there is no identified or championed (by the government) credible exit strategy from our current eliminationist/closed-borders model – but would strongly echo the call for serious, open, analysis on the issue and options.

Martin Lally’s latest paper on a cost-benefit approach isn’t that analysis –  we need proper marginal analysis on the costs and benefits from here, with what has happened to now in principle largely irrelevant (sunk costs and all that).  But Martin’s paper, which he has given me permission to share

Martin Lally cost-benefit assessment of Covid lockdown August 2020

is still a useful look back at the merits of choices made over recent months, and probably sheds at least some light –  poses some questions –  on how the choices going forward might look.

His conclusion is as follows (QALY = “quality-adjusted life year”)

This paper considers the effect of the New Zealand government adopting a suppression policy versus a milder mitigation policy, with the actions of other governments taken as given. The cost per QALY saved from doing so would seem to have been vastly in excess of the currently used value for a QALY of $45,000. Consideration of alternative parameter values and recognition of factors omitted from the analysis would not likely reverse this imbalance in cost per QALY saved versus currently accepted figures for the value of a QALY. The suppression policy was therefore dramatically inconsistent with long-established views about the value of a QALY.

The broad approach is to look at lives saved by the government’s elimination approach and the (primarily) economic costs of that strategy.   Neither is necessarily straightforward.  On the economic side, one sometimes hears champions of the government touting a view that there is no such economic cost –  in fact, I heard former Labour leader Phil Goff make exactly that claim this morning. Locking down hard, while costly initially, is –  these champions conveniently claim – its own reward; initial losses more than outweighed by the subsequent gains (faster sustained recovery etc).  But there is no actual evidence for these claims –  at best such an outcome could be considered as one scenario.  (In the early days, the PM was claiming support from 1918, suggestions I looked at here.)

Perhaps that line might have seemed more plausible to some just a few days ago.  But then, with essentially no notice, our largest city was flung back into a partial-lockdown, and whatever choices the government announces today, we are told to expect more of these events, timing and size of course unknown and unknowable.    So we take further real output losses now and –  perhaps at least importantly – fresh huge uncertainty (affecting all manner of firms, and households too).    Perhaps the government can finally fix up border testing –  isn’t it just staggering that two-thirds of people working at aiports/MIQ facilities etc haven’t been tested at all? – reducing the chance of further outbreaks/lockdowns.  But even if that were done as best as humanly possible, it wouldn’t change the limitations of the closed border itself.

And the difficulty for champions of the “own reward” model is the absence of a compelling exit strategy.   If we could count on the virus simply dying out, going away, by some clearly defined date next year, the calculations change quite a lot.  There is a credible exit strategy then, and we just have to hold on til then.  Similarly it we could count on a highly effective vaccine being generally available by some clearly defined date next year, again things look more encouraging for the “own reward” story.   Perhaps those too are scenarios to add into a serious evaluation of the strategy.  Along with scenarios in which there is never a very effective vaccine and/or the virus remains much as it is indefinitely.

In any case, what Lally does is to assume that some –  quite moderate –  proportion of the difference between the Treasury’s GDP forecasts from last December and those from this year’s Budget should be treated as the cost of the elimination approach.  His central case assumes 25 per cent.  That may be too high.

The other side of the equation is, of course, lives saved (and reductions in impairments to the quality of life, of those with serious but non-fatal Covid).  Of course, some of that early modelling suggested catastrophic losses if we hadn’t gone to a fairly severe lockdown.  But if, as Harrison suggested, those numbers didn’t look that plausible at the time, they look much less so now.    Lally focuses on the case of Sweden, which has pursued –  not always well –  something closer to a mitigation policy.

To date Sweden has suffered 570 deaths per 1m of population and the increase in the rate is tailing away to zero.  New Zealand’s population of 5m implies 2,850 deaths under a Sweden-style mitigation policy. The QALYs saved would then be (2,850 – 22)*5*0.5 = 7,070.

It is a sample of one, but again he illustrates that you can assume a materially higher numbers of QALYs saved and the calculations still don’t end up very favourable to the New Zealand approach.   A further caveat is that, although he notes the point, Lally does not explicitly allow for the QALYs saved in respect of the people with serious non-fatal Covid cases.  The Productivity Commission piece does include some estimates, and if I’ve read document correctly, the effect is to double the overall QALYs saved.

Lally is very conscious of the sensitivities in his analysis. This is the last extract I’m going to quote.

The parameters used in this analysis are debatable. The death rate under a mitigation policy may be much larger than estimated here. If it is doubled, the cost per QALY saved would halve to $4.25m, but would still be 94 times the usually accepted figure. The GDP loss from the current path relative to that if there is no curtailment in economic activity could be smaller. If it were halved, in addition to the death rate being doubled, the cost per QALY saved would fall further to $2.12m but this would still be 47 times the usually accepted figure. The remaining parameter is the proportion of the GDP loss due to lockdown rather than mitigation, which is unknown. However, any reasonable proportion will produce a cost per QALY saved well in excess of the usual figure of $45,000.

(Incidentally, I prefer a high number for the value of a QALY –  the Productivity Commission paper discusses some of the options.)

My point in this post is not to articulate a strong personal view on what the government should have done, or should do now.  As I’ve said in past posts, my visceral reactions tends to be more cautious than my analytical one, and one shouldn’t discount visceral reactions.  And in the last lockdowns, my bigger concerns were about the overreach in many of the non-economic restrictions –  remember the government that totally banned funerals, or a solitary swim at a quiet suburban beach.

But I reckon there is crying need for more analysis –  open and transparent, disciplined analysis, exploring a wide range of asssumptions and scenarios.  As I noted, Lally’s paper isn’t that for the period ahead –  we need marginal analysis from here, that explicitly takes account of the uncertainty of the relevant end dates –  but it is still worth reading, perhaps especially so in conjunction with the (slightly longer, more detailed, and better-tabulated) Productivity Commission piece, which represents the sort of analysis we should be expecting from our core government officials –  notably The Treasury –  were they adequately (well, excellently) doing their job.   And as the government ploughs on –  apparently supported by all other parties –  with their eliminationist approach, we deserve a credible, carefully evaluated, exit strategy.  At present, there is none.

UPDATE: Lally has responded to my point that his paper is not a marginal approach (costs and benefits from here) and so can’t shed light on choices from here, and has added a paragraph (in this version) offering one way of looking at that question concluding that

“Switching to a Sweden-style approach is therefore clearly warranted.”

Those who believe that virtue is its own reward (as above) will certainly not be persuaded.   My own reaction is that  –  as per my final paragraph –  more analysis is needed, drawing on the combined expertise of economists and epidemiologists.