How much should we rely on stress tests?

A commenter, Kelly, writes as follows:

Michael – is it necessarily true that the failure ( or lack thereof) of a bank stress test is the benchmark for assessing the worth and more specifically the legality ( in terms of consistency with the RBNZ’s act) of regulatory action? The relevant section of the Act says if I remember correctly that the Bank must use these powers for the purpose of ensuring the soundness and efficiency of the financial system. While the stress test results are one potential indicator they are not a necessary condition to justify regulatory action. Indeed I suspect that very little prudential regulation might be justified if this were the case. Stress tests are indicators of what might happen given a number of important assumptions any of which may not be true in real life. Also stress tests almost always are calibrated to ensure banks pass them ( or else additional capital should be added by failed institutions). History shows that stress tests have a poor record at predicting financial crisis. I think many reasonable people would consider a large fall in Auckland house prices might challenge the stability of the financial system in NZ. It’s true that in some circumstances it might not but would you expect the Bank to bank on being that lucky? The spillovers to the rest of the country would potentially be great.

The Bank’s actions in this area are in step with the generally successful international experience with regional LVR type measures. I think that this alone makes it difficult to argue the Bank is acting in an ultra vires manner. The IMF’s advice in this area is that macro prudential measures can be employed assuming key macro policy settings are also supportive. Your stance on monetary policy settings thus seem further supportive of the Bank’s stance. Although I am sure that the government could be doing more to help as you point out.

I’ve always enjoyed debates with Kelly, and I thought it might be worth addressing a few of the points he makes:

  • The Reserve Bank Act (sec 68) requires that the regulatory powers in Part V of the Act be used (by the Governor General, the Minister, and the Bank, but here the Bank’s discretionary action is in focus) “for promoting the maintenance of a sound and efficient financial system”.  Note that it is not about “ensuring” such an outcome, and neither has a “sound” financial system ever been regarded as one in which no bank failed.
  • Kelly suggests that very little prudential regulation might be justified if stress tests were to be used a key indicator.  I don’t think that is right.  After all, the risks that the Reserve Bank and APRA assessed last year were those that existed given the current panoply of regulatory controls, including the earlier LVR restrictions.  In other words, some mix of market disciplines and regulatory measures had produced loan books which appeared to be pretty robust to some pretty severe shocks.  The issue here now is whether a case can be made, in terms of the Reserve Bank Act, for further, quite intrusive and quite unconventional, controls.   That case would have seemed much easier to make if the stress tests had produced results which appeared more threatening to the soundness of the financial system.
  • Kelly alludes to widespread scepticism about the use of stress tests by some regulatory and supervisory agencies.  It is certainly true that one would not expect a regulator to publish stress test results, particularly those undertaken in the midst of a financial crisis, suggesting that the financial system was in great danger without having first taken remedial action.    But context matters.  Clearly New Zealand and Australian banks are not in crisis now.  Moreover, the timing of the recent NZ stress test was determined by APRA, not by the Reserve Bank of New Zealand, and occurred at a time last year when, if anything, the Reserve Bank seemed to be looking at the possibility of winding back, or lifting, the avowedly “temporary” LVR restrictions.  When the results of the stress tests were published in the November 2014 FSR there was no hint of any undue concern about risky lending practices or region-specific controls.  In other words, even from outside, I think we can acquit the Reserve Bank of any suggestion that it undertook the stress tests with a particular end in mind.  Instead, they really wanted to assess the health of the New Zealand system, posed some deliberately searching test scenarios, and were pleasantly surprised by the results.
  • Stress tests aren’t the only possible measure of financial strength.  Indeed, in the day-to-day activity of prudential supervision they play a much less important role than regulatory capital requirements.  In fact, it is unease about the use of not-very-transparent, hard to evaluate, internal ratings-based models for calculating capital requirements that has encouraged many to put more weight on stress tests in recent years.   But in any case recall that New Zealand banks have high capital ratios by international standards, and that is so even when higher risk weights are applied to housing exposures than in is done by banks in many other advanced countries.
  • Kelly notes that “many reasonable people would consider a large fall in Auckland house prices might challenge the stability of the financial system in NZ”.  As a matter of description that is certainly true, but that is why, for example, stress tests are done and why, for example, Parliament mandated the publication of Financial Stability Reports. Those reports are supposed to enable us to better understand, and to assess, the risks and the Bank’s regulatory activities.   And I’m certainly not suggesting that New Zealand should rely on luck to get through an eventual house price adjustment (rather, high capital ratios, and slow growing balance sheets are what suggest pretty moderate risks)   Indeed, I was one of those who was a bit surprised when we saw the first stress test results.  I pushed and prodded, and wondered if they were missing something important.  Sunny optimism is not in my nature, but in the end I was persuaded.  There are no doubt some things the stress tests don’t fully capture –  the possibility of sustained deflation is one of those risks, but that can’t be the concern the Bank is relying on (since the word does not even appear in the FSR).  And frankly nor would I expect it to be at this stage.    It seems likely that the Governor does not really believe the stress tests, or the capital ratios.  There might well be a good reason for his doubts, but he owes it to us, and to Parliament (through FEC), to explain the nature of those doubts, and allow them to be scrutinised and challenged.  It isn’t enough –  given his Act –  simply to worry that Auckland prices are too high and might come down one day.   We particularly need to be able to scrutinise the Bank’s analysis given that it is a unelected single individual  –  with, frankly, not that much experience or background in these policy areas –  who is making these regulatory calls.
  • Finally, Kelly notes that “the Bank’s actions in this area are in step with the generally successful international experience with regional LVR type measures. I think that this alone makes it difficult to argue the Bank is acting in an ultra vires manner.”   There are at least two separate points here.  First, as a matter of law I’m not arguing that the Bank’s proposed new controls would necessarily be ultra vires.  Some others might.  I would argue simply that they are using the legislation for purposes Parliament never envisaged, which is not good for the legitimacy of policymaking, and that – at present – they have not met any sort of burden of proof regarding why such arbitrary restrictions on investment property lending (of the sort never before used in New Zealand, even under the Nash to Muldoon years) are either necessary or desirable to further enhance the soundness of the financial system, especially in face of the inevitable efficiency costs.  It is important to remember that whatever the preferences of the Governor, the Reserve Bank has been given powers by Parliament for the promotion of the “soundness and efficiency of the financial system”.  That is different from the mandates regulatory agencies in many other countries have.    Whatever other countries have done, the Reserve Bank must look primarily to its own legislation.   Second, Kelly suggests that Auckland-specific restrictions are in step with “generally successful international experience”.  I think that is wrong, at least in the sort of countries, market-based economies, and systems of government we typically like to compare ourselves with.  I’m not aware of regional-based banking regulatory controls in any of the other Anglo countries for example, or in the Nordic countries.

Perhaps all will become clear when the Reserve Bank releases its consultative document on the proposed new controls.  I certainly hope so.     But there is a lot to clarify.

2 thoughts on “How much should we rely on stress tests?

  1. Hi Michael – an interesting debate as always. In terms of your point on the international evidence I would say you really can’t focus on Anglo type countries for leadership in the macro prudential space as its Asia and Latin America that have led the charge in the area. The Anglo countries are very much in catch up mode. The following reference has some useful material on the use and effectiveness of measures ( but a quick literature search will yield many more. Effectiveness seems encouraging – within limits of course – you can’t overcome poor macro policy settings with macro prudential. China and Korea have both had some success with regional LVRs and many others have had sectoral targeted LVR measures. A priori would we expect these to be less effective in NZ with its relatively better developed financial system? I think maybe as the banking system plays an important role in intermediating credit and there are stronger linkages between the cost of capital and finance in NZ than in less developed countries.


    • Thanks Kelly

      I guess I’m reluctant to look to Asian and Latin American countries for policy guidance on anything much. Every single Latin American country is much poorer, and less successful on almost any count, than New Zealand (and as you know, I’m not exactly one to champion NZ’s success over the last 50-100 years). A few of the Asian countries now have higher levels of productivity than we do – Korea is about the same – but then Singapore and Hong Kong are hardly exemplars of the sort of political system we want. So, frankly, it worries me when I see Anglo countries being lured down these sorts of paths.

      In terms of effectiveness, I think the Bank’s earlier LVR controls had about the impact that was expected – documented in that Chris Blor Analytical Note. That meant a pretty small, but real, impact on prices for a year or so. As you note, you can’t overcome poor macro policy settings with banking regulations, but I’d also note that you can’t overcome poor micro – the land supply vs population pressures – with banking regulation. Especially when we already seem to have a very sound banking system!



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s