Please improve immigration data, not undermine it

Michael Reddell Immigration, Statistics

On her visit to Australia, the Prime Minister has been quoted as suggesting that departure cards might soon be discontinued, and that she will be pursuing her Customs and Statistics ministers on the matter.

I’m sure airlines and airport operators hate the cards.  There have been prevous efforts to get rid of them.  They are, nonetheless, a core element of the data collections (in conjunction with arrivals cards) that give us some of the very best immigration data anywhere.  In a country with –  year in, year out – some of the very largest immigration, and emigration, flows anywhere in the advanced world.

I wrote about this a few months ago when, under the previous government, Statistics New Zealand publicised the possibility/likelihood of departure cards being discontinued.  At the time, SNZ suggested that

“In the near future, the outcomes-based ‘12/16-month rule’ is expected to become a key component in how we determine the number of migrants in New Zealand.”

The “12/16 data” are the new series of permanent and long-term movements derived by lining up, using passport details, people coming and going, and waiting until more than a year after the initial movement to see if the movement loooks permanent or long-term.    It is all very interesting – I’ve praised SNZ for putting the collection in place –  and provides a more accurate measure of actual long-term comings and goings than the (stated intentions based) arrival and departure card.   But it is only available with a very long lag  (ie more than 16 months), whereas the existing PLT data are available monthly, with a few weeks lag (and in principle could be produced even more frequently).

I’m reproducing here the concerns I expressed in September

I’ve explained here previously why the resulting PLT data has its limitations.   It isn’t a good basis to use to look at immigration policy itself.  Approvals data from MBIE is better for those purposes –  and would be better still if they made the information available in an accessible format on a more timely basis.     And the PLT data are based on self-reported intentions, and intentions aren’t always what people end up doing.  Some people think they are leaving permanently, and are back six months later, and vice versa..   But intentions data isn’t nothing either  (just as business surveys capture intentions/expectations and things don’t always turn out as they expect).    The patterns –  and especially the cyclical patterns, the turning points –  in the PLT data tend to match those in the (lagged) 12/16 data quite closely.

There are quite enough gaps (and long lags) in New Zealand economic data as it is –  monthly CPIs, monthly manufacturing data, quarterly income measure of GDP just for starters –  that I’m just stagggered that key economic agencies are apparently willing to let SNZ/Customs go ahead and consider dropping departure (and arrival?) cards.  Where are Treasury and the Reserve Bank on this?

How, specifically, does it matter?   Without departure or arrival cards we would, of course, still have immigration approvals data for most non-citizens (other than Australians).  In principle, they could be published weekly or monthly with just a day or two’s lag, and be available in quite accessible formats.  Since approvals lead actual arrivals, there is certainly useful information in those approvals numbers (it is just that they aren’t made easily available now).

We could presumably also have data on the total number of people crossing the border (gross and net) from passport scanning.   I’m not aware that those numbers are published at present, but they could be.  And presumably they could be broken down by nationality (or at least by the passport the person happened to be travelling on).    That would be useful –  relative to having no arrivals or departures data –  but not very.   If you look at total net arrivals or departures (or net) data it is enormously volatile, and thrown around things like Lions tours –  in other words, holidaymaker and other short-term visitor numbers swamp movements of migrants.   Using that data alone, we’d have no ability to pick turning points for some considerable time after the turn had already happened.

The gaps would be particularly serious for the movement of New Zealanders, and more than half the variability in the 12/16 measure of net migration has arisen from fluctuations in the movements of New Zealanders.  We would have no secure way of knowing if someone leaving was planning to be off for a week’s holiday, or intending to stay away for ever.  The 12/16 method would eventually tell us what they did –  but there is a lag of almost 18 months on the availability of that information.    And even if the new plan involves keeping arrival cards and only getting rid of departure cards, most of the variability in New Zealanders’ migration movements is in the numbers leaving, not the numbers arriving.

Less importantly, without the departure cards we would seem likely to lose the ability to analyse migration (including reflows outwards by migrants who become NZ citizens) by the birthplace of the migrant.

Perhaps someone has done a robust cost-benefit analysis on getting rid of departure (and arrival?) cards.  If so, I would be keen to see it, and particularly keen to see how the relevant officials have factored in the loss of some of world’s best migration data to macroeconomic monitoring and forecasting, in a country with some of the most volatile immigration flows in the advanced world (and not a great track record of getting monetary policy, or housing markets, right as it is).  And even if one sets aside the macroeconomic analysts interests, it is not as if net migration numbers are one of those issues of no political salience at all.  Put an 18 month lag on decent data, and you risk not silencing debate – which some might wish for – but allowing all sorts of misconceptions and concerns to flourish, which no one will be in a position to allay.  It would, frankly, seem crazy.    Immigration has a economic and political salience here which it might not have in a country with land borders and small permanent inflows/outflows.

Frankly, it looks like a pretty irresponsible proposal.   The departure cards provide the only information on what New Zealanders are doing, and the comings and goings of New Zealanders are a big part of the PLT migration story (and aren’t, of course, under government control).

And in case anyone thought the PLT numbers were simply flaky measures, with no information

…here are the total net flows on the two measures [12/16 in blue, PLT in orange]

overall

They don’t match up perfectly –  one wouldn’t expect them to, and there is information even in the differences (eg what led people to change their plans) –  but no analyst would happily give up a series that provided a 17 month lead this (relatively) good on the 12/16 series.

Turning points matter a lot for macroeconomic analysis and monitoring, and the turning points in the two series are very similar.

The claim from Statistics New Zealand is that they can fill the gap with estimates that

will be generated through a probabilistic predictive model of traveller type (ie short-term traveller, or long-term migrant), based on available characteristics of travellers. Such a model will provide a provisional estimate of migration, which we can then revise (if required) as sufficient time passes for us to apply the outcomes-based measure.

I hope that they plan to rigorously evaluate the accuracy of such models, including when they’ve worked well and when they haven’t, and how well they capture the effects of policy changes, and that they expose their models and evaluation to external scrutiny before scrapping such a valuable source of hard data as the departure card.

And talking of data gaps, I’ve also written here before about the very long lags in MBIE making available, in readily usable form, the summary administrative data on actual immigration approvals (and estimates of the stock of migrants).   Some of the data you can get yourself, if you don’t mind manipulating spreadsheets that are hundreds of thousands of lines long, but for most people, for practical purposes, the data are really only available annually, and typically with quite a long lag.    That is really inexcusable.  Like it or not, immigration policy is a major instrument of government economic and social policy, and approvals data (and associated stock estimates) are a valuable part of informing the public debate.  Information is almost always better than no information.

[UPDATE: A reader highlights that not even the spreadsheets are currently available.]

MBIE publishes the summary results, and accessible tabular data, in their annual Migration Trends and Outlook publication. In many respects, it is a very useful publication, even if (a) the data are only annual (whereas, say, building approvals data are available monthly), and (b) the publication has a minimum lag of 4 to 5 months (in other words, data for the full year to June 2016 was only published in late November 2016).  That isn’t remotely good enough, especially for administrative data.  Neither MBIE nor SNZ has to collect the data –  it all sits in MBIE’s own systems, generated every single working day.  There is no obvious reason why the data  – all the summary data (number of approvals in each category, occupation, age, sex, country of origin etc) –  couldn’t be made accessibly available monthly within a few days of the end of each month.

I’ve made these criticisms previously.  And that was when Migration Trends and Outlook  was coming out on its normal slow timetable (a 5 to 17 month lag).   But go to the MBIE website looking for the 2016/17 publication –  in March 2018 –  and it still hasn’t been published, more than eight months after the end of the year in question, 20 months after the start of the period to which the data would releate.

Some readers might be inclined to suspect MBIE of some deliberate strategy to keep the information from the public.  I’m not.  That is not only because I’m not naturally a conspiracy theorist, and have had plenty experience of the failures of bureaucracy. It is also because a few months ago I was invited to a meeting by an MBIE official who was part of a team working on improving the Migration Trends and Outlook publication, looking for my comments/ideas on data, immigration research etc  The official seemed quite genuine, and enthusiastically told me of the efforts MBIE was putting in to improving the publications, and (if I recall rightly) the timeliness of the data (even while stressing that it was quite hard and there were “systemss issues”.  That meeting last year would have been before the usual publication data of the Migration Trends and Outlook publication, so I came away from the meeting quite encouraged.   I’m still quite willing to believe that MBIE has the project in hand, but in the meantime……where is the 2016/17 data?  It is now March 2018.

30 thoughts on “Please improve immigration data, not undermine it”

  1. With electronic passport scanning it can’t be too hard to add two questions. Where are you going? And will be permanent?

    Like

    Reply

    • It becomes hard when the definition of a Permanent and long term migrant includes international students, foreign workers, returning kiwis and also long stay tourists.

      Like

      Reply

  3. Here is the NZ Customs and Immigration departure card

    Click to access nz-departure-card.pdf

    This card is aimed at NZ residents and citizens – not tourists

    The bulk of the journeys and resulting departures are “visitors” or “tourists”

    The missing data is “if you are a departing visitor” what is your visa number – if you have one – but if you look up the number of exempt waiver countries don’t be too surprised. So that wont work

    The redundant data is the Flight Number and departure date which can be scanned from the bar code on the “passenger ticket flight sheet” which everybody gets – its used to automate the booking in process

    Considering we have Number Plate Recognition you would think we would be using Passport Recognition coupled with Blockchain + database technologies – then everything becomes redundant

    Talk about Border Control and Protection – check out the 200 Malaysians currently being deported from Auckland – they have been moving back-and-forth freely using different names and passports

    Liked by 1 person

    Reply

  4. As a lover of relational databases I used to download the .csv into MSSQL and play around with summaries. It was my data queries that resolved my concerns about elderly immigrants from countries with no Superannuation scheme (minimal in recent years) and also potential gender imbalances (minor even from strongly paternalistic cultures). And it was by chance I found the surprising imbalance between investors from ethnic Chinese countries and India. However recently when wanting to check the most recent figures to see if the change of government has made any difference to processing different categories this message is displayed:

    “”For privacy reasons the CSV files have been temporarily removed from the website. The decision follows concerns raised that some information previously published has breached individuals’ privacy. INZ has removed the files for the time being while we review how we provide this information in a way that ensures INZ meets its legal obligation to protect personal information.””
    https://www.immigration.govt.nz/about-us/research-and-statistics/statistics

    This puzzles me; obviously if you happen to know the only immigrant from Benin it is easy to work out
    what date his residency was approved. But that may still be in the .pdf that is supplied. Having last downloaded data to mid-2017 do any of your readers have any idea of what privacy has been breached?

    The government and especially MBIE should realise that it is when data is withheld from the public that rumours run rife. For example the many people I meet in Auckland North Shore convinced the parents reunion scheme is bringing in numerous Asians who are just looking for the NZ superannuation (worth about $400,000) and the various accommodation allowances (the category was arbitrarily frozen a couple of years ago). It is tough being Asian in North Shore – half the time you are blamed for being too poor and on benefits and the other half for being too wealthy and buying all the houses. When the data was available it was possible to prove that there was little reason to be concerned in recent years.

    Liked by 1 person

    Reply

      • Another possible use of privacy to protect bureaucrats is in a reply from Iain Lees-Galloway to a letter I wrote suggesting his department’s fraud detection department would be more effective if immigrants Visa approvals were matched with their IRD returns. My letter had been triggered by concerns raised by the recent successful prosecution for worker exploitation. Five immigrants in nearby Birkenhead were to quote the judge treated as ‘modern slaves’. The case came to light when one worker reported their troubles to his embassy who then informed the NZ government resulting in the labour inspectorate and police getting involved. However it could have been detected earlier by matching the workers visas with their IRD returns because wages of $16 per hour for 30 hours per week clearly meant they were not the skilled ‘chefs’ described on their visas.
        My suggestion was to matching with IRD data; this would eliminate unnecessary investigations and help target the INZ investigations. Of course no particular match would ever definitely mean fraud because the immigrant could be ill or on extended holiday or like myself retired but it probably would have caught the Chinese drug smugglers who were caught after residing in NZ for 9 years with no IRD returns.

        Mr Lees-Galloway replied very politely and promised a review of visa fees and immigration levies later this year and explained how “”INZ seeks to manage risk to the immigration system at the first possible point….”” via “”attempts to reduce the likelihood of fraud being committed or reaching NZ”” with relationships in partner countries and border activities. All quite reasonable but when an immigrant makes it through the system they seem to be generally forgotten by INZ but a sad subset turn up in Prof Stringer’s ‘Widespread Worker Exploitation’ report. Maybe it is hard to get INZ to investigate after approving a visa – that would only reveal any mistakes they have made. However most professionals do continually review the quality of their work; for example surgeons impartially review one another’s work with their aim to minimise repeating surgical mistakes.

        My suggestion was dismissed with “”However, I understand that there are broad protections for tax information provided to Inland Revenue in particular, which are focused on protecting NZ’s revenue base”.
        To be honest I cannot see how. Let IRD identify the odd looking data for immigrants, then salt the data with a few random immigrants and just return the list of names to INZ without any pay details for further investigation.

        Liked by 1 person

      • Utter Humbug from Lees-Galloway – he needs a new ENIAC

        This what is being done and been done over the road since 1990 – 27 years
        https://www.ato.gov.au/General/Building-confidence/In-detail/Data-matching/?page=7

        They have set up a Data-Matching-Agency (DMA) which shares data between ATO (IRD) Centerlink (WINZ) and Immigration and Border Patrol

        “We provide income information derived from tax returns to the Data-Matching Agency (DMA) – a separate agency within Centrelink – on a cyclical basis (up to nine cycles per year) on selected agency clients. This is used to determine the eligibility criteria for benefits and to help detect fraud within the welfare system. The data is provided by us under the provisions of the Data-Matching Program (Assistance and Tax) Act 1990”

        ATO
        Centrelink
        Child Support Registrar
        Department of Immigration and Border Protection

        Liked by 1 person

      • I wouldn’t criticise Mr Lees-Galloway; he may be unaware of what is available in Australia. I was impressed that he or his department took my suggestion seriously and gave a considered reply. I can imagine in many countries a layman would be too scared to write anything to a cabinet minister including the writers name and address.

        Database methodology is poorly understood but the concept is most satisfying so taking your typical ‘payroll’ system as an example; before a single line of code is written the database is designed so the unique employee number links to pay data and to employment history but the payroll staff are secured to only see the former and the HR staff only the latter. [Admittedly the database administrator has to be watched carefully].

        Privacy seems to be an easy answer rather than saying ‘cannot be bothered because it might make my department look bad’. Michael Reddell has mentioned it being used to block some of his Official Information Act requests.

        Thank you for the link.

        Like

      • Do a google search on the search term “nz data matching IRD” and trawl thru the SERPs

        Whoever wrote the reply for Lees-Galloway is inviting you to conclude that the privacy protections in place for the IRD are to protect NZ’s Revenue base.

        That is utter nonsense.
        IMO the privacies being put forward do the opposite – it hamstrings and cripples the revenue
        Trawl through those SERPs and see if you have been fobbed off

        ps: I worked for the IRD for a number of years

        Like

      • Getting depressed. I am not persuaded by the arguments that are pro high level immigration but it doesn’t bother me enough to read this blog let alone contribute comments to it. I am an immigrant in a suburb full of other immigrants and I rather enjoy the mix of people with different origins so immigration in itself is of minor concern to me (traffic, pollution, waiting times for hospital, loss of parks, opportunities for my children, etc). The reason I contribute is the sense of shock that I have eaten a meal in a local restaurant that paid staff $2 per hour and I believe also at another restaurant which employed the five so called Chefs who were treated like slaves (in the words of the judge). This is purely a predictable result of a badly designed immigration policy.
        William Wilberforce spent 20 years leading the anti-slavery movement before achieving success; my effort is minimal but I refuse to be silent when I can’t be assured my local eateries don’t have slave like conditions. So I sit writing this when I would prefer to be beer in hand watching the TV.
        Surely a Labour minister will see sense and institute the fairly simple procedures that would make his labour inspectorate more effective? Surely the honest businesses ought to be lobbying the government to remove competition from dis-honest businesses paying 3rd world wages? Surely the IRD want building workers to pay tax?
        A change of immigration rules is underway and hopefully will reduce the problem in future. But the data matching used in Australia as linked to by Iconoclast could be used to seriously assist in targeting the cheats already in NZ. Given the data volumes maybe 15 years ago the data mining would have been slow and expensive but now I could do it on my typical home PC.

        Like

      • It was only recently that the high skilled category has got a salary threshold. That salary threshold is now $50k. One of my recent migrant staff on a work permit after her studies has just been caught out as she was in the middle of an approval for her residency for some months now and have just been rejected as the goal posts just got changed. She has now asked for a pay increase to $50k from $40k. My local kiwi born boss will now lose $10k of his profits as we have to pay this new migrant $50k to try and keep her skills as she is a rather hardworking and clever worker but has some English language proficiency issues. Similar skilled Local kiwis to do that same job want $80k so at $50k I still can’t hire a local of a similar output and skill level.

        Unfortunately in the services sector, profitability is a function of lowering labour costs as we are already at peak productivity in terms of computerisation. That is until robots become better and costs less than the million dollar robots that can just barely open a door.

        Like

    • The only reason we have a optimum population of 2 million is because cows numbers 10 million which create a feeding frenzy and waste production equivalent to 200 million people. As we have reached a peak cow population we have no space for people.

      Like

      Reply

    • I am quite shocked when David Carter yesterday on the Nation declares that he signed the TPP because we have to be involved. The reason is that we must sell to those countries like our meat to Japan needs free trade access as Australia has a Free trade agreement that have seen their beef trade increase at our expense. He says NZ can’t sell anything else so we have to sell something in order to buy the stuff we need.

      We heavily subsidise our agricultural exports through, pollutants that these countries are not prepared to incur in their own countries. Primary production is unproductive as it takes 10 million cows to generate export GDP of $15 billion. It takes only 1.5 million people in Auckland to generate $75 billion in GDP. A highly subsidised Samsung company trades $350 billion with only 300k people.

      Our subsidies are just being put in the wrong industries. The Greens just need to articulate much better and they need a vision of what type of industries will be required to be subsidised and grow as an alternative to primary production.

      Like

      Reply

    • I spent 2 years working in Malaysia in the 1980s when Mahathir Mohammed first launched into his 2020 plans for Malaysia. At the time Malaysia was 100% plantation and mining based exports. These plantations, rubber, palm oil and mining Tin were its primary based production. Low skilled production similar to NZ of ttoday. It took a visionary in government leadership to decide to change and adopt heavy manufacturing and high tech manufacturing and to upskill its population. NZ has taken a reversed path giving up our high skilled jobs and relegated to low skilled primary production.

      Like

      Reply

  6. I’m right now looking at an Air NZ e-Ticket covering a planned return trip to USA, by a NZ Permanent Resident in June 2018. On the first page of the e-Ticket top-right-hand-corner is a Booking Reference and immediately below that is a QR-Code and below that is the legend:-

    IMPORTANT
    Customers require this document for check in, customs, airport security, immigration, and duty free purchases

    The body of the front page of the e-Ticket contains the flight details and Flight Number(s), and dates and times of the outward journey, destination or termination point, and same for return journey. There are 5 pages in all, 1 is a receipt for payment, 3 are all the usual contracting out of any responsibilities. The travellers passport number is not printed on the face of the e-ticket but embedded in the QR-Code

    Check out what a QR-Code is and think how much is contains
    https://en.wikipedia.org/wiki/QR_code

    When the traveller arrives at the airport and goes to the self-check-in kiosk they insert their passport which is then matched to the already held QR-Code and a card is printed which gives permission for the passenger to enter the customs hall

    Passports contains the holders full name, Date of Birth, Date of issue, Date of Expiry, and Nationality

    This an example of the digital sophistication of the airline industry and the scope of the data held and it hasn’t even been executed yet, but come the day of departure it will either be activated, or not

    One other issue is troublesome. Why are 2 government agencies on the case? SNZ+MBIE. That’s without considering Treasury and RBNZ

    Hopeless

    Like

    Reply

  7. Why go fast when you can go slow

    E-tickets in the airline industry were devised in or about 1994 and have now largely replaced the older multi-layered paper ticketing systems. Since 1 June 2008, it has been mandatory for IATA members to use e-ticketing

    https://en.wikipedia.org/wiki/Electronic_ticket

    addendum 1
    Above post – self-check-in kiosks issue a boarding pass

    addendum 2
    e-passports have been in use in NZ since about 2006. With the change to 5 year duration all NZ passports in use today must be e-passports

    A biometric passport (also known as an e-passport, ePassport or a digital passport) is a traditional passport that has an embedded electronic microprocessor chip which contains biometric information that can be used to authenticate the identity of passport holder

    addendum 3
    What possible system issues can MBIE be experiencing in this modern age

    Like

    Reply

  8. “In the near future, the outcomes-based ‘12/16-month rule’ is expected to become a key component in how we determine the number of migrants in New Zealand”

    What we do know is …
    Inbound travellers are digitally and automatically logged on arrival
    Outbound travellers are digitally and automatically logged on departure

    At midnight tonight it is possible to press a button to determine the NET movements for the day, whether it be more arrivals or more departures. The only missing data is the historical past data. If that process is repeated every day for 12 months the cumulative balance at the end of each day will represent the number of “migrants” in NZ at any one time. As time elapses the data becomes more accurate. Arrival and Departure of NZ passports are easily determined and isolated. The remainder are non-NZ’ers. What is not known is whether the movements are PLT’s and what the intentions are. Over time such data will be slightly distorted as some arrive as non-NZ’ers, acquire citizenship, and then depart on a NZ passport

    If the objective is, is to determine the number of migrants in NZ (at any given time), that can and should be done immediately. Of course it can be done. Why it isn’t being done is a different question altogether. That’s either political, or a question of indifference

    From your article it appears that an desire to move into the modern age is suspended for want of being able to extract from the data the “probable” intention of travellers

    Like

    Reply

    • Yes, total (net) movements data are easy to get, and will no doubt go on being reported. But those data are extremely noisy – because of tourist movements (and individual events like Lions tours). We also know that, as noted in my post, the reported intentions PLT data provide pretty good signals of turning points in actual PLT migration (captured with a 17 month lag by the 12/16 data). That is a classic example of a valuable piece of data for official and private analysts trying to make sense of the NZ economy.

      If the current PLT were a terrible indicator I’d have no qualms about dropping the departure cards. As it is, they are a pretty consistently good macro indicator, and we’ve seen nothing to suggest that SNZ”s ideas about model-based estimates will prove to be robust through time.

      Like

      Reply

  9. Auckland’s development or the madness of delayed building of infrastructure in Auckland is based on a plan that got the numbers wrong; they had a predicted range and it went higher. It takes so long to plan, design, consent and then build in Auckland that potentially adding another 16 months is a concern.

    Do we need to ask everyone? I thought there were formulas that would give the numbers needed to be interviewed in depth to give a 95% probability of future population? I suppose it is difficult to identify and contact the Kiwis abroad who may be considering returning home and that is maybe the biggest factor out of the control of INZ.

    Like

    Reply

    • Auckland Councils debt is tied to income which limits its ability to borrow to invest for population growth. The bulk of Auckland Councils Income comes from ratepayers. Existing ratepayers will not accept substantial rate increases to fund future but as yet non existent ratepayers. Therefore the funding mechanism for infrastructure is mainly dependent on the Government for future population growth. With the Government still heavily committed to Christchurch and Kaikoura disasters and EQC severely underfunded, the twin disasters weigh heavily on the Governments infrastructure spending books. This is still a $15 billion to $20 billion commitment over the next 10 to 15 years. The government basically cannot afford Auckland Infrastructure spending. It is a simple answer, there is no money available no matter how accurate your population statistics is.

      Like

      Reply

      • Most of the problems with accommodation and housing is mainly driven by the 4 million tourists at the moment and the associated workers needed to service them. This morning on RNZ 101.4, the Kaikoura businesses have a major problem with finding enough workers to service the sudden surge in tourists flocking to see the wonders of Kaikoura and there are not enough housing available for workers. Not enough workers just point to the new Labour Government tightening up on issuing those International Students work permits, I guess.

        Like

      • 4 million tourists. You make it sound a big number. But the impact on Auckland is the unplanned growth – so assuming tourism has doubled in the last 15 years and then allowing for a fraction of the tourists finding other parts of NZ better than Auckland (say 80%) and an average stay of say 2 weeks then you are saying our congestion is mainly due to 2m / 5 / 25 or 16,000 unexpected tourists at any given time. But how can they be unexpected when ATEED is spending so much money attracting them here?

        Like

      • Auckland airport handles 18 million inbound and outbound passengers every 12 months. 4 million tourists coming in but not to forget also domestic tourists and business travellers also come into Auckland from throughout NZ. Now does it sound a lot more?

        Like

Leave a comment