Can't live with or without data
If it weren't for Kim Dotcom, it would seem laughable that the United States would place much value on having a "no spying" agreement with New Zealand as part of both countries' involvement in the so-called "Five Eyes" agreement.
The agreement sees the US, United Kingdom, Canada, Australia and New Zealand maintaining a global intelligence-gathering network that shares what it finds and where the parties have a rather old-school agreement not to spy on one another.
Of course, spies being spies, they probably do spy on each other, but it's a convenient fiction to claim otherwise.
The US wasn't really supposed to spy on the German Government either, so when it was caught this year, there was diplomatic hell to pay.
In truth, the real problem was getting caught. Countries spy on each other. That's not a secret.
In fact, cyber-security experts will tell you the most active spies on the planet are Chinese and when they get caught, they barely bat an eyelid. Perhaps that's a more honest stance.
The net effect of all this skulduggery, not to mention the sorry record lately for too many government agencies to fumble their handling of private citizens' information, is an erosion of trust in the capacity of data-holding agencies to keep that data safe.
Yet at the same time, the capacity to collect data on the myriad activities of any citizen who carries a credit card or cellphone, pays tax or goes to the doctor is growing exponentially. Many people don't help themselves: hands up who doesn't have a password on their cellphone or laptop.
But they are unforgiving when a third party releases or misuses their private information.
Yet the capacity to use this information for good is obvious. It's astounding to see them access a five year-old CAT scan just by inputting your health system identifier.
Likewise, there is no public pushback against using big data to shut down child porn rings, the theft of banking information or thwarting terrorist cells.
However, where governments must tread more carefully is in seeking greater licence to combine their massive and disparate data-holdings to try to solve knotty social and economic problems.
This week's final report of the Government-appointed Data Futures Forum paints a future in which government data-sharing moves way beyond today's ability to match tax and social welfare records.
Initiatives it suggests include better public health interventions, natural resource management and urban planning, to name but a few.
"We think there are huge economic, social, environmental and personal opportunities for New Zealanders if we can strengthen trust, control and inclusion in the data-use eco-system," says the very first sentence of the report.
Noble intentions, but that's still a very big "if", especially as the trust and control so vital to such a future are undermined simultaneously by the "data-holic" activities of intelligence services, online platforms that harvest personal preferences for unsolicited marketing, and cyber-criminals, let alone benign but sometimes bumbling government agencies.
The report is intended to spark debate on the strategic, competitive advantage that a relatively trustworthy country like New Zealand could gain from making the cleverest possible use of its "big data" and of becoming a hosting place for others to pursue such activity securely.
Over at Finance Minister Bill English's office, you can almost hear lips smacking at the potential for savings on government spending in the health, welfare and justice systems if joined-up thinking using existing public sector databases can help solve big public policy problems.
Cynics will deride this as code for cost-cutting, but that's not the issue at all. The most important consideration will be assuring the public their information will remain confidential, that the integrity of databases will be maintained, and that information collected for the public good remains appropriately accessible.
If any of those are compromised, the vital social licence required to make such advances could easily be lost. In a week where private provider Talent2 threw in the towel after its disastrous implementation of a public sector payroll system, the public and private sector consequences of getting it wrong are obvious.
At its heart, the issues raised by the potential of big data to create both good and ill boil down to questions of trust. In proposing, for example, that the public sector should take a less risk-averse approach to using its databases or public good outcomes, it will be essential to ensure that the security systems underpinning such initiatives are not just robust, but demonstrably so.
Most importantly, individuals must be sure that however their data is used by the state to solve big problems, that data will be anonymised before it goes into the hands of any policy analyst and that only those who need to see the data are ever permitted to do so.
This, after all, was the lesson learnt the hard way by the US National Security Agency by Edward Snowden's mega-leak of highly sensitive documents. In retrospect, it's hard to imagine why a contract administrator based in Hawaii had any ability to access documents that had nothing to do with his work. Yet he did and the rest, as they say, is history.