Big Data, Smaller Bureaucracy
By Rhema Vaithianathan
By 2040 Big Data should have shrunk the public sector beyond recognition.
The age of Big Data has come to consumers rapidly, reinventing how we shop, socialise, bank and get from A to B. And a Big Data revolution is slowly rolling through the public sector too.
If we do this right, in 20 years or so the Government ministries that currently employ thousands, and sprawl across Central Wellington should each fit into a small Thorndon villa.
The vast bureaucracies of Wellington serve two main purposes.
Firstly, they monitor. For example, ensuring that schools are open the requisite days, that police are on the beat and other taxpayer funded services are working as they are supposed to.
Secondly, they collect information about these services useful for planning and policy advice. This helps Government decide how best to spend our money. Is the existing service working? Should it get more, less or no funding? Is a new intervention needed, and how much funding should it get?
Fortunately, when it comes to monitoring and informing, Big Data can really deliver. So once our data is up to the task, these jobs won’t need to be done the old fashioned way by armies of civil servants.
The information and insights will be immediate, real time, bespoke and easy to compare over time. And, ideally, agreed by all to be perfectly apolitical!
Accountability and transparency of decision-making should take a leap forward.
For example, we have just put a call screening tool into a frontline service for Pittsburgh child protection services. The senior managers now have a day-by-day analysis of the calls coming in about allegations of child abuse. More importantly, this data is automatically scored as to severity, allowing the organisation to pin-point exactly where their system is neglecting high need children. What was previously a “black-box” is now transparent.
New Zealand’s Integrated Data Infrastructure (IDI) is a major step forward in the public sector Big Data revolution. Researchers can use anonymous linked data about New Zealanders to zero in on the factors that make bad outcomes more likely and to measure what effect existing programmes are having.
In the not too distant future, ministers will be able to find out at the press of a button what is happening at the frontline. But to be really useful, our future data infrastructure will have to go well beyond raw facts.
The Minister of Health or the Director General of Health may be interested in knowing how many patients are on the waiting list at Whangarei Hospital. But more likely they want to know how the list compares to last year, the lists of other hospitals and current Government targets. And what it is likely to look like in a month, a year or two years.
Ministers may want to find out how many vacant teaching positions there are in South Auckland this week, compared to a year ago. Or be alerted when waiting lists get too long, crime rates go up suddenly, or school rolls drop. Or search for communities that are ‘beating the odds’ for family violence and figure out how to replicate protective factors elsewhere.
The current public service model tends to report at set intervals on crime statistics and hospital waiting lists. Imagine if your bank can only send you a bank statement once a year. You want to buy those amazing new Spring sandals but don’t know if you can afford them. Of course you can ask for a special bank report. But then a long line of middle managers at the bank will relay your request until the person keeping tabs on your bank balances relays the details back up the line. Great, you can afford the sandals, but now it’s Winter!
Future Governments should be able to rely heavily on an amazing, integrated data platform that generates real time insights with speed and ease.
Downsizing of the public sector should follow, and this could be a huge windfall for our frontline services. Once the best and brightest minds of Wellington are freed up it would be fantastic to see them sharing their considerable sector expertise on the frontline, either as professionals like teachers and doctors or as coaches and mentors helping to maximise positive impact.
Careful planning of the data collection infrastructure, and development of world leading analytics are the priorities now, so we can all accept the accuracy and reliability of data and insights down the track.
If we build the right foundations now, by 2040 data analytics will be able to answer almost any question a Minister can think to ask.
This opinion piece first appeared in The Dominion Post on 18 October 2016.
Professor Rhema Vaithianathan is Co-Director of the Centre for Social Data Analytics in the School of Economics at AUT. She gave her inaugural professorial address “Social Policy in a Time of Big Data: Egalitarian utopia or Orwellian nightmare?” at AUT on Tuesday 18 October, 4.30pm. Watch the livestream here: https://livestream.com/accounts/5183627