The imperative need for machine learning in the public sector

The imperative need for machine learning in the public sector is growing. This is in part a result of the rapid innovation we are witnessing in the AI space, and increased demand for sophisticated decision-making support as technology solutions to public sector problems proliferate. But it is also caused by the challenges of public sector challenges, in areas such as:

Public sector organisations are under-resourced, and the public interest is at risk because of limited resources being directed to higher priority needs.

Public sector organisations are heavily regulated, with rules governing the procurement of technology that can restrict the choices available, and impede the timely implementation of best practice.

Governments are facing budgetary cuts. While there is a need for spending to improve public services, more stringent public sector budget constraints need to be addressed to free up funds for those who most need it, rather than continue this spending treadmill.

Technologies that could significantly improve people’s lives are still unavailable for them on a broad scale—this affects many areas of the world, including developing and developed nations, the well-off, or those within a specific demographic.

People want the public services they receive to improve—but they also want to control what they receive, how decisions are made, and how outcomes are monitored. And they want to decide for themselves about what they want to achieve, and how to best achieve it. We cannot let these demands, the growing impact of technology on people’s lives and the future of public services, and current and upcoming challenges for the public sector, go unnoticed.

To do so would be to risk being left behind the ‘digital haves and have-nots’ by their competitors. Or worse, to become collateral damage by a technology that seems to be solving everything.

So where do we stand?

Government needs to understand the implications of the current state of play and ensure the best possible outcomes. Government decision makers should be well-educated about machine learning (ML), and must understand where it is most needed and for what outcomes.

The public sector is generally averse to technology risk. A recent poll in the UK found that 76 per cent of senior IT decision makers have made at least one IT decision that has not worked out.

That’s not good enough. Public sector decision makers can make sure their decisions aren’t the wrong ones for the right reasons. Machine learning is increasingly used by public sector organisations to tackle such issues in a wide range of circumstances. Its ability to automate data analysis in particular has an enormous range of potential applications.

A better understanding of machine learning, and the wider data science space, is necessary to overcome current barriers to its use. The public sector must work with data scientists to ensure that their data is available and well-structured to facilitate its use. They should also provide training for data scientists, so they can start applying ML approaches using public sector data.

Government must also realise the potential for ML technologies to transform areas that are difficult to define or to solve. This is a big challenge, but the potential is there. Many industries are already using ML in their value chain to automate routine tasks or to improve processes. There are also emerging examples of ML at every stage of the life cycle of public sector services from conception to operation.

There are three main barriers to the use of machine learning in the public sector. These barriers need to be overcome if governments are to realise their potential for applying machine learning to improve the performance of the public sector.

The first barrier is a lack of skills and a culture of risk aversion

Public sector decision makers are understandably cautious when considering new technologies. It might be hard for them to fully see all the potential, especially when considering how it might affect their own jobs, budgets, data and intellectual property. The second barrier is insufficient access to data, as discussed in more detail below.

It’s also the case that there are many reasons for this caution, from the way public service providers are used to working, to the nature of the services they provide, the way work is organised and the way people behave. They also include the ‘risk-aversive’ culture of many public sector organisations. However, to overcome these barriers we need to challenge the status quo. This is easier said than done, but is the only way to ensure that governments’ best interests are fully served.

The public sector should be using machine learning. But the process takes time and patience to build capability.

There are some public sector decision makers who are very aware of the potential of machine learning. The good news is that such leaders are acting to change the culture in the public sector and build the capabilities they need to support the most important parts of the machine learning journey. They will need people in every organisation to support them; it is not just technology, the real challenge is ensuring the skills are in place to support the technology.

The current rate of change in AI makes it very hard for organisations to find the people to employ, to find the skills to employ and to find the money to train and equip those they want to employ. There’s a strong argument for government support to ensure that public sector leaders are well-placed to support the adoption of the skills (and capabilities) required to realise the value of this technology.

The last barrier is a lack of the required data

The public sector has vast amounts of data, but public sector data is not in a state of readiness that is easily integrated, analysed, communicated and analysed well enough to be useful.

Machine learning algorithms have a wide range of applications, and there are currently very few suitable datasets to test them in. Public sector organisations could use this as an opportunity to develop a digital asset for machine learning that would make their data machine-readable. This requires a long-term commitment and a high degree of organisational change, since changing the way a public sector organisation works is always a risk. But it would be a great investment, as this would allow better analysis of the data already in their possession, and also give them the data they need to develop new areas that can be automated or improved.

The public sector could do much more with AI-enabled data insights in the public realm

The public sector data that exists today is of variable quality and coverage. If it was better structured it would be more valuable to the government. However, there has been a tendency in the public sector to not get involved when it came to structured and clean data, even when given the opportunity to do so. If data organisations take the initiative and change their processes to work with a wider range of data, they can support the delivery of more accurate, timely insights. This, in turn, will help the government service or process they are supporting to improve.

For example, with accurate data they can focus more closely on delivering the right outcomes to the right people or the right places at the right time. It can help to automate processes that are currently manual, and reduce errors that occur at the interface between government and citizens. All these benefits can be made easier, more affordable and more accessible to citizens as a result.

The public sector can also take advantage of machine learning to drive innovation and collaboration in other areas. If policymakers are aware of how machine learning can help them to make effective policy decisions, they will be more willing to adopt the technology, as will the organisations that supply the raw data to public sector decision makers. This means that they can collaborate more on problem-solving and help to overcome common problems in the public sector.

For example, it is clear that the traditional methods used by the traditional financial service industry are less effective and more costly than the automated data-driven digital alternatives, making it more difficult for banks to stay relevant to the public in new areas like wealth management.

The public sector needs to embrace AI. One clear use case that could help the UK government is in the world of welfare and social care: in September 2018, the UK government proposed ‘Future Workplaces’ to transform the public sector. This includes an ambition to use AI to automate ‘as much of the human and manual activity that the public sector carries out.’

This goal is not simply about reducing staff numbers—it’s about re-imagining the relationship between the public sector and both citizens and business. In a world with a greater demand for access to services, as well as a more sophisticated consumer base, the public sector needs to take advantage of technology to provide better outcomes, more cost effectively, and in different ways.

There are now opportunities for business-government partnerships to help the public sector overcome the barriers to the adoption of machine learning. Many public sector organisations already have an appetite to learn more about the capabilities and applications of AI. But as the success of AI has become more readily apparent, the traditional barriers to working with business on this have started to fade. In this space, the public sector can begin to see this as an opportunity—and it should.

There is the potential to bring together the skills and capabilities already available in public sector organisations and those of the private sector to create new capabilities that are specific to the public sector and its services.

A public/private partnership would offer the best of both worlds—and a chance to transform the public sector into a much smarter, more effective organisation, one that is at the forefront of the development of AI approaches and an exemplar of their positive value to society.

Innovation, collaboration and better outcomes in a digital and data-driven service and data-driven economy

This is a very ambitious agenda. However, its outcomes are absolutely vital to society. The potential for machine learning in the public sector, and for the public sector to harness and use the capabilities that it has to offer, is huge. It is vital, therefore, that it is given the opportunity to flourish and deliver benefits to citizens.