Skip to content
  • Technology & data

Is the public sector doing enough with its data?


by Sarah Finch

Data is key to unlocking better services for citizens, if the right processes can be put in place.

Looking after the most vulnerable people in society is a fundamental aim of organisations such as local government, housing associations and citizens advice teams. With the right services, delivered in the right way, they can make a real difference to peoples' lives. But how do they identify — and engage with — those who need their help?

In order to deliver benefits in the most impactful way, frontline organisations must have an accurate picture of the people who require support, and what kind of support this should be. This relies on data.

In a complex benefits system, it can help to ensure that support is given where it is needed most and that everyone gets what they are entitled to. Data also enables authorities to take pre emptive action, intervening before situations deteriorate too far.

Yet unlocking the power of data in the public sector is a complex challenge, with many different kinds of factors to consider. Not least, the sensitive topic of data sharing and governance.

We don't understand because we don't have the data

For Deven Ghelani, founder of Policy in Practice — a social enterprise working with public sector institutions to help them analyse the impact of policy, identify and engage the people impacted, and track the effectiveness of interventions — data is key to delivering citizen-centred services. But older style approaches to data and static data sets have limited potential when it comes to improving the lives of disadvantaged people.

“A lot of what we don't understand about poverty is because we don't have the data,” he says. “Take something like the Family Resources Survey, which measures different markers for poverty across a few thousand households in the UK. It provides some information, but the sample is too small to really understand what is going on. It doesn't allow you to be operational, doesn't allow you to be local; or understand what is happening to separate sub groups. It's also pretty useless right now, during Covid-19, because things have changed so much in a short space of time.”

A variety of administrative data sets — such as those used to deliver benefits, for example — exist across local government, but their application is currently fairly limited. It is when data from different sources are combined, on the other hand, that organisations can unlock much more impactful opportunities.

“We work with administrative data from local authorities,” Deven says. “We pool data sets on low income families, linking them over time to create a single view of how different social policies affect people — both now and in the future. This enables organisations to proactively identify who needs what support, to engage people and to also track the impact of interventions.”

Putting appropriate data governance in place

Unlocking more value from local authority data might positively impact households, and help to inform policy, but it is not without its challenges. The majority of these, according to Deven, come not from technological barriers, but from the cultural and relational difficulties that emerge when trying to get different institutions to work together.

“The technical aspect of the work we do is quite mature,” he says, “and a lot of the technical problems have been solved. The challenges come more from that softer, cultural relationship with local authorities and the data governance side of things. You need to show you have thought through that governance well, convincing data controllers that this is something they are able to do and that they can trust us to work with them.”

So what might a typical data governance conversation look like?

“When you link or map data onto each other, you need to go through lengthy governance processes with the owners of that data,” Deven notes. “Say you want to track the relationship between poverty and children who end up in the care system — you need agreement from the heads of child services and revenues and benefits to make that happen.”

Is government too risk averse?

The likelihood of getting heads of institutions to co-operate on these kinds of initiatives — and of securing the long term administrative support such projects need to get off the ground - is arguably higher in local authorities than in central government. So it is handy, then, that this is where the delivery of most services is focused.

But could central government do more when it comes to connecting up and extracting benefits from data?

“Central government focuses on the risks of sharing data,” Deven says, “but it doesn't focus on the risks of not sharing data. If some of the data assets the public sector has, such as DWP data, for example, could be shared — if trusted government departments or local authorities were allowed to work with it... The value of sharing this data, and the impact it could have would be huge. Imagine if people were automatically paid their share of the £16bn of benefits that go unclaimed each year, it would have a measurable, tangible impact on peoples' lives.”

A siloed mentality

The capacity of government to use data in a much more connected way shows promise, but again, cultural and attitude changes arguably stand in the way. One of the most challenging barriers is a fairly siloed approach to the way central government departments operate and a longstanding division between policy and digital delivery.

Does Deven foresee these gaps being bridged at some point in the future?

“DWP for example is making good use of Universal Credit data, and will continue to do so,” he says. “But I think there's a limitation, in that they will make good use of Universal Credit data for their own purposes. Which are actually quite narrow. There is a siloed mentality — and I'm not attaching blame or criticism there — which undermines the immense value of that data across the wider public sector.”

The potential of a more connected approach to data is easy to see when it comes to tackling something like poverty, which touches on the work of every government department. If these departments were better able to understand and measure its impact, it would inform work across everything from children's educational outcomes, to health inequalities, the marginalisation of certain social groups, and even how we rebuild after the pandemic.

Creating the digital citizen

This then is really the point when it comes to data. If a particular data set can be used to solve a particular problem, then it has some value. But if data can be shared more widely amongst those in the public sector who are using it for complex and interrelated purposes, then we can create a much deeper understanding of what is going on, of how people are falling through the gaps in our society, and of how to help them. This not only improves the lives of disadvantaged individuals but also helps to shape policy at a much broader level.

It is something that government, according to Deven, doesn't currently have the right mindset for.

“Policy in Whitehall tends to think about the impact of one policy on one set of people — it's really a siloed approach,” he says. “Whereas we want to think about the way all policies impact on all people.”

“We see this as the future of social policy. The commercial sector is used to working with data at an individual level, we share our data with them to get better services. We want this for the public sector too.”

A data-driven and connected public service should work better for everyone, as it starts from a basis of understanding. Understanding the issues people face, designing interventions accordingly, and measuring the impact of this activity. There might be barriers to overcome, but it's certainly an admirable goal.

Sarah Finch's avatar

Sarah Finch


Contact Sarah

Our recent insights


Common misunderstandings about LLMs within Data and Analytics

GenAI and LLMs have their benefits, but understanding their limitations and the importance of people is key to their success.

Shaping product and service teams

How cultivating product and service teams to support the needs of the entire product lifecycle can ensure brilliant delivery.

Building ‘The Chatbot’ - Our experience with GenAI

Learn how we harnessed to power of Generative AI to build our very own chatbot.