The National Data Library: A new plan for joined-up government data

In the public sector data community, we’ve been talking forever (it feels) about joined-up data for joined-up government, fixing the plumbing, data as infrastructure, and effective data sharing. But while you can point to various individual examples of progress, everyone still identifies it as a problem. 

The enduring challenge of joined-up government data

Our plumbing remains stubbornly unfixed, our data unjoined. It seems that it’s a harder problem than most of us have naïvely assumed. The latest iteration of the data initiative cycle is the National Data Library (NDL).  What’s going to be different this time?

The planets seem to be more aligned than before. Even if the NDL remains a vaguely defined concept, at least in public discussions, it has high-level support and prominence, with government committing to it in the spending review, the Industrial Strategy and, of course, in the 2024 Labour Manifesto.

Government is more data-aware than ever, and data and digital skills across government have been nurtured. The AI avalanche amplifies the importance of data and opportunities for innovation, while the concentration of programmes in the ‘digital centre of government’ in DSIT will hopefully build momentum.

The real barriers to data sharing

But the NDL and the public sector more broadly need to make sure they try to solve the right problem.

Technology alone won’t solve this. We’ve known for ages how to do the techy parts of data exchange and dissemination, including at a large scale. It’s happening all the time within focused communities such as weather forecasting, scientific research or financial trading.

The difference with government data is that the objectives of data users and the potential applications of the data are often far more diverse. 

It’s a highly complex organisational problem to ‘engineer’ the ecosystem of producers and users of public sector data, aligning objectives, responsibilities and incentives. Many previous initiatives have failed because stretched government departments have not had the resources to make high-quality data available, when the users of that data lie mainly outside the organisation.

The NDL as an enabler of data spaces

The key point to recognise is that there is no one-size-fits-all solution to this. However, the NDL has the opportunity to curate an environment where multiple data ‘markets’ can thrive.

The technical challenges will vary a lot depending on the kind of data; the legal and ethical challenges will vary with sector and with organisation type; incentives and capacities are vastly different for different organisational types.

There is much we could learn from the concept and practice of Data Spaces, defined by the Data Space Support Centre as ‘an infrastructure that enables data transactions between different data ecosystem parties based on the governance framework of that data space’.

Each data space has known participants, data products and rules (technical standards, business model, governance framework) that together define how the space operates.

In this approach, the NDL could be the enabler of a new collection of data spaces.  There are many ways this could be done. It could include offering high-level governance, guidance, patterns and re-usable toolkits. It could potentially include access to shared technical infrastructure or common services such as identity and access management. There is also the need for a centre of shared experience, a provider of initial funding or resources, even a promoter and cheerleader. 

Creating sustainable and fair data ecosystems

Using data to help address critical issues will create substantial value.  Collecting, managing, processing and analysing that data costs money. The art and science of a sustainable data space business model is to ensure a fair allocation of costs and benefits. There will be many opinions on what is fair (see Jeni Tennison’s policy brief “The National Data Library and Public Benefit”), but this needs to be addressed to create long-lasting successes.

It’s not going to be easy, but the NDL doesn’t have to solve everything at once. Start small and ‘test and learn’. While limiting the scope of early work, take ‘vertical’ slices, one or two high-value datasets with clear applications, perhaps aligned with one of the government’s five stated missions, and work to put all parts of the value chain in place.