Data discovery in action: Why lessons from Social Housing matter for all Departments

Much of our discovery work for the public sector involves investigating the role of data in broader policy, service design, or operational challenges. While the objectives of each project differ, we are usually trying to answer the same  questions:

• What do you want to achieve? 
• What do you need to know to do that?
• What data exists to create that knowledge, and is it good enough?
• If it’s not good enough, what should we do?

A great deal of complexity lies behind those simple questions. To answer them, we must understand how everyone involved in a process works together and how information moves between them. You cannot just look at the data in isolation. 

Here’s one example of how TPXimpact applied this in practice. For the Local Digital programme at MHCLG, we carried out a discovery project to look at how improved use of data could support the challenge of providing high-quality social housing.  Our approach was guided by the essential questions above. 

*The final report of the project has been published by MHCLG.

We defined the objectives of the study by identifying the two most common and important user journeys, then investigated them from end to end: “Find a home”, on the process of finding and applying for social housing, and “Fix a home”, around how repairs are reported and carried out.

Example user journey summary from the MHCLG Local Digital report.

At each stage of these user journeys, we examined the data and information stakeholders needed to achieve their objectives.  We also investigated the challenges for them at each stage of the data lifecycle:

• Capturing data: staff often have to re-enter information in multiple systems, or copy from paper into electronic forms. This leads to duplicated effort and inconsistencies. The lack of practical standard definitions reduced the reliability of the data.
• Accessing and using data at the right time: data sharing happens through individual contacts rather than systematically through governance and integrated systems. The lack of standard data models or APIs makes integrating systems difficult.
• Combining data sources: workflows often include manual data transfers and reprocessing. This makes it challenging and time-consuming to combine and compare data from different sources.
• Making strategic data-informed decisions: the challenges above make it difficult to get a clear view of what’s going on, to understand patterns or make predictions. More time is spent collating and cleaning data than analysing and responding to it.

The data lifecycle and barriers to success: from the MHCLG Local Digital report

We gathered examples of how these problems manifest in practice, as well as the organisational, technical and cultural barriers to change.

Based on the understanding gained and in consultation with stakeholders, we agreed on a vision for the future. In this vision, effective use of data within a housing provider helped them to understand the status and needs of their residents and their homes throughout the service journey. 

We identified specific interventions that would help make this vision a reality. These included creating a common data model to provide a single view of the resident and their home, and the API specification, data export and import requirements and taxonomy to implement the model.  

These specifications should be tech-agnostic. Different organisations might make different choices about software or cloud providers, but all should be able to follow a standard approach to ensure interoperability.

A key requirement for the adoption of these changes would be that the details of standards and how to implement them were co-designed with housing providers, suppliers and sector stakeholders. 

We examined the role of MHCLG as the responsible department in creating the circumstances and shaping the market for these interventions to succeed.

In this article, we’ve drawn heavily on one example in the housing sector. Still, the same principles apply across many public-sector challenges: where data comes from many different sources, it needs to be consistent, comparable, and made available to the people who need it to make informed decisions.  

The solution involves understanding the data lifecycle and its relationship to business processes, as well as the roles, objectives, and constraints of each participant in the overall ecosystem.