Cookies on ons.gov.uk

Cookies are small files stored on your device when you visit a website. We use some essential cookies to make this website work.

We would like to set additional cookies to remember your settings and understand how you use the site. This helps us to improve our services.

View cookies

You have accepted all additional cookies. You have rejected all additional cookies. You can change your cookie preferences at any time.

Skip to main content

Blogs : Using the Integrated Data Service to shape policy

An icon representing data linkage

Government has big ambitions for improving people’s lives, whether through making public services more efficient and effective, raising economic productivity, or increasing people’s wellbeing and security.  

To do this government has an underutilised tool at its disposal, data. In particular, safely linked data, which can unlock the insights needed to shape policy and lead to better outcomes for citizens. This is why we have established the Integrated Data Service (IDS). 

But working with data has its challenges! Especially when we are integrating sensitive personal data from across government in a safe and secure way with privacy as our highest priority. It’s taken some time to get to the point where we have a compelling range of linkable data in the IDS, with over 100 datasets now available. 

Figure 1 shows the three core data indices – the demographic (people) index, the business index, and the address index, which in combination are opening up a new world of analysis that enables improved data-driven decision making.  

The three core data indices – the Demographic (people) Index, the Business Index, and the Address Index
Figure 1: The three core data indices – the Demographic (people) Index, the Address Index, and the Business Index

Underpinning this linkage and matching of data is a service we have developed and named the Reference Data Management Framework (RDMF). The RDMF allows analysts to easily link data in a standardised and secure way that allows us to protect individuals’ privacy and use datasets ethically and safely.  

It indexes each dataset once - people, business and address - creating an anonymised identifier. So, Bob Smith, who runs his own plumbing business in Derby, would be given an identifier (a meaningless set of characters) that protects his privacy; whilst Bob Smith who runs his own plumbing business in Swansea would be given a different identifier.  

This means there is no need for the personally identifiable data from each of those datasets to be brought together (a time-consuming process requiring elevated levels of security). The RDMF’s "linking first" principle has been endorsed by all major government departments. 

The IDS is already supporting a host of innovative analytic projects that are feeding into the government’s missions and informing the important policy questions to improve the lives of citizens. One example is how IDS is enabling linkages between health and labour market data, which allows exploration of the impact of health treatments on people’s ability to work. This analysis securely links data from NHS England on health treatments, from HM Revenue and Customs (HMRC) on employment and earnings, and from the Census on people’s demographics.  

Another of our current accredited research projects is using Census 2021 data linked with Nursing and Midwifery Council registration data to identify the proportion of registered nurses and midwives who were not working within the NHS at the time of the 2021 Census.  

The findings will be published and made available to stakeholders to provide evidence-based insights into the workforce, informing policy and strategic decisions to address nursing and midwifery shortages within the UK healthcare system. 

The data on IDS also contains a large and growing amount of linked business data to understand how businesses grow and interact. In particular, business survey data is being linked to HMRC’s administrative tax data – with Pay as You Earn (PAYE) data already on the system, trade in goods data being ingested and Corporation Tax and VAT data in the pipeline.  

Finally, the IDS is enabling new insights for national, regional, and local government, whether in the devolved nations, combined authorities, or local authorities. For example, we have a number of projects developing analytic partnerships with local areas – including work on labour markets and employment being undertaken by the Tees Valley Combined Authority; and on the operation of nighttime economies with the Manchester and West Midlands Combined Authorities to inform decisions that benefit local economies. 

There’s a lot more going on in the IDS as improved data sharing continues to be in the spotlight. We’ll update you on our developments over the coming months.  

Julian McCrae, Deputy Director (Strategy), Integrated Data Service