Authentication and authorisation Data distribution Transforms Query engine Message gateway Subscription store manager

Technical

This section describes the technical details of the Discovery Data Service; the architecture that underpins the data service software, the components that make up the service, the testing and assurance processes, and the resources, technologies, and software that are used to host, develop, and support the service.

For details see:

Architecture

The following diagram explains the underlying architecture and the basic publication to subscription pathways that data must follow.

Publisher data is identified by:

  • Organisation - it is assumed that published data will be accompanied by an ODSOrganisation Data Service (NHS) code , or similar.
  • Software format and version - for example, EMIS CSV 5.6 or Adastra XLM 1.0.

Important: Existing data processing and sharing agreements are checked to validate that the DDS has permission to process data from that organisation and in that format/version, and then share that data with specified subscribers.

Components

The following diagram illustrates the various logical components that make up the Discovery Data Service:

APIs

Access the following Discovery APIs at:

Test - https://devgateway.discoverydataservice.net/eds-api/

Important: You must have a valid discovery username and password to obtain a valid authentication token. You must have a valid authentication token to call the secure APIs.

API Details Notes
Authentication Used to obtain a valid authentication token for calling subsequent APIs.  
Get FHIR resource types Returns a list of FHIR resource types, and associated (human readable) names, that Discovery currently recognises, stores, and supports.  
Get patients for NHS number Returns a FHIR bundle containing summaries for all patients known to discovery with the given NHS number.

See also:

FHIR profile resources

GitHub FHIR Profiles repository

Get resources for patients Returns a bundle of FHIR resources of the given types, for the given patients.
Get referenced resource Returns details of the requested resource reference such as doctors, locations, or organisations.
Get flag for patient Returns the current state of the given flag for the given patient, for example frailty.  
UPRN lookup Used to return a unique property reference number, or an unmatched response, from a patient address.  

Important: The APIs and algorithms will only use data, return resources, or identify patients that you have protocols, agreements, and access for.

If you want to test our APIs against your own software, register your interest.

Management tools

The Discovery Data Service hosts a number of web applications and web application APIs for managing the data service.

We have a skilled and experienced team developing tools and utilities to maximise the benefits of a single source of data.

The tools, which are in various stages of development and currently being prioritised in line with the use cases being presented to the East London Discovery Project, include:

Query & Reporting Tools

The Discovery Data Service hosts a number of web applications and web application APIs for record viewing and analytics/reporting purposes.

We have a skilled and experienced team developing tools and utilities to maximise the benefits of a single source of data.

The tools, which are in various stages of development and currently being prioritised in line with the use cases being presented to the East London Discovery Project, include:

Published data

Numbers indicate the number of GP practices that are sending live data.

Numbers and text in red * indicate data that is not yet live.

**ADT feed only.

***ADT feed plus daily file feed that includes CDECommissioning Data Extracts from Power Insight (Millennium data warehouse software) and CDSCommissioning Data Sets files.

1 Barts Health Trust

Current data sets

The following table shows the datasets that are currently published by different supplier systems:

  EMIS Web SystmOne Vision Adastra Cerner Millennium
Rio
Data sets
Primary care GP
Primary care community
Primary care GP
Primary care community
Primary care GP
Out of hours
A&E
Inpatient
Outpatient
ADT feed
Community
Allergies
Appointments
Care episodes
Diagnoses
Documents
Encounters/consultations
Family history
Follow ups/diary recall
Free text
Immunisations
Inpatient activity
Medication 1
Non-scheduled activity
Observations2
Outpatient activity
Patient demographics
Problems
Procedures
Questions & answers
Referrals
Templates
Test orders
Test results/
Pathology results

1non-coded.

2includes signs, symptoms, biological values, and pathology.

= Data sets available and published to the DDS

= Data sets available but not published to the DDS

= Not applicable

= Data sets coming soon

FHIR profile resources

The following list shows the FHIR profiles that we currently support:

See also:
Get FHIR profile resource type
GitHub FHIR Profiles repository

Data assurance & testing

The Discovery Data Service receives data from publisher organisations that could have:

  • Multiple IT systems that are responsible for capturing the data.
  • Multiple mechanisms of publishing the data to the DDS.
  • Several message or file formats.
  • Multiple content taxonomies or code schemes, some standard and some local, and some with no code schemes at all.

This many-to-many relationship between the technical data exchange formats and an individual patient record creates a significant challenge when we try to aggregate and link the data for direct care and secondary uses.

The approach that we have taken consists of testing each and every system, extract mechanism, format, taxonomy and code scheme independently against clinical or operational scenarios to make sure that the system is fit for purpose.

It should be noted that when data is moved from one system to another it always loses some data or some context; this is referred to as degrade. The objective of the assurance process is to prove that the mechanism involved in transferring the data is fit for purpose and that the data content is good enough for the subscriber use cases.

The starting point of the overall assurance process is a publisher data entry scenario, and the end point is a subscriber data usage scenario; test scenarios and test packs are derived from the overall specification, modified by the system extract capability, and narrowed to the scenario of interest. There is no requirement to test the entire data service at any point.

Integral to the Discovery Data Service is a comprehensive set of tools for monitoring all access and configuration changes on our cloud platform, we log everything a user does from sign in/sign out to any change to a server or configuration item to create a complete audit trail. Access rules are continually reviewed with only minimal permissions applied unless a change is scheduled to be made. All access is governed by the Clinical Effectiveness Group (CEG) Barts & the London Queen Mary University Information Security Policy.

Notifications are generated as data is received at any of our endpoints, this data is then processed into the service with audit logs generated along the way to ensure accuracy and consistency.

Our data testing and assurance team have developed an advanced audit log, which is implemented in every transform, meaning that we can view every transform result to make sure that all data published into the DDS is validated at every step in the process and to allow technical and clinical teams to validate and sign off the accuracy of the data feed.

Technologies

The Discovery Data Service uses the following resources, technology and software:

For technical and API queries, please contact info@discoverydataservice.org