Accessing NHS Data

… in progress

Abstract

This project implements code to access live NHS data from a web page – mainly related to COVID-19 but the techniques apply to any data feed (api) available.

Implemented during March 2021


Background

I have been downloading data related to the COVID-19 pandemic on a daily basis, preparing my own graphs and interpretation, in order to provide an unbiased examination of the data quoted by politicians and the media. In the early days of the pandemic it was difficult to obtain data which was often released sporadically by health authorities and the public had to rely on figures quoted and interpreted by the Government. The Johns Hopkins University of Medicine, Coronavirus Resource Center emerged as a useful resource and has been gathering data from countries around the world since the outbreak of the pandemic. They made the data available to the public in an easily digestible form.

https://coronavirus.jhu.edu/

Other countries, however, record the data in different ways and in the UK, Health and Social Services are devolved matters for Wales, Scotland and Northern Ireland (who do their own thing related to COVID-19). This made it difficult to compare the outbreak in the UK with other countries on a like to like basis. For this reason I have concentrated on data produced and released related to England only.

Recently, I discovered that the data is available from a data feed which is also used to provide the Government COVID-19 dashboard available at

https://coronavirus.data.gov.uk/

This Government website provides a suite of Application Programming Interfaces (APIs) to make the data as open and reusable as possible and this project uses this suite to provide its own interpretation and presentation of the data.

Read an article about the dashboard at

Behind the scenes: Expanding the COVID-19 dashboard


Getting Started

Details of how to use the Government Dashboard API is available from the Developer Guide at

https://coronavirus.data.gov.uk/details/developers-guide

There is a Python toolkit provided for using Python to access the data but I opted to use Javascript to embed the data directly in a web page.

I found the documentation a little unclear but soon worked out how to use the data feed.

All API requests are over HTTPS and accessed from

https://api.coronavirus.data.gov.uk

The documentation refers to version V1 but I note that the utility tool to generate a request URL references V2

I found the documentation a little difficult to follow but determined the steps are:

– Create a URL to request the data that you want to access.

– The data is returned in JSON, CSV or XML formats.

A simple way to create the URL is to use the “Download Data” page where you select the options you require and the system will generate the necessary url

For example, selecting the options

will generate the url

https://api.coronavirus.data.gov.uk/v2/data?areaType=nation
&areaCode=E92000001
&metric=cumPeopleVaccinatedFirstDoseByVaccinationDate
&metric=cumPeopleVaccinatedSecondDoseByVaccinationDate
&format=json

Submitting the URL will generate the appropriate JSON file

{
"length":12,
"body":[
{"date":"2021-02-28","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":17179491,"cumPeopleVaccinatedSecondDoseByVaccinationDate":598345},
{"date":"2021-02-21","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":14960341,"cumPeopleVaccinatedSecondDoseByVaccinationDate":513311},
{"date":"2021-02-14","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":12909968,"cumPeopleVaccinatedSecondDoseByVaccinationDate":490752},
{"date":"2021-02-07","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":10600380,"cumPeopleVaccinatedSecondDoseByVaccinationDate":472582},
{"date":"2021-01-31","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":8193779,"cumPeopleVaccinatedSecondDoseByVaccinationDate":463263},
{"date":"2021-01-24","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":5989517,"cumPeopleVaccinatedSecondDoseByVaccinationDate":454648},
{"date":"2021-01-17","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":3738781,"cumPeopleVaccinatedSecondDoseByVaccinationDate":448000},
{"date":"2021-01-10","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":2127865,"cumPeopleVaccinatedSecondDoseByVaccinationDate":407397},
{"date":"2021-01-03","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":1172675,"cumPeopleVaccinatedSecondDoseByVaccinationDate":20660},
{"date":"2020-12-27","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":842638,"cumPeopleVaccinatedSecondDoseByVaccinationDate":0},
{"date":"2020-12-20","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":574829,"cumPeopleVaccinatedSecondDoseByVaccinationDate":0},
{"date":"2020-12-13","areaType":"nation","areaCode":"E92000001","areaName":"England","cumPeopleVaccinatedFirstDoseByVaccinationDate":55576,"cumPeopleVaccinatedSecondDoseByVaccinationDate":0}
]}


Access the data using Javascript

The previous section describes how the data is generated. This section explores how to access the data using Javascript in a a web page. Although there are numerous ways to access and use the data this project uses a very simple approach.

The data represents the number of vaccinations (dose 1 and dose 2) performed on a weekly basis since vaccinations started in December 2021.

The code will process the data returned in response to the request described earlier and generate a graph of the data and a table listing the values on a webpage:

I will be using a number of Javascript libraries to simplify the coding and other Javascript features including

jQuery

jQuery is a fast, small, and feature-rich JavaScript library. It makes things like HTML document traversal and manipulation, event handling, animation, and Ajax much simpler with an easy-to-use API that works across a multitude of browsers. With a combination of versatility and extensibility, jQuery has changed the way that millions of people write JavaScript.

more info at

https://jquery.com/

Bootstrap v 4.0

is used to create a responsive web page suitable for multiple devices and also to create a “pretty” table to display the data.

Quickly design and customize responsive mobile-first sites with Bootstrap, the worldโ€™s most popular front-end open source toolkit, featuring Sass variables and mixins, responsive grid system, extensive prebuilt components, and powerful JavaScript plugins.

more info at

https://getbootstrap.com/

chart.js

to create a graph from the data

It’s easy to get started with Chart.js. All that’s required is the script included in your page along with a single canvas node to render the chart.

more info at

https://www.chartjs.org/

JavaScript Fetch API

The JavaScript Fetch API interface allows a web browser to make HTTP requests to web servers.

useful information at

https://www.javascripttutorial.net/javascript-fetch-api/


Code

html (apiTest.html)

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="description" content="Dashboard Live">
  <meta name="author" content="Mick">
  <link rel="icon" type="image/png" href="./favicon.png"/>
    
  <link rel="stylesheet" 
  href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" 
  integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" 
  crossorigin="anonymous">

  <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
  <script src="https://cdn.jsdelivr.net/npm/chart.js@2.8.0"></script>	

  <title>gov.uk dashboard api test</title>

</head>
<body>
  <div class="container">
    <h1>UK Government Dashboard API tester</h1>
    <p>
      Using the <b>UK Government Dashboard COVID-19 API</b> 
      to access data on the number of vaccinations completed 
      weekly to date.
    </p>
    <canvas id="myChart1" width="400" height="200"></canvas>
    <div class="table1"></div>
  </div>	

<script src="apiTest.js"></script>


</body>

</html>

javascript (apiTest.js)

//
// api test(s)
//

$(document).ready(function () {
		
  const endpoint = ('https://api.coronavirus.data.gov.uk/v2/data' +
  '?areaType=nation' +
  '&areaCode=E92000001' +
  '&metric=cumPeopleVaccinatedFirstDoseByVaccinationDate' +
  '&metric=cumPeopleVaccinatedSecondDoseByVaccinationDate' +
  '&format=json');
    
  var myData;
  
  // LIVE VERSION			

  async function getLiveData() {
    try {
      let response = await fetch(endpoint);
      return await response.json();
    } catch (error) {
      console.log(error);
    }
  }

  async function processLiveData() 
  {
    var i, count; 
    myData = await getLiveData();

    count = myData.length;

    let dates = [];
    let dose1 = [];
    let dose2 = [];

    let weeklyDose1 = [];
    let weeklyDose2 = [];

    count = 0;
    for (const val of myData.body){
      dates[count] = val.date;
      dose1[count] = val.cumPeopleVaccinatedFirstDoseByVaccinationDate;
      dose2[count] = val.cumPeopleVaccinatedSecondDoseByVaccinationDate;
      count++;
    }

    dose1.reverse();
    dose2.reverse();

    weeklyDose1[0] = dose1[0];
    weeklyDose2[0] = dose2[0];

    for( i = 1; i < count; i++) {
      weeklyDose1[i] = dose1[i] - dose1[i-1];
      weeklyDose2[i] = dose2[i] - dose2[i-1];
    }

    weeklyDose1.reverse();
    weeklyDose2.reverse();
    
    vaccinationTable( dates, weeklyDose1, weeklyDose2);

    dates.reverse();
    weeklyDose1.reverse();
    weeklyDose2.reverse();


    // 1st chart 
    var ctx = document.getElementById('myChart1').getContext('2d');

    var barChartData = {
      labels: dates,

      datasets: [
      {
        label: '2nd Dose',
        backgroundColor: 'rgb(99, 132, 255)',
        data: weeklyDose2
      },
      {
        label: '1st Dose',
        backgroundColor: 'rgb(255, 99, 132)',
        data: weeklyDose1
      }
      ]
    };

    var chart = new Chart(ctx, {
      type: 'bar',
      data: barChartData,

      options: {
        title: {
          display: true,
          text: 'Weekly Vaccinations'
        },
        scales: {
          xAxes: [{
            stacked: true,
          }],
          yAxes: [{
            stacked: true
          }]
        }
      }
    });
  }

  function vaccinationTable( dates, doses1, doses2)
  {
    let html = "";
    let total = 0;
    let n = 0;
    let num = doses1.length;
    let totalDose1 = 0;
    let totalDose2 = 0;
    let totalDaily = 0;

    html = `${html}
    <div class="table-responsive">
    <table class="table">
    <thead>
    <tr>
    <th>#</th>
    <th>Date</th>
    <th>1st Dose</th>
    <th>2nd Dose</th>
    <th>Total Doses</th>
    </tr>
    </thead>
    <tbody>`;
  
    for( i = num-1,n = 1; i >= 0; i--, n++){
      totalDaily = doses1[i] + doses2[i];
      totalDose1 += doses1[i];
      totalDose2 += doses2[i];
      
      html = `${html}
      <tr>
      <td>${n}</td>
      <td>${dates[i]}</td>
      <td>${doses1[i].toLocaleString()}</td>
      <td>${doses2[i].toLocaleString()}</td>
      <td>${totalDaily.toLocaleString()}</td>
      </tr>`;
    }

    html = `${html}<tr>
    <td><b>Totals</b></td>
    <td></td>
    <td><b>${totalDose1.toLocaleString()}</b></td>
    <td><b>${totalDose2.toLocaleString()}</b></td>
    <td><b>${(totalDose1+totalDose2).toLocaleString()}</b></td>
    </tr>
    </tbody></table>
    </div>`;

    let table1 = document.querySelector('.table1');
    table1.innerHTML = html;
  }

  processLiveData();

});

Notes

html

defines the content of the the results page

the node myChart1 will be used to display a graph of the data

and node table1 will display a table generated from the data

javascript

line 007: the endpoint variable contains the url to request the required data from the API

line 018: the function getLiveData uses the Javascript fetch mechanism to submit the request for the data and is called from the main function processLiveData

line 027: is the main function processLiveData to obtain and process the data. The data is sorted as required and passed to the vaccinationTable function (line 111๐Ÿ™‚ to produce a table of the data. The graph from the data is also created within processLiveData