Creating a heat-map with D3

Here at JUST EAT we have regular 3-day hackathons, where we get the chance to work on whatever we believe will help empower consumers to love their takeaway experience. I’ve found they are a great opportunity to work in an area outside that of my current day-to-day work, and also to familiarise myself with new technologies. My first hackathon at JUST EAT was a great example of this. Along with others I spent the 3 days putting together a reporting dashboard using the following stack; most of which were new to me at the time:

  • Redshift – AWS’s data warehouse service
  • ServiceStack – framework for the API that queries our data in Redshift
  • Backbone – providing the framework for our client-side application which consumes the API
  • Rickshaw – graphing toolkit for displaying time-series (built on top of D3)
  • D3.js (Data-Driven-Documents) – JavaScript library for manipulating documents based on data

Seeing as we already had D3 in the mix, and having postcode dimensions in our data warehouse, I thought it’d be fairly simple to add a D3 choropleth (or heat-map) to our prototype to add a bit of visual flair. After reading the excellent Let’s Make a Map tutorial by Mike Bostock, I knew that the hardest part was going to be sourcing the postcode shape-files.

Sourcing the data

Shapefiles are the standard data format for describing geospatial information – shapefiles of UK postcodes are available to buy from the O/S – unfortunately outside of the budget for the prototype we were creating.
After much experimentation, and learning some new tools, I managed to cobble together some UK postcode area data. I found that there is an amazing array of open source geospatial tools – www.maptools.org is an excellent resource for these; the most useful of which was QGIS for visualising and editing geospatial data.
Once I had the shapefile, it was an easy process to simplify the geometries (hence reducing the file-size), and export to the TopoJSON format that D3 maps consume.
I’ve posted a full description of what was involved to create the TopoJSON at github.com/roblascelles/uk-postcode-map/wiki/Cobbling-together-UK-postcode-area-data

Displaying the data

Drawing the postcode map was simply a case of replacing the boundary data in the tutorial. After that, it was almost trivial to add a fill value for each postcode-shape based on external data. The following snippets are from github.com/roblascelles/uk-postcode-map/wiki/Displaying-the-data, which describe all the steps of how to display an example choropleth.
Assuming that we’ve already obtained a JavaScript associative array called “areadata” that holds the values we need to represent for each postcode – we can use D3 to create a function that will map each value to a colour like this:

   var color = d3.scale.quantize().range([
        "rgb(198,219,239)",
        "rgb(158,202,225)",
        "rgb(107,174,214)",
        "rgb(66,146,198)",
        "rgb(33,113,181)",
        "rgb(8,81,156)",
        "rgb(8,48,107)"]);
    color.domain(d3.extent(_.toArray(areadata)));

Note, the RGB values are a nice range of blues from learning-d3-part-7-choropleth-maps. Also, we’re using the excellent underscore.js library here to flatten-out our data into an array so D3 can use it.
Now we just need to pass the value in the color function for each postcode (the .style(“fill”.. section below)

   svg.selectAll(".postcode_area")
        .data(topojson.feature(uk, uk.objects['uk-postcode-area']).features)
        .enter().append("path")
        .attr("class", "postcode_area")
        .attr("d", path)
        .style("fill", function(d) {
            var value = areadata[d.id];
            return (value)? color(value) : "#AAA";
        });

As an example, let’s generate some test data based purely on the first character of the postcode:

   var areas=["AB", "AL", "B", "BA", "BB", "BD", "BH", "BL", "BN", "BR", "BS", "BT", "CA", "CB", "CF", "CH", "CM", "CO", "CR", "CT", "CV", "CW", "DA", "DD", "DE", "DG","DH", "DL", "DN", "DT", "DY", "E", "EC", "EH", "EN", "EX", "FK", "FY", "G", "GL", "GU", "HA", "HD", "HG", "HP", "HR", "HS", "HU", "HX", "IG", "IP", "IV", "KA", "KT", "KW", "KY", "L", "LA", "LD", "LE", "LL", "LN", "LS", "LU", "M", "ME", "MK", "ML", "N", "NE", "NG", "NN", "NP", "NR", "NW", "OL", "OX", "PA", "PE", "PH", "PL", "PO", "PR", "RG", "RH", "RM", "S", "SA", "SE", "SG", "SK", "SL", "SM", "SN", "SO", "SP", "SR", "SS", "ST", "SW", "SY", "TA", "TD", "TF", "TN", "TQ", "TR", "TS", "TW", "UB", "W", "WA", "WC", "WD", "WF", "WN", "WR", "WS", "WV", "YO", "ZE"];
    var areadata={};
    _.each(areas, function(a) {
        areadata[a]=a.charCodeAt(0);
    });

We can now see those character values, represented as colours on our choropleth:
uk-postcode-areas
We’re not going to win an information is beautiful award, but it’s much better than a table.
The actual map I finished with during the hackathon was obviously loading actual data from the service; I also added a zoom function from this click-to-zoom via transform example (was just a few extra lines of code). My next step is to display more details on the map as you zoom in – but that’s for a future hackathon.

Comments are closed.