How to Transform an Air Traffic API to GeoJSON to Render on a Map

There are many services and APIs that provide valuable data to use in an application. Unfortunately, that data isn't always the complete dataset we need or it may have structural differences that reduce its utility. This project demonstrates how to retrieve data from an API for global flight tracking and then transform it into GeoJSON to store in HERE's XYZ Geospatial Storage API and then render it as a 3D map with Three.js.

Finished map showing air traffic in and out of various cities

Finished map showing air traffic in and out of various cities

Data Query

The ADS-B Exchange provides a public API for fetching unfiltered air traffic details. To get started — you'll need to install node.js and npm install the popular requests http-client library. The source code in this article assumes that you have node.js installed and that you know how to work with the text editor on your system to create and edit source code files that take advantage of that installation. With installation complete you can begin making queries to fetch data about the arriving and departing flights from a given airport with just a few lines of code like the following:

const request = require('request');
 
// KSFO, YSSY, RJTT, or KJFK are good examples
function queryAirport(airportCode) {
    let uri = 'https://public-api.adsbexchange.com/VirtualRadar/AircraftList.json';
    request(uri + '?fAirQ=' + airportCode, function(error, response, body) {
        console.log(response.body);
    });
}

The results of this query shown below have geocoordinates of latitude and longitude. This is useful along with the other metadata but isn't a standard geospatial format that other tools already understand.

{   ...,
    "acList": [
        {
            "Id": 5025667,
            "Rcvr": 2,
            "HasSig": false,
            "Icao": "4CAF83",
            "Bad": false,
            "Reg": "EI-GEP",
            "FSeen": "/Date(1551760599030)/",
            "TSecs": 1,
            "CMsgs": 1,
            "Alt": 34500,
            "GAlt": 34639,
            "InHg": 30.0590553,
            "AltT": 0,
            "Lat": 21.895935,
            "Long": -80.42921,
            "PosTime": 1551760599030,
            "Mlat": false,
            "Tisb": false,
            "TrkH": false,
            "Type": "B763",
            "Mdl": "Boeing 767 323ER/W",
            "Man": "Boeing",
            "CNum": "24040",
            "Op": "Blue Panorama Airlines",
            "OpIcao": "BPA",
            "Sqk": "",
            "VsiT": 0,
            "WTC": 3,
            "Species": 1,
            "Engines": "2",
            "EngType": 3,
            "EngMount": 0,
            "Mil": false,
            "Cou": "Ireland",
            "HasPic": false,
            "Interested": false,
            "FlightsCount": 0,
            "SpdTyp": 0,
            "CallSus": false,
            "ResetTrail": true,
            "TT": "a",
            "Trt": 2,
            "Year": "1988",
            "Cos": [
                21.895935,
                -80.42921,
                1551760599030,
                null
            ]},
            ...
}

This API is pretty useful as is, but in the next step, we should transform it into a standard like GeoJSON.

GeoJSON

You probably are familiar with the rules for JSON, but GeoJSON is a specification for how to represent geospatial objects so that any tools or libraries can semantically process it consistently. The IETF formalized it as a standard in RFC 7946 to define the core concepts of types of geometry (points, lines, polygons), how to represent coordinates, and how to include additional metadata or properties.

The JSON we get from the ADS-B exchange is not far off but requires a bit of a transformation. An npm install of the underscore library will give a helpful utility method to iterate over each item in the list as the following code section demonstrates:

const request = require('request');
const _ = require('underscore');
 
// KSFO, YSSY, RJTT, or KJFK are good examples
function queryAirport(airportCode) {
    let collection = {
        "type": "FeatureCollection",
        "features": []
    }
 
    let uri = 'https://public-api.adsbexchange.com/VirtualRadar/AircraftList.json';
    request(uri + '?fAirQ=' + airportCode, function(error, response, body) {
        var json = JSON.parse(response.body);
 
        _.each(json.acList, function(row, index) {
            // Only consider data with both lat,lon that is valid
            if (row.Lat & row.Long) {
                if (row.Lat > -90 && row.Lat < 90 &&
                    row.Long > -180 && row.Long < 180) {
 
                    collection.features.push({
                        "type": "Feature",
                        "properties": row,
                        "geometry": {
                            "type": "Point",
                            "coordinates: [row.Long, row.Lat]
                        }
                    });
                }
            }
          });
    });
 
    return collection;
}

This function queries the API endpoint, generates a feature for each record, and returns it as a feature collection.

Geospatial Storage

Websites like geojson.tools let you quickly visualize GeoJSON output like this on a map. This is good for a quality check or some quick validation, but it isn't the best way to integrate into an application.

HERE XYZ is a geospatial service where this data can be stored and queried via an API. The advantage of this is that I can modify the dataset, add additional airports, or make bounding box queries for subsets of the data based on the geospatial coordinates of a particular viewport in my application.

You can run npm install here to get access to a command line tool for working with HERE APIs more efficiently. It allows you to run commands such as the following to create a 'space' to hold GeoJSON datasets that allow you to easily run queries for tiled responses. You'll also need to configure your credentials by running here configure set. More verbose details can be found in the here cli tutorial for installing and configuring your credentials if you run into any trouble.

here xyz create —title 'aircraft locations'

With a space like this, I can start to add data for each airport. You may also want to use the async library which is available with an npm install async for helpful utility methods to make asynchronous calls with node.js.

var async = require('async');
 
function addDataToSpace(geojson) {
    async.eachLimit(geojson.features, 10, feature => {
        var posTime = new Date(feature.properties.PosTime);
 
        var tags = [
            "from-" + (feature.properties.From ? feature.properties.From.substring(0,4) : ""),
            "to-" + (feature.properties.To ? feature.properties.To.substring(0,4) : ""),
            "reg-" + (feature.properties.Reg ? feature.properties.Reg : ""),
            "month-" + (posTime.getMonth() + 1),
            "day-" + posTime.getDate(),
            "hour-" + posTime.getHours(),
        ]
 
 
        feature.properties["@ns:com:here:xyz"] = {
        "tags": tags
        }
 
        var options = {
            method: 'PUT',
            url: 'https://xyz.api.here.com/hub/spaces/' + config.spaceId + '/features',
            headers: {
                'Authorization': 'Bearer ' + config.token,
                'Content-Type': 'application/geo+json'
            },
            body: JSON.stringify(feature)
        }
 
        request(options, function(error, response, body) {
            if (error) {
                console.log(error)
            }
        })
        }, function() {
            console.log('done ' + geojson.features.length + ' features!')
    });
}

The two methods we've referenced can be packaged up in a workflow to query the ADS-B API, transform its results, then upload and store it in HERE XYZ. The reason we'd want to do this is so that we can query it in a meaningful way for rendering a map.

Rendering a Map

Three.js is a useful library for rendering 3D objects with JavaScript. There is a bit of boilerplate HTML and CSS to add for a project to style it the way you'd like, but initializing Three.js will look like this:

<script>
    var scene = new THREE.Scene();
    var camera = new THREE.PerspectiveCamera(75, window.innerWidth /
        window.innerHeight, 0.1, 1000);
 
    var renderer = new THREE.WebGLRenderer();
 
    // The events are to make XYZ API bbox queries for the viewport
    document.body.appendChild(renderer.domElement);
    var domEvents = new THREEx.DomEvents(camera, renderer.domElement);
 
    // add lights
 
    var world = new THREE.Object3D();
 
    var geometry = new THREE.SphereGeometry(1, 32, 32);
    var material = new THREE.MeshPhongMaterial({
        map: new THREE.TextureLoader().load('8081_earthmap4k.jpg'),
        bumpMap: new THREE.TextureLoader().load('8081_earthbump4k.jpg'),
        bumpScale: 0.007,
        specularMap: new THREE.TextureLoader().load('8081_earthspec4k.jpg'),
        specular: new THREE.Color(0x0E0E0E)
    });
 
    world.add(new THREE.Mesh(geometry, material));
    scene.add(world);
</script>

For brevity, I've left out some of the details and context of setting up an animated 3D globe like this. A link to the full source code is provided at the end if you want to reproduce the end result but this highlights the most important requirements.

With the basics out of the way, we can use the HERE XYZ API to fetch air traffic data back out and render the scene using only aircraft within view.

function fetchPlanes(airport) {
    var spaceID = '[HERE XYZ SpaceID]';
    var accessToken = '[HERE XYZ Access Token]';
    var p = fetch('https://xyz.api.here.com/hub/spaces/' + spaceID + '/search?access_token=' + accessToken + '&tags=to-' + airport).then(function(response) {
        return response.json()
    }).then(function(json) {
        var p2 = fetch('https://xyz.api.here.com/hub/spaces/' + spaceID + '/search?access_token=' + accessToken + '&tags=from-' + airport).then(function(response2) {
            return response2.json()
        }).then(function(json2) {
            //window.data = json;
            planeData = [...json.features,...json2.features];
            console.log("planeData number of records: " + planeData.length);
            planeData.forEach(a => {
                // tag to and from
                if (a.properties.To.indexOf(airport) !== -1) {
                    a.direction = 'to';
                } else {
                    a.direction = 'from';
                }
                // track which is most recent
                    if (!mostRecent.hasOwnProperty(a.properties.Call)
                        || mostRecent[a.properties.Call] < a.properties.PosTime) {
                        mostRecent[a.properties.Call] = a.properties.PosTime;
                    }
                });
                    addPlanes(planeData.filter(a => a.properties.Lat) // check to be sure we have coords
                        .filter(a => { return a.properties.PosTime === mostRecent[a.properties.Call];}) // only display most recent
                          , world);
                   });
                });
            }

The addPlanes function uses d3, a helpful library for working with data-driven documents for some of the math around 3D trajectories to place a point in the right location. The final rendering of a map looks like a globe with some geometry representing the aircraft flying between whichever airports are of interest.

First stage of demonstrating geometry being rendered as cones

First stage of demonstrating geometry being rendered as cones

A bit more styling and editing can get us to the map shown initially.

Wrapping Up

The full source code is available if you want to bundle this all up into a final application like the one pictured and then modify it to suit your purposes. There are many APIs from third-parties with very interesting data to explore, and sometimes all we need to do is a few minor transformations to allow the API to work cleanly with a variety of new tools and libraries for exploring the data.

 

Comments (0)