Why The UK's Government Data Service Takes an API-First Approach To Data.Gov.UK

In a blog post that's posted to data.gov.uk (where much of the UK government's data is posted for public consumption), Ross Jones has published some incredible insights into why API-first is so important to the agencies ongoing endeavors.

So, what is "API-first?" Well, depending on who you ask, you might get different definitions. For example, to some, "API-first" refers to a class of SaaS vendor like Stripe or Twilio whose only or primary offering is one or more APIs. When it comes to generating revenue their API comes first. Other ancillary services or offerings come after. These are said to be API-first companies. Another definition of API-first has to do with more of a microservices-like mandate that nothing -- no Web apps, no native/mobile apps, no reporting tools, no administrative tools,  and no third party developers -- get to touch atomic functionality or data without going through an API or layers of APIs first. APIs are esssentially the front-end to everything. For example, partly as an outcome for dealing with performance bottlenecks, Etsy realized that single-thread blocking architecture of PHP was holding back it's ability to scale. When it realized it could deal with scale by moving concurrency to the HTTP layer using a customized version of cURL, it also realized the implication that requests would have to be serviced by HTTP-based APIs. The company has since moved to an all-API or API-first framework to service everything from AJAX requests on Etsy.com to the data feeds that power internal analytics.

But in an API design context, API-first means something entirely different. When taking an API-first approach to designing APIs, you essentially lock yourself in a clean-room in a way that you ignore any legacy constraints (like your existing IT estate) that might otherwise inform your design. With an eye towards delighting developers and encouraging adoption, the basic idea is for your API's design to be influenced by what you or your API designer believes to be the ideal API to offer developers and not by existing factors. So, what if your existing software stack isn't up to the task of delivering that optimally designed API? You fix your stack instead of settling for an API with a mediocre design. API-first design can overlap with Etsy-like API-first infrastructure because of how bespoke APIs for servicing specific types of clients (Etsy learned this from Netflix) can arise.

The UK GDS' API-first approach falls more into the Etsy-like approach to infrastructure and application access though it should be noted that part of the Data Service's rationale had to do with consistency in API design, a very important issue to it. According to Jones, all requests -- be they from Web sites or from other sources -- go through the data.gov.uk data layer. This architecture is widely driven by the agency's choice to make CKAN a central component of its stack. Said Jones:

The CKAN framework means that any functionality added to the user interface automatically makes it available as an API.  This promotes designing, building and using APIs as the first task once you begin developing a system - not as an afterthought, which is quite common.

Regarding API-first design, Jones went on to write:

Adding an API to an existing system can often lead to APIs that are incomplete and hard to use - especially when they are not actively employed by the development team. By using our own API to build data.gov.uk we are forced into designing APIs from the very beginning, making sure that they are usable. It’s important that they make sense both with respect to data.gov.uk and to external users and systems.  

The agency is seeking feedback on its APIs and their features.

Original Article

API First at data.gov.uk

David Berlind is the editor-in-chief of ProgrammableWeb.com. You can reach him at david.berlind@programmableweb.com. Connect to David on Twitter at @dberlind or on LinkedIn, put him in a Google+ circle, or friend him on Facebook.