Known for its use of data and API-enabled journalism and content provision, The Guardian is moving to a hypermedia-based API infrastructure. Sébastien Cevey, software developer at The Guardian, spoke at the API Strategy & Practice and APIDays Conference in Berlin about the value of hypermedia APIs and how he encourages developer teams across the publication company to consider using them in production.
Starting With REST Models
To bring everyone up to the same (single) page, Cevey commenced his talk with an outline of how REST APIs operate:
- First, you pick a baseline protocol like HTTP
- Then you model the things in the system as resources, with an URI
- Then you can map actions onto HTTP methods (GET, POST, PUT, PATCH, DELETE)
Cevey sees this model as demonstrating how hypermedia is “the key constraint of the Web and therefore of REST,” with hypermedia’s objective to achieve what REST creator Roy Fielding called the “simultaneous presentation of information and controls” so that the information itself helps the user make choices and take actions.
Introducing Hypermedia Control
Cevey said that for hypermedia functionality, “we can embed links to related resources and use forms to describe actions.”
For example, API-optimized media types (JSON or XML + hypermedia semantics) can be written in HAL, Siren, JSON-LD, Uber or Argo formats:
Cevey described the process of Hypermedia as the Engine of Application State (known by the unpronounceable abbreviation HATEOAS) in which GET is used to obtain a client resource from the server as a state, then the client can use hypermedia as affordances by following the links in the resource and have actions defined in forms in the resource. Finally, hypermedia allows end users to carry out those actions defined in the resource, and changes to the resource state are applied asynchronously.
Benefits of Hypermedia
Cevey shared his conclusions from having used hypermedia APIs in production for some years now at The Guardian. He points to key benefits in performance, discoverability, flexibility, simplicity and standardization with using hypermedia APIs.
Cevey says one of the key issues with APIs and Web-scale architecture on a network in general is that best performance is actually achieved when the network is not used, as that is often the slowest part of any application.
Hypermedia APIs allow greater caching of data, whether on the browser cache, through a content delivery network or even in a caching proxy on the server, thus avoiding reliance on the network server infrastructure altogether.
Cevey also briefly discussed remote procedure call (RPC) approaches. For example, at The Guardian, dynamic elements are often edge cached. This lets the popular articles section of the website be updated every 15 minutes (as content changes less frequently), while the comments sections of an article might be updated every minute (as it is much more dynamically used by the readership).
Hypermedia APIs enable applications to browse an API from the root or from any resource, just like how the Web works with hyperlinks. This requires API federation, for example, where media APIs are linked within the content API.
“You can have a microservice architecture and use a service index in the media API, and then applications can manage their own integration because they can discover everything themselves,” said Cevey.
He also said that he is taking this idea “one step further” by using a Dove client to enable accessing the API directly via a command line interface, a technique that is in proof-of-concept stage but has worked with Argo-formatted APIs.
Cevey spoke to the flexibility and evolvability of the hypermedia APIs he creates, noting that because persistent URIs are used, all configuration is done at the API root, so there is no URL construction that would be broken if changing the API.
Hypermedia APIs are also extensible, as more properties, links and forms can be added without breaking the API, and they will simply be ignored if unsupported by the application on the client side.
Cevey argued that hypermedia APIs enable scalability from bookmarklets to large, single-page apps.
Finally, Cevey talked up the benefits of hypermedia APIs in enabling some types of standardization in how API calls are made and the data they return. Links and forms are affordances in the hypermedia structure and can be driven by client-side validation. Business logic can be stored on the server side, and as server rules change, they can be applied to all clients.
This type of benefit is still being tested in research and development at The Guardian, Cevey said, but he pointed to a use case example where the client side would receive data about media types, such as whether they are valid for cropping. Then the client-side application can decide whether to display a cropping tool when showing the image, depending on the valid attribute in the media type.
Limitations of Using Hypermedia APIs
Cevey did acknowledge several shortcomings in using hypermedia APIs, but these are not overwhelming barriers to adoption.
One key argument against hypermedia APIs is that the serialization of JSON can be expensive, unlike a Thrift object. “The payload is bigger because you have to return bloated responses,” Cevey said. He noted that while “Argo+JSON over plain JSON might grow payload by 40%,” as more API actions are taken, less data needs to be transferred, so a call session might only actually increase payload by 15%.
The lack of a data schema available for hypermedia and weak tooling for hypermedia are also holding back greater developer adoption in Cevey’s opinion.
Cevey said that, for the most part, introducing hypermedia APIs to The Guardian often evolves from organic internal discussions and not really from philosophical fights (which is often the case in forums and blog posts, as summarized recently by Patricio Robles). The hypermedia APIs discussion often arises based on use cases and performance needs, giving Cevey an opportunity to share this presentation with various Guardian developer teams.
Cevey’s slides (from which the above images are taken) are available online.