Best Practices: How to Engage Developers with a World-Class API Portal

Continued from page 3.

3. The Top-Notch API Reference

At the heart of any portal is the API reference, which helps users understand how to use the API. Once developers are comfortable using your API and have begun working with it, the reference is where they'll go when they run into a specific issue.

It's important that we make a distinction here between an API reference and API documentation. Pronovix has discussed some of the differences between the two and how, at times, the phrases are used interchangeably. For our purposes, an API reference is the set of detailed instructions that includes all the resources available through the API, any parameters and their values, detailed requests and expected responses, header formats, and error/success codes. Many portals go well beyond this list, but this is a minimum standard for having an acceptable reference. API documentation may include content such as getting started guides, links to tooling, and a discussion about API security. The API reference is a necessary subset of the API documentation.

The Basics

To make your API reference truly great, you'll need to meet a few criteria. To start, it is imperative to make the reference an online web page(s) as opposed to a PDF document that the user must download (a mistake that we see all the time). Most APIs will undergo changes during their lifetime and eventually a static document is nearly guaranteed to contain out-of-date information.

The reference needs to be complete and includes all of the resources that are available through the API. To go a step further, including a full description of what each resource does is a good practice. The bank BBVA does a great job of this on its API portal, providing detailed notes on each resource's function and what the associated HTTP verbs (POST, GET, PATCH) do when applied to those resources. This is a nice feature that allows users to understand exactly what to expect when working with any endpoint.

Figure 12: BBVA's reference documents never assume a user has a complete understanding of a resource, so the bank includes notes that explain them.

Figure 12: BBVA's reference documents never assume a user has a complete understanding of a resource, so the bank includes notes that explain them.

However, what continues to surprise us is a large number of API references that fail to include complete URLs (usable in cURL requests) in their content. Many, such as the BBVA example above, only show the relative resource. Others, such as Twitter, do a better job and show a complete resource URL.

Figure 13: Twitter's developer portal shows the complete URL for usage in API calls.

Figure 13: Twitter's developer portal shows the complete URL for usage in API calls.

Let's take a moment to clarify the difference between endpoints and resources with regards to RESTful APIs. Many API references use the two terms synonymously which can lead to some confusion. The term endpoint is the base URL that developers use to address an API. In the Twitter example in Figure 13, the endpoint is https://api.twitter.com/1.1 (the base URL of version 1.1 of the Twitter API). Any call made to the API will have that endpoint as part of the request's full URL.

A resource is usually a collection of data, often represented by a noun, that can be acted upon by an API request. In Figure 13, the resource is /search/tweets.json where the API will return a collection of tweets that match any additional request parameters.

$ curl --request GET 
 --url 'https://api.twitter.com/1.1/search/tweets.json?q=from%3Atwitterdev&result_type=mixed&count=2' 
 --header 'authorization: OAuth oauth_consumer_key="consumer-key-for-app", 
 oauth_nonce="generated-nonce", oauth_signature="generated-signature", 
 oauth_signature_method="HMAC-SHA1", oauth_timestamp="generated-timestamp", 
 oauth_token="access-token-for-authed-user", oauth_version="1.0"'

Stripe, an online payment processing company, provides a best-in-class development experience by also showing the complete endpoint and resource URL as part of a full cURL call, and it also includes an API key. cURL is a language-independent way of calling an API directly from the command line interface (CLI) of an operating system like Linux. In addition to including sample API calls in the various programming languages that are best suited to the API, great API portals should always show what an API call looks like in the form of a cURL command. As shown in Stripe's developer portal, and the portals of other API providers, full cURL commands can be copied and pasted directly into a CLI and executed from there. With some API providers, you may have to change the API key and other parameters to the ones that were assigned to you. With Stripe, however, as presented in Figure 14, even if the user isn't logged in, the site automatically generates a working test key.

Figure 14: Stripe demonstrates an API call that uses a complete cURL request.

Figure 14: Stripe demonstrates an API call that uses a complete cURL request.

Let's take a look at another important, frequent omission from many API reference materials. As part of an API request, the header often includes authorization parameters that are used for all endpoints. Because of this, most providers will document header parameters in a separate section that covers authorization requirements. This seperate placement is a missed opportunity to provide a better developer experience (DX) by listing the header parameters on each resource. Who does it right? As shown in Figure 15 below, Postman is a leader in this criteria by showing full header parameters with each sample API call that's listed in the reference.

Figure 15: The Postman reference contains some of the most detailed requests and responses for each resource that we've seen, including full cURL calls and parameters for both the header and the endpoint.

Figure 15: The Postman reference contains some of the most detailed requests and responses for each resource that we've seen, including full cURL calls and parameters for both the header and the endpoint.

A good reference also describes all of the possible parameters for calling the API, to what resources those parameters are applicable, and whether any of them are required or not (to make a successful API call). These often include the parameter name, its type (string, integer, object, etc.), and a short description of the expected contents or a range of possible values (as shown in Figure 16 below).

Figure 16: GitHub's API reference does a very good job presenting the complete set of parameters by including the parameter name, type, and description.

Figure 16: GitHub's API reference does a very good job presenting the complete set of parameters by including the parameter name, type, and description.

Even better, as shown in the last row of the description column in Figure 16 above, if Github has deprecated certain parameters (which invariably happens with any API), the documentation of those parameters are marked as such with instructions on which parameter to use in its place. However, there is a caveat to this presentation. As there is no globally accepted definition for the term "deprecate" across the API economy, it's not clear whether the deprecated parameter ("line") has been deactivated to the point that it won't work if used (which might cause older applications to break), or if Github is suggesting that developers use the newer "position" parameter because the line parameter will lead to application dysfunction at some point in the future. Wherever possible, using words and phrases that are unmistakable in their meaning (eg: "deactivated" and "scheduled for deactivation on YYYY/MM/DD") is strongly encouraged.

The final piece required for a minimally viable reference document is a robust set of status codes that indicate the degree to which an API call succeeded or failed. Developers like to see that an API has a status response code framework and that it's explained within the reference. In the case of Web-based APIs that depend on HTTP, merely including some baked-in response codes due to HTTP being the underlying protocol is simply not enough. You must go to the trouble of designing and implementing a response code architecture. For example, in addition to an error glossary, Mailchimp does a simple, but great job with a page that explains how error handling works.

Figure 17: An example of one of Mailchimp's error handling explanations, which is a necessary addition to any API reference.

Figure 17: An example of one of Mailchimp's error handling explanations, which is a necessary addition to any API reference.

Mailchimp's glossary goes beyond listing the status code with a generic response to include details specific to the particular success or error. As an extra step, the company also explains a common reason for the error. If we look at error code 401 - APIKeyMissing, the glossary explains that a developer may have forgotten an API key from the request or formatted something improperly in the header. Then it links to more information on authenticating with the API. This makes for a very readable set of error codes that naturally leads developers to a solution.

Beyond the Basics

Having the basics in place means your reference is usable and satisfies the needs of most developers. Next, we list some criteria that will further improve your DX and even delight them. You need to show that using the reference adds great value to their consumption of the API while increasing their confidence in you as a reliable API provider.

One very useful feature of an API reference is when you present it as one page. This may seem solely like an aesthetic concern (and it works against the design principles for many other Web page types), but it has the practical advantage of making the reference fully searchable with a browser's built-in page-search functionality. A common presentation for a single-page reference is a three-column design. In this design, you'll show navigation links to the API's various resources in the left column (sometimes referred to as the "left-hand nav"), show the details for each resource in the middle column, and the right column shows example requests and responses, often with tabs for various languages. This is an elegant way to structure a reference that was pioneered by TripIt with what's known as the Slate design, an open source static documentation tool that allows you to write your API reference by hand using markdown.

Figure 18: Here we see an example of the Slate-inspired three-column design on Stripe's reference page. As mentioned above, this makes the entire reference easily searchable, contributing to how Stripe's DX truly shines.

Figure 18: Here we see an example of the Slate-inspired three-column design on Stripe's reference page. As mentioned above, this makes the entire reference easily searchable, contributing to how Stripe's DX truly shines.

Figure 19: While browsing Stripe's API reference documentation, the Ctrl-F/Cmd-F keystrokes present a search box for querying the entire reference and auto-completes entries with suggestions based on what the user starts to type. If the user prefers the browser's native search, then the user can type Ctrl-F/Cmd-F once more. This feature is a huge help when searching and the exact name of the method is unknown.

Figure 19: While browsing Stripe's API reference documentation, the Ctrl-F/Cmd-F keystrokes present a search box for querying the entire reference and auto-completes entries with suggestions based on what the user starts to type. If the user prefers the browser's native search, then the user can type Ctrl-F/Cmd-F once more. This feature is a huge help when searching and the exact name of the method is unknown.

Sometimes, it's the little pleasantries that add a nice touch to the DX. When developers are browsing an API reference with their web browser, they'll often have their development environment (eg: Eclipse, Sublime, Atom, etc.) or a command line open alongside it in another window. Therefore, having a button that lets them copy a snippet of code from the reference into their own coding environment is an important timesaver that seems unimportant at first, but can actually add up over time. As hard as it is to believe, we've seen developer portals with screenshots of code that cannot be copied to the clipboard, let alone pasted into another application.

Figure 20: In this API reference example from cloud storage company Box, the user interface offers sample API calls for cURL, Java, C# (.NET), Node.js and Python. In each case, one click of the Copy button is all it takes to copy the code to the developer's clipboard.

Figure 20: In this API reference example from cloud storage company Box, the user interface offers sample API calls for cURL, Java, C# (.NET), Node.js and Python. In each case, one click of the Copy button is all it takes to copy the code to the developer's clipboard.

A great feature that's both helpful and adds a personal touch is when a portal can recognize a registered user and dynamically adjust the reference accordingly. ProgrammableWeb has covered how weather and climate data company PlanetOS does this by adding the user's personal API key anywhere it's required as a parameter into the sample API calls found in its reference. Clearbit, provider of a marketing data engine, also does this, as shown in Figure 21. This is a handy function that quickly readies sample calls for cutting and pasting.

Figure 21: Clearbit recognizes registered users and tailors its reference to them. In this partial screenshot, we've obfuscated the personalized API key.

Figure 21: Clearbit recognizes registered users and tailors its reference to them. In this partial screenshot, we've obfuscated the personalized API key.

Navigating Different API Versions

Some providers will run multiple versions of their API concurrently. In these cases, having a version selector from within the reference page is a great idea because users can toggle back and forth easily to better understand the technical differences between the versions. Cloud-based scheduling platform Humanity's reference is one example of this.

Figure 22a: Users can select the version of Humanity's API they need to research.

Figure 22a: Users can select the version of Humanity's API they need to research.

Although we find Amazon's approach to be less appealing than an actual version navigator, a regular text link that points to the "legacy APIs" (as shown in the screenshot below) is better than nothing at all.

Figure 22b: Although Amazon doesn't have a version navigator for its Dash Replenishment APIs, it does offer a text link that allows developers to explore the older "legacy" version of the API. Similarly, the legacy version has a text link that points back to the "new API.

Figure 22b: Although Amazon doesn't have a version navigator for its Dash Replenishment APIs, it does offer a text link that allows developers to explore the older "legacy" version of the API. Similarly, the legacy version has a text link that points back to the "new API.

Discord, the maker of a communications app for gamers, handles versioning in a slightly different way than most API providers. If the version number is omitted from the request URL, the portal routes the request to the most current version. This is a nice touch to document in your API's reference and to include in your API's functionality. But, be forewarned; while Discord is very clever in what it has done, it requires some advanced thinking when it comes to designing APIs in a way that new versions are backwards compatible with their predecessors.

Figure 23: Discord's reference offers versions for older APIs, but defaults to the current one.

Figure 23: Discord's reference offers versions for older APIs, but defaults to the current one.

A great benefit for users is an easily downloadable API description in your reference (one that complies with one of the several open standard API description specifications such as the OpenAPI Specification aka OAS or RAML). An API description of this sort lets you describe your API in detail while also making it machine-consumable by other members of your ecosystem. This not only helps automate API-related processes (eg: auto-generating documentation, API test harnesses, client tooling, etc.) for you as the API provider, it lets API consumers, both humans and machines, keep up to date with any changes introduced into the API (provided you keep your descriptions up to date).

In the split screen below (Figure 24), we see two examples of downloadable descriptions files: one from Puppet Forge, a modules repository for Puppet IT automation software, and the other from FYB bitcoin exchange. Whereas Puppet Forge uses OAS to create machine and human-readable descriptions of its APIs, the API Blueprint description specification is FYB's choice. The choice of description format is a matter of personal taste (OAS is the most popular) — but what matters most is that both providers have chosen to include an easily located description that adheres to a standard description specification as a part of their API reference.

Figure 24: Puppet Forge (left) places its API description prominently on its reference page. FYB (right) also offers a download link to retrieve an API Blueprint-based description.

Figure 24: Puppet Forge (left) places its API description prominently on its reference page. FYB (right) also offers a download link to retrieve an API Blueprint-based description.

An API is bound to undergo changes over its lifetime, and these can include the replacement or elimination of resources and parameters; an act which many in the API economy refer to as deprecation. As the U.S. Energy Information Agency's (EIA) API has evolved over time, the deprecation of certain fields have been reflected in the sample output that the agency includes in its API reference. This is a great visual signal to developers as to the material changes from one version of an API to the next. However, the EIA could have gone one step further to indicate when exactly (ie: with which new version number, or on what date) those fields were deprecated. Even so, it's a nice touch that we rarely see.

Figure 25: To help developers understand how a current API version may differ from its predecessor, the EIA clearly labels deprecated items.

Figure 25: To help developers understand how a current API version may differ from its predecessor, the EIA clearly labels deprecated items.

Let's now turn to another example. Postman Collections are a group of pre-built API requests that are meant to be run in a specific sequence. Developers use these collections within the Postman client to demonstrate common use cases, such as authentication, cookies, customizing headers, etc. These collections can also simplify the process of testing and implementing APIs, especially for developers that are new to them.

In the context of an API reference, Postman Collections accomplish a couple of things. First, they allow you to build a simple use case for your API that demonstrates to developers how it can be used in the real world. Second, if you are unable to include interactivity into your API reference, Postman Collections offer you a simple way to let developers interact with your API outside of your portal.

Figure 26 below is an example from the Kentico Cloud API reference. A simple click of the Run in Postman button will open the collection in a developer's Postman client.

Figure 26: Below is an example from the Kentico Cloud API reference. A simple click of the Run in Postman button will open the collection in a developer's Postman client.

Figure 26: Below is an example from the Kentico Cloud API reference. A simple click of the Run in Postman button will open the collection in a developer's Postman client.

Interactivity

One of the most important features that you can add to a portal is the ability for the developer to interactively experiment with the API. Ideally, this would take place within the API reference. But it can also happen as an extension of the overall developer portal in the form of a standalone console, explorer, playground, or sandbox. No matter how you implement it, interactivity adds value to users of all levels by demonstrating the API's actual behaviors, thus increasing their understanding of it.

The terminology can sometimes be confusing, so here are a few definitions:

  • Consoles: A console provides users with a way to interact with the API so they can explore various resources, sometimes using live data. A console may allow them to compose requests with various optional parameters, view the server's responses, and perhaps export the corresponding cURL commands. Often called API explorers or playgrounds, the terms are used somewhat interchangeably by providers. Figure 28 below shows how Google has embedded a console into the third column of its API reference for the YouTube Data API.
  • Sandboxes: An API Sandbox is a protected environment where API consumers can run queries against sample data in a way that mimics the characteristics of the production environment. By doing this they receive realistic responses for APIs that reflect real system behavior. Uber's developers site provides a good example.
  • Playgrounds: As mentioned above, most providers use the terms playground, explorer and console interchangeably. Google is an example of a provider that offers a playground for a specific use case. The Google OAuth 2.0 playground is a web based application that lets users interactively play with various Google APIs to understand how they work. In order to make use of the APIs, OAuth based authentication is required and the playground helps developers to better understand Google's authentication workflow.

The YouTube Data API, previously covered by ProgrammableWeb, offers a good example of how an API provider can embed an interactive console directly into its API reference. Using the three-column layout discussed above, the right column is used to display the console, while the middle column contains the reference details for each resource, including the potential request and response parameters.

The console lets users play with the parameter values and see what the ensuing API request and response looks like. Because it uses live data, users are asked to authorize with OAuth and provide an API key.

This console integration has a couple of nice features worth highlighting. The first is how Google supplies a table of common use cases for the resource. When the user selects one of the predetermined use cases, sample parameter values are automatically loaded into the console, quickly giving the user a feel for how the API works.

Figure 27: The YouTube API portal carefully and extensively defines common use cases within the console.

Figure 27: The YouTube API portal carefully and extensively defines common use cases within the console.

The console can also be toggled into a fullscreen UI, so the developer has a better view of the request and response payloads, a button to copy the sample request, and the ability to see the corresponding source code in a few different languages.

While having a console integrated directly into the reference provides huge value to portal users, this isn't the only way to implement one. Many API providers choose to have a standalone console where the user can experiment with the API. This offers the advantages we mentioned with the YouTube console UI (a more complete view of request/response payloads, a button to copy the request, and the ability to see the request code in various languages). Keep in mind that when offering a separate console be sure to link to it in a meaningful way from your API reference and vice versa. Look at how application performance management company New Relic's reference does this:

Figure 28: Note how, from within its API reference, New Relic has contextual links that not only take the developer to its API Explorer, it preloads the explorer with the relevant API call (see figure 29, below).

Figure 28: Note how, from within its API reference, New Relic has contextual links that not only take the developer to its API Explorer, it preloads the explorer with the relevant API call (see figure 29, below).

Within the reference, each resource has a link that opens the console (dubbed the API Explorer by New Relic) with many of the required parameters preloaded. This saves the user from toggling back and forth between tabs and having to fill in the necessary parameters.

Figure 29: New Relic's console, already pre-populated with an API call based on the contextual link the developer clicked from within the API reference.

Figure 29: New Relic's console, already pre-populated with an API call based on the contextual link the developer clicked from within the API reference.

Uber's sandbox "provides development endpoints for testing the functionality of an application without making calls to the production Uber platform. All requests made to the sandbox environment will be ephemeral." In the sandbox environment, users make calls to the sandbox's base endpoint, which differs from the production endpoint. Users get the advantage of a safe application development and testing environment without harming their live accounts or impacting the provider's production service.

Figure 30: When it comes to APIs, a sandbox is where the developer can go play with an API without impacting production data. It's not a physical box however. As shown with Uber's sandbox, it's simply a non-production endpoint that responds as though real transactions are taking place (even though they aren't).

Figure 30: When it comes to APIs, a sandbox is where the developer can go play with an API without impacting production data. It's not a physical box however. As shown with Uber's sandbox, it's simply a non-production endpoint that responds as though real transactions are taking place (even though they aren't).

Continued on page 5.

 

Comments (0)