API Godai: 5 Elements of an Amazing API

The Japanese have 5 essential elements of life, known as the Godai: Earth, Water, Fire, Wind and Void. They believe that everything in the universe is a complex combination of all these elements.

Through our firsthand experience and philosophy in building the ClickMeterTrack this API and RebrandlyTrack this API APIs, we’ve come to the conclusion that an amazing API requires its own kind of Godai: Design, Scalability, Reliability, Security and Documentation.

Using this complex mix of 5 elements, we’ve been able to create APIs that power over one billion global link redirects and tracking every month.

Building an API is not an easy task. It requires a good dose of experience in:

  • Designing something any developer can use with ease.
  • Making the right choices about what architecture to use (or if you’ll delegate that to a PaaS of your choice).
  • The tech stack (maturity and stability).
  • The various mandatory considerations on the security model.
  • What to release to the developers in terms of  documentation, SDKs and other tools to help them use your services and associated APIs.

Obviously nothing will likely be perfect at first, but with some iterations and proper versioning, an amazing API can be built.

Design

The design of the API is the first basic step:

How do we model our product concepts? What paradigm should we use? REST? GraphQL? Your decision will depend mostly on what entities you’re going to expose and how they’re connected.

A REST interface is very useful for simple entities with minimal connection and nesting, and promotes overall Create, Read, Update, and Delete (CRUD)-like operations. GraphQL is more for entities that are intertwined, and makes it easy-to-explore the graph (duh) connecting your entities together with easy-to-use semantics for filtering operations.

Overall, GraphQL is a good choice. However, if your model is simple, you can go with a basic CRUD REST interface: That’s what we’ve done, due to the simplicity of our model.

Scalability

Scalability is one of the elements that doesn’t often get considered first.

You don’t necessarily need to be focusing on scalability right at the beginning, as it requires a lot of architectural decisions and money. It’s often easier to just move it as a concern to a later stage when it’s more feasible. In other words, push out a prototype first and work on scalability afterwards.

Most of the design choices for scalability today are focused on using a microservice architecture as well as on the new serverless movement in which you can just write your code (and not much of it) and let someone else (the serverless providers like AWS Lambda and API Gateway) run your API logic.

Most of these decisions really depend on the availability of your tech resources. Building your own microservice architecture is cool, but it might not be what brings you value in relation to the resources needed to make it happen. Instead, most of the time, you’ll end up using a PaaS (like Heroku or Google Cloud Platform AppEngine).

Heroku is a PaaS, platform as a service. This means, you load your code while Heroku (or whichever PaaS provider you pick) takes care of the machines (at a higher price than using an IaaS where you manage your own instances).

For the Rebrandly API, the first prototype was developed as a part of our ClickMeter API. When we started developing it, we didn't feel it necessary to think about traffic and scalability for Rebrandly as a standalone API, as we felt these elements were just a drop in the ocean compared to what ClickMeter could already achieve. So we basically borrowed the stack resources that were dedicated to ClickMeter, to get Rebrandly through the prototype phase.

Only after this first stage did we give Rebrandly its own resources. When we rewrote the prototype into the standalone Rebrandly API that’s available today, only then did we start spending time thinking about how to scale the new service.

Once you get to a stage in your product development where you need more control over your architecture and how it scales, you’ll be able to work out many of your infrastructure decisions.

For example, our platform contains some complexities that didn’t allow us to use a PaaS (like Heroku) for everything. We had to focus on an IaaS like AWS that gave us enough freedom to deal with some complex subtleties, but that also had easy to manage PaaS-like parts for some standard Web facing components: So we chose AWS + BeanStalk + AutoScaling.

Security

Besides the security of the overall architecture (which you probably don’t fully control if you’re on a PaaS), the API(s) must be secured.

Aside from the obvious, like always using SSL and never storing sensitive data as plain text, you can also choose to outsource the authentication part to services like Mashape, Google, and AWS, thus shedding some responsibilities.

For authentication, you might also think of using OAuth instead of a simple API key, or use both. While the latter is easier to implement and maintain, it has been proven to be an exceptionally weak means of authentication for security purposes. In the long run, OAuth brings some added advantages by offering developers a widely understood and commonly used API security standard for integrating easily with your system.

We’ve chosen to rely on both. This way, users can generate some API keys for their favorite Apps (like Tweetbot), but we also offer a fully-fledged implementation of OAuth for tighter integration with the Rebrandly ecosystem.

Reliability

Monitoring the performance of your API(s) is crucial.

While most *aaS gives you a way to monitor what’s happening, it’s important to always use an external service (like Pingdom) to keep you in the know about the health of your API and the systems it depends on. This will help you monitor the behavior of your API(s) from various points of presence in the world so you get a clear idea of how your service is behaving.

Most *aaS offerings provide the ability to to auto-scale in response to workloads and response times. So, in the name of continuity and performance, it is worth taking a bit of time to properly configure such features, in order to be sure that spikes are handled without the need for human intervention.

Obviously be sure that your codebase is tested (at least in the core parts), because no amount of virtual or physical hosts will save you (and your bill) if your code doesn’t behave properly. For example, if you’re using long synchronous, pending computations in an async event loop platform a-la node.js or querying a SQL database on a large data set without indexes, etc, you’ll get some unintended outcomes if it’s not properly tested.

Do load tests and analyze the critical sections.

Documentation

Documentation is important for your team and for everyone else who wants to use your API(s). This is probably a good example of eating your own dog food: if your team has trouble understanding your own docs, imagine the difficulty that someone else outside your company will have.

Specs like Swagger, RAML, and API Blueprint can help when creating proper core documentation and can also be used (via one of many third party tools) to build SDK(s), create nifty interfaces and so on. Also, many 3rd party services use them as a neutral language to build on - For example, you can use a Swagger specification to generate a client SDK and a skeleton server-side implementation that’s compatible with your API(s) from which you can expand by adding validation/persistence and other layers.

Be sure to keep your documentation updated and in sync (something that can be quasi-automated or fully automated) and always publish the newest versions for your developer community.

Tools like Readme.io (which we use) are really useful for generating documentation and produce nice results.

Of course, every API has it’s own unique set of priorities, which you’ll need to decide for yourself. However, at some point along the product lifecycle, all five of these elements are going to play a part in its evolution.

If you’re developing an API and want it to be amazing, I recommend you keep all of these in mind.

Francesco Furiani Francesco Furiani, CTO at Rebrandly (https://www.rebrandly.com) Francesco is an expert in big-data management, custom url shorteners and API architecture. He has represented Rebrandly as a speaker at many tech events such the AWS conference.

Comments