APIs are the core mechanism for decoupling front ends from back ends and for decomposing monolithic infrastructures into composable enterprises in the spirit of what's known as digital transformation. They're currently the most significant enablers of innovation, mobility, and the Internet of Things (IoT). APIs enable teams to focus on their core value proposition while allowing customers to achieve bigger goals by connecting to data and functionality with tools they prefer to use. But to deliver on these myriad benefits and objectives, teams must design APIs with scale in mind. However, the need to build high-performing APIs that scale with the business ecosystem is pressuring many development teams to build APIs that may be restricting business growth.
APIs that are built without scale as a consideration will suffer from poor usability, have limited availability, are open to security issues, and more.
Here at Raygun, we recognize that a high-quality API is pivotal to our business growth, and cite scalability as a critical success factor. When we need to grow to meet customer demands, we must handle billions of data points comfortably and with minimum disruption. In our particular case, we're ingestion heavy, meaning we receive billions of requests a day and any delay to our APITrack this API's response would result in a potentially bad experience for our customers.
The issues we discuss here apply to both reading and writing data through APIs. We'll discuss how Raygun's development team manages the infrastructure and maintenance of our APIs to enable growth.
Why is a scalable API so important to software businesses?
Raygun drives better API-driven customer engagement through our use of SDKs. One of the goals of our SDK architecture is to be lightweight and have little to zero impact on the customer's application performance. While our API itself is a relatively straightforward endpoint, we found that providing SDKs means we can make API access easier, less error prone, and include niceties (e.g., if connectivity is interrupted, be sure to send the data later when it's restored).
As we accept large volumes of data from customers, managing our API effectively is critical to our business. If we can't receive data at volume, then our product would be useless. On average, we get thousands of requests per second to our API, with spikes into the hundreds of thousands per second, so we need to be able to handle a wide-ranging load.
Our product development is not all about the data handling, however. A great UI and nice features are what customers want on the front end, which isn't possible without a robust API.
As Uri Sarid, CTO of Mulesoft, articulates so well, "Much like a great UI is designed for optimal user experience, a great API is designed for optimal consumer experience."
For our survival as a company, our offering a large customer superior data management and a great experience on the front end is mission critical, and we must be able to scale to meet larger customer's needs.
So how do we do it? At Raygun, we look at two main areas when building a scalable API: infrastructure and maintenance.
Infrastructure of a scalable API
When you create an API, it exposes the business logic of your system to the outside world, which needs to be protected.
To build a scalable API, mitigating vulnerabilities should be the first port of call. As ProgrammableWeb's editor-in-chief David Berlind explains, attacks are becoming sophisticated, multi-dimensional, and hyper-targeted.
The sensitive data Raygun collects must be protected with a multi-layered system that extends beyond the infrastructure of the API. As cyberattacks evolve, the systems we put in place need to be dynamic enough to change and evolve with them. (You can read more about security in APIs here.)
Figure 1. Raygun's API security layers
To mitigate risks, Raygun uses several layers of security for our APIs. All calls are done with a customer's API key and authentication credentials.
A simple first layer is to offer a "regenerate authentication credentials " option. If you choose to re-generate your credentials, the original credentials are no longer valid.
The reason this is essential to protecting your system is to prevent anyone with malicious intent gaining access to your account. For example, if a developer accidentally checked your credentials into a public repository, you're safe because that key will no longer be valid.
After authenticating your credentials, we'll then generate a time-based token for subsequent API calls, expiring after 15 minutes.