This part of the API Testing Series looks at API Metrics, an API monitoring tool. API Metrics has two main functions: API monitoring, for taking the pulse of your API product and user experience; and SLA measurement to monitor the performance of third-party APIs that integrate with your product.
In this installment of our API Testing Series we'll review API Metrics, a second tool that (like API Science) falls into the category of API monitoring. API Metrics has two main functions: API monitoring and SLA measurement. One solution is used in development and production environments to take the pulse of your API product and user experience. The other, SLA Measurement, is designed to monitor the performance of third-party APIs that integrate with your product.
Let's take a deeper look.
API Monitoring is a sophisticated tool to replace homegrown methods, like a ping or cURL call, to see if an API is alive and responding. Rather than calls that generate a generic response (something that yields a simple "200 OK"), API Metrics makes real functional calls to an API endpoint. These calls include authentication, JSON bodies, and also checks for responses. Agents can be installed in several different cloud platforms (such as AWS, Softlayer, Google, and Azure) as well as devices and IoT services. Information from each platform and API are brought into one unified reporting dashboard to help you observe regional and global trends in your API performance. Security support is built into API Monitoring. This product supports multi-OAuth services such as JWT (JSON Web Tokens) and JWS (JSON Web Signature).
Production monitoring systems are essential for companies that are aiming for DevOps or Continuous Delivery. In these contexts, software is often developed in a complex tangle of feature branches. Each change has some sort of test automation at the unit, API, or UI layer. Once a build is green, the code is pushed to production, which might occur once or hundreds of times every day. The process involves a large number of changes daily, despite there being fewer eyes on the finished product as companies prune their IT teams to the bare bones these days. Monitoring a production API will help you discover problems quickly when they slip through, helping you reduce customer exposure to software failures.
Figure 1. For a deep look at how your APIs are working, API Metrics offers a clear and simple presentation. Here are Google+ activity search latency and search response size graphs.
Figure 2. One important measure of API performance is latency, or the amount of time it takes for an endpoint to respond when called. In this image, you can see the Google+ Search latency graph detail as displayed in API Metrics.
Figure 3. Services that run on cloud platforms can occasionally have poor performance, or fail altogether in one region, but be fine in others. Here you can see API Metrics monitoring an API call from AWS South America East cloud.
Software products are rarely independent. They depend on other software for data and services. Any interruption in those dependencies can mean downtime for your customers. API Metrics has created a scoring system called the Cloud API Service Consistency Score (CASC). This score measures API performance from multiple places in the world. While the score on its own may not tell you much, over time you can see trends in the performance of an individual API, or compare the performance of multiple APIs against each other. This information allows you to ask educated questions and make decisions about which third-party APIs you want to integrate into your product. The scores you see are configurable based on your Service Level Agreement (SLA) with each third-party product. Here is an example of external API monitoring minus the CASC scoring system.
Figure 4. Service Level Agreements, or SLAs, are contractual agreements between a service customer and a vendor. The latency SLA reporting configuration seen here can help alert teams when their service is outside of the SLA specification.
Figure 5. Many products are dependent on public APIs such as GitHub, Facebook, or Heroku. Parts of your service may appear to be dead when these services go down. Here you can see the API Metrics public test comparison report, which displays information about various popular public APIs.
Most commercial software contracts come with an SLA. This will usually include uptime, how long an endpoint takes to respond, and how often there are failures. Configurable SLA monitoring ensures you'll know when a contract has been breached. For example, if you have an uptime requirement of 99.9% with GitHub, and it is down for three days in a month, SLA Monitor will throw an alert and notify someone in your development staff. Then you can renegotiate or reconsider contracts based on actual uptime.
Pricing and Packages
API Metrics is a commercial product with four different pricing options based on the number of calls made per day, cloud configurations, number of licensed users, and access to support.
- Tall is $75 per month and will accommodate 2,500 API calls each day.
- Grande is $200 per month and gives access to 10,000 API calls per day and access to chat support.
- Venti is $400 per month for 30,000 API calls per day and phone support access.
- Enterprise packages begin at $900 per month for 50,000 API calls per day (or more depending on pricing), custom cloud setup, five licensed users, and a dedicated support team.