Why API Gateways Still Matter in Service Mesh Deployments

Most of the organizations have become brave enough and spend a lot of time on R&D to convert their underlying business infrastructure to match with the next generation. Most of the time people are focusing on totally revamping their architecture by choosing to go with a microservices model, leaving behind their monolithic architecture. Even though this has been the trend, organizations find it difficult to take this leap, as it would require some specific set of knowledge and technologies, and their architects and IT employees need time to become experts of these novel ideas.

Having expertise in implementing Microservice architecture (MSA) and studying different technologies that provide the capability gives you another set of problems when actually trying to design your architecture. I will try to discuss a few of these problems here. First I will try to explain the transition from monolith to microservices.


In the most common cases of monolithic architecture, there will be a single application that provides the business capability, and any Scaling of that application can be done as a whole based on the demand. When certain types of governance are required and the business functionality can be abstracted with APIs (as the organization moves to modernize its legacy IT), API gateways can take over the role of requirements like security, authorization, Rate Limiting, and transformations from the different business components.

Transform to Microservices

One of the initial steps in the journey of moving towards an MSA is to break down the monolith application into smaller services each of which handles specific business logic. The services are loosely coupled with each other and communicate with each other using typical API calls over the network based on HTTP, GRPC, etc.

But microservice sprawl can lead to another problem; their management and governance. The challenges include but are not limited to:

  1. Monitoring the microservices for their health, metrics and logs
  2. Hot deploying a microservice with a bug fix (so as not to disrupt the other services that depend on it)
  3. Adding new microservices and handling the routing for them
  4. Securing service to service communication and handling security in a standard manner across all services
  5. Managing the traffic with circuit breaking, timeouts and rate limiting
  6. Adding new policies to govern the microservices and manage those policies

One proven way of dealing with these and other challenges raised by microservice expansion is to build a Service Mesh by adopting a service mesh solution provider. 

A service mesh is a network of microservices that, when taken together, form the basis of a  composable  application. A service mesh provides infrastructure to handle the interaction between those microservices. It consists of a data plane which is the mesh of microservices and a control plane that governs and manages the data plane.  A service mesh injects a proxy as a sidecar to each microservice. This sidecar proxy governs how microservices communicate with each other and how the control plane communicates with the sidecar proxy in order to manage the data plane. 

Deploying a service mesh adds a new set of questions at the architectural level. The main problem is where do I put my good old API gateway in the service mesh or if I even need one. Do API gateways and service meshes solve the same set of problems. For example:

  • A service mesh provides secure service to service communication. Do I need API gateway security any more?
  • A service mesh provides request Authentication and identifies the client. Do I still need the API gateway for end-user authentication?
  • A service mesh provides circuit breaking, timeouts and pluggable policies. Do I need the same features from the API gateway?
  • A service mesh also handles the traffic routing within the mesh. Why do I need API gateway routing anymore?
  • A service mesh provides metrics and logging about the included microservices. Why do I need an API gateway for monitoring and analytics?

On the surface, it appears as though API gateways and service meshes solve the same problem and are therefore redundant. Actually they do solve the same problems, but in two completely different contexts. Mainly an API gateway acts in the edge of the deployment facing the external clients, handling north south traffic and service mesh manages the traffic among the different microservices which is the east west traffic.

Let's try to break down the above statement under different facts.

1. APIs vs Microservices

The basic idea of a service mesh is to provide an ecosystem to manage the microservices. Ultimately however, the end-to-end business functionality (eg: the workflow to place an order) is a set of interfaces defined as APIs for external parties and developers to consume. In other words, the API is an abstraction of a workflow consisting of multiple microservices that, when stitched together, performs a meaningful business operation. The service mesh controls those microservices which are behind the API while the API is a digital asset that’s discoverable by external/internal parties at the edge of your deployment. To govern these APIs and to manage them and act as a policy enforcement point (PEP) for externally discoverable APIs, the overall deployment therefore requires an API gateway at the edge.

2. Security

A typical service mesh would provide different mechanisms to secure the service to service communication or the east-west traffic of the deployment. The most common way seems to be the mutual transport layer security ( TLS). So the services can communicate with a set of trusted services only. One of the main capabilities of the service mesh is to manage the certificates required for mutual TLS within the mesh, as the services are updated frequently. A service mesh handles these capabilities via its own control plane by rotating the certificates. When a new microservice is deployed or existing certificates expire, the control plane updates all the other microservices by distributing newly added microservices’s public certificate with other microservices. This helps to hot deploy new services within the mesh while maintaining secure mutual transport layer security.

But once these microservices are exposed as APIs to the external parties, there should be a way to authenticate the external parties consuming these APIs. Authenticating north south traffic is more serious because it’s coming from unknown external parties. API gateways are specifically designed for this purpose, to act at the edge and stop unauthenticated or untrusted traffic from coming into the system. 
So API gateways provide stronger end user security mechanisms like Oauth2 and OIDC flows that are designed to identify and authenticate the actual end users of these microservices. Once the API gateway authenticates the north south traffic, the service mesh handles the service to service communication with its own internal security mechanisms like mutual TLS.  

A service mesh can provide authorization capabilities as well. The ability to define custom authorization policies are another key feature of a service mesh. But these policies are actually a set of rules that governs the traffic within the mesh. For example allow traffic from a certain IP address, allow the traffic for certain HTTP paths (eg: :/order/{id}) only, etc. These policies do not focus on end user privileges like their roles or permissions. But API gateways provide more end user authorization capabilities with mechanisms like role-based and permission-based authorization checks using fine grained access mechanisms like scope, XACML or via connecting with policy agents.

3. Traffic management

Traffic management is another common functionality provided by the API gateway as well as by a service mesh provider. Service mesh implementations provide capabilities like connection timeouts, circuit breaking, and request retrying when connecting with microservices. API gateways also provide the same set of capabilities when connecting with microservices. 

Where they differ is in the nature of configurable traffic policies. In a service mesh, the rate limiting policies can be applied at the microservice level. We can limit the allowed number of requests or the request bandwidth (bytes per second) for a particular microservice using a service mesh. An API gateway however allows you to define more complex traffic policies based on the end user. It can limit the access to APIs for certain sets of users under different rate limit policies. Such end user based traffic limiting capabilities can be integrated with billing engines in order to monetize the APIs as well. An API gateway can also apply traffic policies to meaningful APIs that provide a particular business capability rather than applying them at microservice level. These capabilities of API gateways make it possible to block a user from using an API. On the other hand, these kinds of traffic management requirements would be difficult to achieve with the capabilities provided by a service mesh. 

4. Traffic routing

API gateways and service meshes both have their own dynamic routing tables for routing requests to the correct Endpoint or microservice. A service mesh does routing at two levels. First, routing at the ingress level (ingress gateway) to route traffic to the correct side car or microservice and second, within the sidecar proxy to route traffic for service to service communication. 

The API gateway engages at the ingress level in the service mesh, and can act as the first layer of routing rather than using an ingress gateway. So an API gateway is perfectly suited to handling the ingress traffic thereby replacing the ingress gateway of a service mesh allowing only secure traffic to the mesh.

5. Monitoring tools

API gateways and service mesh implementations focus on monitoring tools, but they are intended for two different purposes. Service meshes provide metrics related to microservices where each side car publishes data related to latency, Resource utilization, and the throughput of each microservice that is attached to the sidecar. This data is helpful for devops personnel whose job it is to identify issues within the service mesh, and enables them to isolate issues. 

On the other hand, an API gateway monitors the traffic at the ingress level and provides valuable insights regarding the usage of APIs. Examples of this data include which API has the most frequent hits, from which geographical area the most frequent users are coming, which set of users are the most frequent visitors, how many API calls were successful, and how many calls failed. So these kinds of data can be used for analytical purposes to design future improvements of the Platform. They can also give an idea of how much of revenue is generated and how much revenue is lost due to faulty invocations as well.

The above diagram explains how an API gateway fits into a service mesh at the edge of your deployment. So it’s a bad idea to consider these two as competitors just by looking at the features of each. It’s better to view the two as being complementary to one another in deployments that involve both microservices and APIs. 

Be sure to read the next Microservices article: Microsoft Introduces Open Service Mesh With Intention to Donate to CNCF