This is the first part of our series on push technologies. In this part, we will provide a primer on Webhooks and look at real-world APIs that support this style.
Webhooks are used in the style of API where the server pushes, or streams, data to the client. The client does not have to make repeated requests to the server. This push/streaming architectural style of API is well suited to use cases where the underlying data is refreshing itself constantly, such as a stock ticker or a social activity stream.
In a nutshell, a Webhook is a different approach from your typical Web API. As opposed to the usual RESTful API deployment where a server hosts an HTTP-based API endpoint that clients (the "API consumers") pull data from one request at a time, Webhooks reverse the direction of the conversation. It is the client that hosts an HTTP-based API endpoint to which the server pushes data as it becomes available. That endpoint is known as a webhook.
Webhooks are a push notification style that, in comparison to the routing capabilities of other push/streaming-styled APIs, sits at the coarse-grained end of the spectrum in terms of sophistication. Compared to more narrowly focused push mechanisms, this style of push provides limited capabilities for routing to individual application users. It's doable, but Webhooks are better for pushing notifications to one or a small number of endpoints. If the notification is intended for an individual application user, the owner of the endpoint typically takes responsibility for marshaling the notifications received by the Webhook to the correct recipient.
Webhooks employ an HTTP endpoint that supports the POST HTTP method to provide a means to allow an API provider to "call back" an API consumer with the result of a long-running or out-of-band process. The clients in these client/server interactions are almost always servers themselves; therefore, these callbacks are almost always server-to-server integrations. Using Webhooks to push directly to client applications, such as mobile apps, would be impractical and difficult to implement given the need for each client to host an HTTP endpoint and to be in possession of a publicly addressable domain name. Moreover, securing this network using traditional means, such as basic authentication or mutual SSL, would involve an almost unmaintainable administration overhead.
Webhooks have no formal standards at the time of this article's writing, and implementations tend to vary among the API providers that support them. However, a Webhooks implementation would typically include three steps, during which the API consumer calls the API with a request to receive notifications, and the server calls back with its stream. Those steps are:
- An API provider implements an API that invokes long-running processes
that are impossible to wait on over a synchronous connection or that generate out-of-band events. What is next required is the notification of the API consumer. An example could be a help desk API that creates tickets that need human interaction to complete over a number of days.This API would also trigger status updates that the API consumer needs to know throughout the ticket's lifespan.
- An API consumer registers to use the API and configures its settings (via the provider's developer portal) with the URL to his or her publicly available endpoint (with some security features in place). The API provider can "stream" back to this endpoint when the long running process completes or as this process triggers events that should be reported back to the consumer.
- A client-side process might then continue some workflow on the basis of the content of the data that was streamed to its Webhook. For example, in the spirit of programmed-trading, a Webhook might belong to a stock brokerage firm and the stream of data being pushed to that Webhook could include stock prices that might trigger the sale or purchase of a publicly-traded stock.
The scenario described above involves pre-registered URLs, but it is technically feasible to supply a Webhook URL on the fly when an API call is made by the consumer. Both methods have pros and cons:
- Pre-registered Webhooks are less flexible for the API consumers that host them, because configuration changes are required whenever the consumer wishes to change the Webhook address.
- On-the-fly Webhooks could be subject to a security threat if the inbound request is intercepted and changed by a man-in-the-middle style attack. Additional security, such as message signing or certificate pinning, is required to ensure non-repudiation of both parties.
While Webhooks are a coarse-grained mechanism for enabling push notifications, they are also simple and powerful. Many API providers design their Webhooks with features that make sense in the context of the provider's market or product offering. For example, an API provider might offer separate dedicated endpoints for specific event types. Here are some examples of API providers that support a Webhook-based push/streaming API architectural style:
Stripe is a popular payments API provider that employs Webhooks for out-of-band events that are generated as a result of using the Stripe API. They notify the API consumer of disputed charges and recurring billing events. When an event fires, Stripe creates an object that is pushed to the registered URL. Moreover, Stripe also allows API consumers to register multiple URLs and filter which events go to which URLs. The event types are configurable in their developer portal and include account updates, balance changes, etc.
This design mimics the kind the flexibility offered by a true publish/subscribe-based system that employs topics as a means of tailoring the events that are pushed to the API consumer.