Why It’s OK To Design Imperfect APIs

Perfectionism will only get you so far, especially when it comes to API design.

It’s easy -- too easy -- to think that an API is either well designed or poorly designed, and that objective perfection is within striking distance. But the real world is a chaotic and kinetic place. Technology evolves quickly and how people use it changes even faster. The best developers accept that they will never know everything about the problem they’re tackling at the outset. They embrace the ambiguity of the world and carefully design their APIs for that uncertainty. It may sound counterintuitive, but in the long run, an imperfect but flexible API will out-perform a “perfect” but rigid one.

As an example, we launched Stripe Connect earlier this spring. We set out to build an API that was purposely flexible for online marketplaces, and we learned some useful lessons along the way.

But before we get to those, some historical context. Stripe Connect helps marketplaces like Lyft and Kickstarter enable payments to their sellers. Our first API for marketplace payments, released 2 years ago, looked something like this:

POST /v1/recipients
 name=John Doe
 type=individual
 tax_id=867-53-0909
 bank_account=btok_1234

This worked fairly well in the U.S., where identity verification is fairly straight-forward. However, international support posed a problem. Every country has its own requirements for information collection. We initially did the simplest thing: added more fields to this API. For example, collecting business and personal tax IDs as well as VAT for European customers:

POST /v1/provision
 country=US
 first_name=John
 last_name=Doe
 personal_id_number=867-53-0909
 business_type=corporation
 business_name=John’s Socks
 business_tax_id=27-8675309
 bank_account=btok_1234

But that quickly stopped scaling, as every new country would result in the same back-and-forth with our customers: they would ask what we needed for a Stripe API call to work in that country, and we would send them a documentation page with a list of fields. They would hard-code this list somewhere, so we had no flexibility to change these requirements, nor implement any custom logic on a per-platform basis.

If we were confident in our knowledge of the identity verification requirements for each country, we could have published a formal list somewhere. However, in many countries regulations around marketplace business models like ride- or apartment-sharing are still evolving, so these requirements are likely to change over time.
Instead, when we started to build our Connect API from scratch, we tackled the identity verification problem by requesting information, resulting in an “Account” object that looks something like this:

{
 ...
  "legal_entity": {
   "first_name": null,
   "last_name": null,
   ...
  },
   "verification": {
    "fields_needed": [
     "legal_entity.first_name",
     "legal_entity.last_name"
    },
    "due_by": 1459728000
   },
   ...
   }

From one legal entity in one country to another legal entity in another country, this design allowed for the list of required verification fields to differ per local regulations. This design makes for a more powerful API, but also sets a much higher bar to integrate. The marketplace vendor can no longer just build a hardcoded form into its Web site. Instead, the form logic must first determine what fields are regionally required and then build the form based on what it learns.

Through the process of building and launching the Stripe Connect, we’ve learned a few important, general lessons about this model of flexible API design:

Lesson #1: If you require an integrator to build some capability, verify that it’s in place before allowing them to launch

Developers will often (correctly) ignore as much as possible to launch as quickly as possible. This may mean taking the path of least resistance when integrating an API, which can result in unforeseen frustration for them later on. With Stripe Connect, some developers build integrations to collect just the bare minimum of information needed to receive a payment -- leading to problems down the road when they need to collect additional information, but are not yet set up for it.

But as the designer of the API, it’s our job to help developers avoid this pickle to begin with. The ideal solution is to remove as much complexity as possible, making the implementation too simple to gloss over anything; barring that enforcement and alerting helps. For example, it goes a long way when the API provider offers a launch checklist (preferably one that's automatically filled out) in order to make the requirements clear.

Lesson #2: Minimize requirements

But on the flip side, it’s crucial to recognize that a more complicated API leaves less room for platforms to embrace the Paul Graham adage, “do things that don’t scale.” We’re essentially asking startups to invest more upfront to greatly reduce frustration later. You want to minimize their pain, but you also don’t want too high a barrier to entry. Be clear about your requirements, but make sure your requirements are truly just the essentials.

For Stripe, this means providing more of our API functionality via our dashboard and emails to platforms, so small startups can manually handle extra information collection until it becomes unscalable for them to do so.

Lesson #3: Always be improving -- if not the API, then the documentation

Stripe Connect had an extensive beta period with as many and as varied partners as possible. But even then we were overwhelmed with the variety of businesses using our new APIs in unexpected ways. They all came from different perspectives, and wanted to apply our API to their existing view of the world. If they had difficulty doing so, they felt (understandably) frustrated.

A key insight for us was realizing that while the API didn’t need to change, something needed to change. Documentation improvements are often overlooked as a key opportunity to yield smoother deployments and happier developers. Many products set up their documentation for launch, but go on to make only minimal, incremental improvements thereafter. Building a robust, flexible API takes a lot of time and effort. But once you’ve finished that, commit yourself to developing (and updating!) great documentation and communications to support the API.

* * *

These lessons may sound simple enough, but they can be challenging to execute. And their importance, like the overall importance of good API design, multiplies with the complexity of the domain you’re tackling. Ultimately, the more uncertainty and flexibility you build into your API now, the less likely you’ll end up with a “perfect” but outdated API in a few years.

Brian Krausz

Comments