How API First Design Could Have Avoided These Failures

When I speak at conferences and meetups about  some of my experiences consulting and advising on APIs - the most instructive story I could tell dates back to when I contracted as an engineer (many many years ago) to build an API from the ground up.

That story, goes something like this:

Several years ago, I had the chance to contract on an API.  We spent several months building the API - incorporating best practices, and of course the code was perfect (because I wrote it - usually people snicker at this part for some reason).  But then something strange happened, we released it only to realize that no one was using it.  After six months of development, we launched only to have an API that didn’t meet any of our customers’ needs.  We spent the next three months frantically trying to salvage the API, only to eventually scrap the entire project.  In the end, nearly 10 months was spent on building something that nobody wanted.

This experience, with the emergence of specifications like RAML and Swagger was one of the main reasons I became such a strong proponent for “Spec Driven Development,” or taking a design first approach.  Not only does using a specification like RAML significantly reduce the typical redundancies of supporting an API (unit tests, documentation, SDKs, etc), but it can actually save you from making fatal mistakes from the get-go.

Yet, the usage of these specifications and Spec Driven Development is still very limited.  The mistakes that come from the code-first approach, however, remain prevalent.

A year ago or so, I had the opportunity to listen to a lead API Architect share his team’s struggles to create the perfect API.  Many team members had ideas - some great, some very unconventional, and some downright horrible. The team collaborated, the best of minds working to agree on a strategy and design for the API.

Yet the end result was iteration after iteration, with each being a public release that caused more problems than it solved.  In the end, a very widely used API was on life-support - inconsistent in how calls were made, responses, querying/ filtering, and even inconsistent the very notions of REST and HTTP constraints in general.  An API, that his team will soon have to completely rebuild, for the sole fact that it has become more and more difficult to manage - even though the team behind it contains some of the brightest minds in the industry.

But this isn’t limited to just one company.  Just the other day I received word of a highly in demand, new resource for an API I track.  This release was highly touted, and initially received with high praise - UNTIL users quickly realized that it lacked some of the most basic information required to use it.  Again, months of work had been put into something - and the end result was that it was useless.

Imagine how simple it would have been to instead spend 5-10 minutes writing a RAML or Swagger specification, mocking the API, and letting the potential users review it.  Instead of spending months working on something that does nothing - they would have had immediate feedback on what their users REALLY needed and how to fix it - all before writing a single line of code.

The examples go on and on.  Not limited to one company, or one API - but hounding thousands of APIs and costing companies millions of dollars in development time, documentation, SDK development, refactoring, versioning, and marketing/ getting developers to upgrade to the new API upon release.

Yet another case comes from having multiple developers working on a singular API - as many companies do today.  In a particular case, a user resource lacked the ability to do some basic searches/ collection retrieval (again, something that could have easily been averted had they simply take a customer/ design first approach).

As customers demanded these basic components, developer and company priorities shifted, forcing a new developer to take over and add the additional components.

The task should have taken no more than a couple hours of development time, causing no backwards incompatibility or complexity - after all, it was just a very simple filtering mechanism.  Instead, implementation took weeks as developers struggled to implement it.  When the release finally came, it was not the added filtering on the requested ‘/users’ resource, but a brand new resource called ‘/accounts’ that functioned completely differently, and provided different information than the users resource, requiring consumers to first call the ‘/accounts’ resource AND THEN call the ‘/users’ resource to get the necessary information.

Outside of omitting some information, and limiting the ability to view aspects of it - the ‘/users’ and ‘/accounts resources are identical - a highly inconvenient design flaw caused by the fact that the new developer was inexperienced and unaware that a resource for ‘/users’ already existed.  This also means that property names throughout the API are now inconsistent (userId == acountId).

The end result is that the life of this API will be short lived, marred by its design flaws and inconsistencies.  In other words, after all the work the company has put into this API, it is only a matter of months, maybe a year if they are lucky, before they have to completely rewrite it.

The idea of design first is extremely simple, and highly cost-effective.  Many companies are unaware, or simply choose to take a short-cut to save on the development cycle, exchanging months of refactoring and struggles for a couple of weeks to make sure it was done right.

The truth is, there are no shortcuts in effective API design. There are ways to be more efficient, there are ways to be smarter - but anytime you cut corners you, and more importantly your consumers will end up paying the price.

Learn more about Spec Driven Development, API Specifications, and API Best Practices in my book, Undisturbed REST.  You can download a free copy here.

Mike Stowe is the Developer Relations Manager at MuleSoft.




Nice article, Mike!! Good to see you out in front sharing lessons with the API community, on one of the best API content sites, and linked to from one of the best API newsletters!