Automating APIs Is a Good Thing--Some of the Time

Automation is a good thing--except when it isn't. There are many tools available for auto-generating APIs. In fact, tools for automating the processes around APIs are popping up all over and being put to use by an increasing number of developers and organizations. But there are times when only an "artisanal" approach will be effective.

But, first, a little history and perspective.

Oct. 24, 2009, is an important day in tech history. For the first time, the nation’s flagship government website, Whitehouse.gov, launched on an open-source content management system, Drupal.

A handful of my colleagues and I did not have time to celebrate. Camped out at the small Alexandria, Va., offices of Phase2 Technology, we had worked extra hard through the months leading to the launch and needed to make sure everything went smoothly as millions of visitors--not all of them friendly--poured onto the new website.

Once things had calmed down, I had time to reflect on the enormity of this event: The small step for an open-source CMS meant a huge step for the civic digital participation movement, and nothing would ever be the same again.

Indeed, only a couple months later, on Dec. 8, the White House issued The U.S. Open Government Directive, requiring that all federal agencies post at least three high-value data sets online. The idea was that by publishing valuable data sets, government would enable third-party developers to create apps for citizens--crucial tools for transparency and civic engagement. This was the official start of the powerful Open Data/Open Government movement in the United States, and initiatives such as Code for America were made possible thanks in large part to the directive.

APIs Are Products

It is fair to say that most federal agencies were not quite ready for the Open Government Directive in 2009. Some of them did have valuable data sets in a digital form, stashed away in highly secured databases, behind military-grade firewalls. Getting those data sets out of the “demilitarized zones” and putting them into the hands of independent app developers was a formidable challenge. Many agencies resorted to just publishing PDF files or, at best, CSV files to data.gov. Even then you would have been hard-pressed to find an independent app developer excited at the prospect of parsing data off of a PDF, but something was still better than nothing.

Enter the API--a standard programmatic interface for mining previously difficult-to-reach data and functionality. But they aren’t just for government agencies, of course. The government directive doesn’t apply to large enterprises, and for-profits and non-profits of all kinds, but these organizations are under enormous pressure from their respective constituencies (internal developers, customers, stockholders, partners, and so on) to provide access to the data and functionality formerly stashed away in the deepest confines of the organizations.

You might call it the “show me your APIs” movement, and with this movement the pendulum has swung from old-school “Web services” to more novel Web APIs.

While Web services were largely infrastructural elements (“plumbing”), Web APIs are widely viewed as products in and of themselves. Indeed, Web APIs enjoy markets and customer demand targeted at the APIs themselves, regardless of the specifics of applications those APIs will be used in. There are many companies now that have no other product but an API they monetize. A classic example of such a company is Twilio, which provides APIs for text messaging, VoIP and voice in the cloud.

Multiple Levels of API Design Maturity

As with any other kind of products, APIs are judged in large part on the user experience they deliver--also known within the API economy as the “developer experience.” This developer-oriented user experience is something you design for. Sometimes we design products intentionally, and sometimes design is unintentional. But every product and, consequently, any API has a design.

Renowned user experience expert Jared M. Spool identifies several levels of user experience design maturity: Unintentional, Experience-Focused, Self-Design, Activity-Focused Design and Genius Design.

Likewise, we can observe several levels of API design maturity:

  • Level 1 (Unintentional): Publishing of unstructured or poorly structured data sets as PDFs or CSV files.
  • Level 2 (Automation): Publishing APIs using tools that auto-generate “APIs” directly from an existing database. This is a bridge between Unintentional and Self Design--a sort of machine-enabled mass-production based on Self Design insights.
  • Level 3 (Self Design): Manual CRUD-centric API design commonly used in “RESTful” API design. CRUD stands for Create-Read-Update-Delete, with each of these methods supported by the official protocol of the Web: HTTP. HTTP-based (a.k.a. Web-based) APIs are the most common form of RESTful API, but usage of HTTP is not required for an API to qualify as RESTful. Regardless, such manually built APIs are self-designed because they are designed from the perspective of the API publisher--reflecting, often verbatim, the data that publisher already has.
  • Level 4 (Activity-focused): Activity-focused design happens when the team bases API design on identifying target users of the API, researching these users and understanding their most common activities. The team uncovers the users’ needs and prioritizes these activities to translate them into top API queries and actions to implement. In final API specs, these activities are commonly expressed as Scrum-style “user stories.”

Fast and Furious API Generation

The explosive surge in market demand for APIs has been pushing organizations--both in the government and outside of it--to quickly come up with a solution to “liberate” data and capabilities out of internal enterprise stores.

The simplest, Level 1 of API “design”--much like Spool’s Unintentional Design--quickly shows its significant limitations. Many developers quickly graduate to Self Design, only to find out that it requires time and experience that isn’t always available, especially on short notice.

Irakli Nadareishvili Irakli is director of strategy at API Academy of CA Technologies. His interests include: APIs, distributed systems and product design. Irakli is highly involved in the startup community and has spent over a decade in Washington, DC building innovative products for media companies, government and international organizations, while also being an active open-source contributor and advocate. You can connect to Irakli on Twitter at @inadarei.

Comments

Comments(1)

Evensteven01

Great article. It's odd how sometimes you need details pointed out to you to be aware of them. I've worked with APIs of all described levels but didn't think much of it. I think experience guides much of this. My most recent and complete experience with an API was with a thought out, intentional logic. It's important then to realize this isn't always the case, and that this isn't even always the best of case; over engineering an API can be worse then under engineering. Thanks for three perspective.