As APIs Proliferate, Can Search Scale to Keep Up?

Questioning the potential for API search scalability is the challenge that two presenters, Steve Willmott and Kin Lane, put before Gluecon attendees in a presentation called “API Discovery on the Web.”  As reported by ProgrammableWeb on May 21, these two principals of 3SCALE and API Evangelist, respectively, announced an effort to define: 1) a new metadata format called http://APIs.json to describe published Web APIs, and 2) a new Web API search engine – – which uses the format.
APIs are becoming the “glue” of the Web because they indicate how software components should interact with each other. It is increasingly critical for companies to both provide APIs and consume those of others. Unlike the Web pages of the human Web, the presenters said, Web APIs are rarely linked together and cannot be “crawled” in the same way to build a picture of all the APIs available. And very little machine-readable information is published about them. The current solution is to use public API directories, which do an admirable job of gathering APIs submitted to them. But such a process will be hard to scale as the number of APIs grows rapidly.

It’s a Big Problem

Here’s how the speakers summarized it:
• Increasing numbers of APIs
• Very little usage of machine-readable formats to describe them
• No obvious place to publish or find them
• For users of APIs: We’ll never be able to build something to crawl the “Web of APIs”
• For publishers of APIs: We have to replicate data everywhere
“What we’re proposing augments what’s already been done on the Web,” said Willmott. “That will obviously continue.” But the new metadata format and search engine being put forth by Willmott and Lane “will help more people get their data out there.”
Lane, who runs the API Evangelist site, pointed out that he’s an independent, focused on advancing the API community, “I’m not selling a product.” His recent experience includes working with the White House to help Federal agencies better integrate their IT operations and improve their efficiencies.
The speakers closed by pointing out that they have made available some initial APIs. JSON and resources for the community -- including a Google Group and a Twitter account -- which they hope will encourage active discussion.

Be sure to read the next API Design article: API Design Should Be About Interactive and Tinkering