Built on top of the real-time apps platform offered by Firebase, a Y Combinator portfolio company, the JSON REST API gives developers access to all of the stories and comments posted by users to Hacker News. Initially, the API provides read-only access and doesn't require authentication, but Y Combinator says it may add OAuth authentication and greater functionality as the API evolves.
The primary reason Y Combinator developed an API is to support modernization of the Hacker News user experience. "The reason we released an API is so that we can start modernizing the markup on Hacker News," Y Combinator's Kevin Hale explained. "Our goal is to switch to a new rendering engine ... in three weeks (Oct. 28). This new markup will allow us to iterate faster on the front end and finally make Hacker News mobile optimized."
Making Life Easier for Developers
Another reason Y Combinator launched an official API is to make life easier for developers who, up to now, have used scrapers to build apps around Hacker News content. While unofficial APIs, propelled by the rise of increasingly sophisticated scraping tools like Gargl and Import.io, are here to stay, many companies and organizations have not yet been convinced to launch a public API. Some even attempt to block scrapers through technical and legal means.
Additionally, recognizing that so many developers have built Hacker News scrapers that rely on the current Hacker News HTML structure, Y Combinator will is waiting to implement its interface upgrades so that developers have time to integrate their apps with the API. "Because there are a lot of apps and projects out there that rely on scraping the site to access the data inside it, we decided it would be best to release a proper API and give everyone time to convert their code before we launch any new HTML," Hale explained.
Despite Y Combinator's good intentions and mindfulness around giving developers time to adopt the API, the reactions on Hacker News to the API launch announcement have been mixed.
One commenter wrote, "I really appreciate giving a 3 week heads up before moving to a new frontend structure. It's a nice gesture, but I have this horrible feeling that there's only about a 10% chance that my Hacker News app gets updated in time." Others suggested that the API doesn't offer enough functionality. One user, Livven, for instance, wrote a detailed explanation of his concerns:
On the one hand, of course it's great that HN is finally getting a proper API and also modernizing its markup, ... but on the other hand this current v0 version is very lacking and impractical for a regular client application.
Since the top stories (limited to 100) and child comments are only available as a list of IDs a client app would have to make a separate HTTP request for every single item, which is obviously not something you'd want to do especially in a mobile environment. Other lists apart from the top stories (new, show, ask, best, active etc.) don't seem to be available at all right now.
Of course this is just the first version, and the documentation promises improvements over time — which I don't doubt at all — but there's no clear indication that the API will be at feature-parity with the current website, even excluding anything that requires authentication, by October 28. So this means that I — and other developers of client apps or unofficial APIs — will probably have to write new scraping code once the new rendering engine (which I assume refers to the website) arrives instead of being able to switch to the new API immediately.
Obviously, issues are to be expected with any new API, but the Hacker News API launch highlights some of the challenges associated with the release of an official API when developers have been using scrapers for years. It also serves as a reminder that involving developers in the design of an API before it's unveiled can help ensure that the API meets the needs of those it's intended to be used by.