How Kimono Fills Need For API-Second Design And Transformation-As-A-Service

 

 

If you don't create an API for your Web site (or for one or more of your Web pages), then someone else -- someone you may have no control over -- will create it for you (potentially wreaking havoc on your infrastructure capacity and analytics). And, with services out there like KimonoLabs.com that make child's play out of "APIfying" any site, that someone could just as easily be a hacker looking to scale an attack, your grandmother working on her first app, or a 5th grader working on a research project.

Within the API economy, there's a growing number of proponents of what's called "API-first design." API-first design is the antithesis of automatic API generators like Kimono with the idea being that you should design an API with almost blatant disregard for the legacy systems that might be called upon to serve it. In other words, design the API in a way that developers will be drawn to work with it rather than in a way that mirrors existing data structures or system functionality. Then, not until after you've designed your Ansel Adams of an API, and realize how your existing systems might not be up to the task, do you make some, ahem, adjustments.

But let's get a real for a second. Sometimes, you just don't have the luxury of the time that it takes to lock yourself into the equivalent of an API clean room, working with prospective developers to get feedback, perfecting your developer experience, and then minding the gap to your legacy infrastructure. Sometimes, you just need a friggin' API to support a recurring (or even a one-time) business process (aka "workflow"). Perhaps you need to build an adapter that sits between two unrelated APIs over which you have no control (in which case, design pretty much doesn't matter). And, as the embedded video above shows, often times, one source of relevant data may not even belong to you. It belongs to Reddit.com, or someone else. There is no option to refactor the existing infrastructure. Don't worry. Us API-first snobs won't judge you. Or at least I won't (hint: I have exactly one of these projects underway myself).

API-Second Design?
But what if that API sausage factory -- be it KimonoLabs.com, Import.io, DreamFactory, Espresso Logic, etc. -- gave you one last chance to modify the endpoint? You know? Tweak the API design to the point that we could probably call it API-second design; where it's more about transformation-as-a-service (TaaS) and API design is very much a primary consideration but circumstances dictate that it can't come first.

That's exactly what Kimono just did when it announced an experimental feature for "transforming your API results."  Instead of taking an API as it is originally created by Kimono's API generator, you can run a bit of custom code written in Javascript to transform the endpoint into something completely different. In the video example, the "stock" API that Kimono generates from Reddit.com is transformed into one that relies on some calculations to return the total amount of karma and the total number of comments for any given sub-Reddit. 

By now, I've already spilled more ink than Kimono did in its own blog post for, what on the surface appears to be a cool but but not necessarily earth shattering feature. As you play with the transformational Javascript, the transformed endpoint's JSON changes right before your eyes. It's pretty neat. But whereas Kimono billed it as an opportunity to do those sorts of transformations, I read between the lines and saw more.

Can it slice, dice and make julienned potatoes?
For example, it's a Javascript engine. So, technically speaking it's not necessarily limited to transformations. It could host an entire server-side app, Kimono permitting. In addition, you can run your transformations on KimonoLabs.com while I run mine. In other words, it's multi-tenant; a multi-tenant, server-side Javascript engine. Sound familiar? How can one not look at Kimono as a potential Javascript PaaS like Joyent, Modulus, Openshift, or Heroku? Think about it. If you're grandma doing your first app, or the 5th grader doing a research project, do you really want to have to spin up your own app infrastructure to consume the newly created endpoint?  Wouldn't it be just as cool if the platform that made the sausage also gave you the opportunity to host a process that consumes it as well? 

Or, how about comparing it to a Javascript-based community playground like JSFiddle or Codepen? Going back to the idea of API-second design, perhaps the sub-Reddit summary API could stand for a little improvement. If there's a Kimono Kommunity, transformations like the Reddit one can be open-sourced, publicly shared, reused, and forked.

So, I reached out to Kimono CTO and co-founder Ryan Rowe to see if he had much bigger plans in mind now that the company was on a slippery slope to becoming a platform as a service. After chuckling and saying "I'm glad you noticed... we knew some people would," Rowe said they envisioned this capability even before launching the company. The idea was born out of a relatively boring, but incredibly important step in most workflows; data cleaning. 

"We had a conversation before we launched the product" Rowe told ProgrammableWeb. "Where data scientists spend most of their time is not in getting data, but rather cleaning it. It's a non-trivial problem. So, providing small modular data-cleansing functions --- like normalizing a date --- in the form of something that can be written once and re-used forever has been on our roadmap since the beginning."

"But, it's true, there's nothing stopping us from becoming a platform as a service once you have a unit of code that can can be modified and shared," said Rowe. That said, right now, Kimono is focused on the sort of transformations that can build a bridge between two dissimilar Web systems like one that outputs a date in one format and another that expects the date in a different format. He spoke of the need for what he calls "Web middleware." Not the sort of middleware represented by a J2EE server. But rather, an easy way for two Web-bound entities like an on-line advertising platform and a media site to talk to each other about what's going on during a given user session in a way that improves the ad delivery and user experience.

Today, the two might have the opportunity to talk to each other through a set of predefined interfaces that do very little to improve the user experience. But, what if, through Kimono's scraping and spidering capabilities, something about any current user's behavior could be easily scraped? Then, using Kimono's platform, that data could be served through a dynamically, on-the-fly-created endpoint that's designed to meet the expectations of the ad-serving system.

The problem for those of us running media platforms is that molding the content management system's endpoint to the expectations of an ad-serving system is normally a 3-month IT project (so long as your developers have nothing else to do). Almost none of these potential integrations come out of the box with today's content management systems, especially if the public facing templates of the CMS have been customized or the ad-serving system belongs to a new startup. But, between Kimono's scraping capabilities and the new found ability to craft an endpoint to your liking, new, more flexible, and agile opportunities are created.

Today, Kimono is no Heroku-like PaaS. Rowe admits that in order to support that sort of PaaS-like activity would require significantly more infrastructure. In fact, even though it's a multi-tenant Javascript host, there are some things you can't do. For example, you can't access the network to call other HTTP APIs or consume too many resources. Rowe wants to start with simple, sandboxed transformations and offered translations as an example.  "Imagine that your subscribed to a bunch of news sites from the US, China, Japan, and Europe, but you want all the headlines and content to be delivered via API in English" says Rowe. "[Kimono] could [scrape the headlines from the various sources of news and] offer a connector to a translation service that can handle that transformation for you and it wouldn't require a lot of developer expertise."

According to Rowe, in a version 2 of the service, Kimono will officially sanction a handful of partners to support various transformations. Some of the details still have to be worked out like how third-party billing will be handled and how Kimono will absorb the additional workloads. However, once those details are worked out, Kimono will offer an SDK that unlocks the capabilities for developers.

Microservices cometh?
If you've been reading up on the microservices trend, then a lot of what Kimono has planned will probably sound familiar. There are a lot of different interpretations of the term "microservices" with many aficionados admitting that no formal definition or pattern exists. Over on APIevangelist.com, Kin Lane wrote, "I’m sure my own definition of what a microservice is--will evolve, ebb and flow, but for now the first rule of microservice club for me, is keep things doing one thing, and doing it very well—enforcing simplicity in every decision."

If we at ProgrammableWeb were to draw a distinction between your typical single function CRUDdy HTTP API and a microservice, it would be that the microservice includes a unit of data processing and or workflow (usually involving some business logic and the consumption of one or more APIs) that will get re-used enough to justify packaging it for such re-use under its own API (this, admittedly over-simplified definition, will no doubt draw criticism, but we'll save the full-blown microservice conversation for another article).

If you've read Martin Fowler's and James Lewis' treatise on the topic, one of these keys seems to be how "These services are built around business capabilities and independently deployable by fully automated deployment machinery."  

Again, enter Kimono. To the extent that automated production of APIs and re-usability are some of the tenants of Rowe's example transformations, one could argue that Kimono is now also an enabler of microservice generation; non-compute-intensive mashups whose output is consumed via API. 

Finally, while Kimono scrapes Web pages today, Rowe says it won't belong before it starts to work with PDFs and other document types. Said Rowe, "We're agnostic to sources of data, and agnostic to where it's going. If we take data in that looks like anything and output that looks like anything, we can fit into a lot of workflows."

David Berlind is the editor-in-chief of ProgrammableWeb.com. You can reach him at david.berlind@programmableweb.com. Connect to David on Twitter at @dberlind or on LinkedIn, put him in a Google+ circle, or friend him on Facebook.

Comments