Today in APIs: Databricks' Spark-Based Cloud Platform Uses a Single API

Databricks launches its Spark-based cloud Platform. Surviving the end of API access. Plus: Netflix's Security Monkey, and Iterus launches the ClearPath Ag Precision weather content system.

Open Source Apache Sparks Creator Databricks Launches Cloud Platform

Databricks' release of its cloud platform aims to ease extracting value from Big Data. The problem is herding the "hodgepodge of disparate tools." Databricks wants to replace those with a zero management platform. A big piece of that is having just one API.

spark ecosystem

As the company commented,

Databricks Cloud is powered by Spark, a unified processing engine that eliminates the need to stitch together a disjoint set of tools. Spark provides support for interactive queries (Spark SQL), streaming data (Spark Streaming), Machine Learning (MLlib) and graph computation (GraphX) natively with a single API across the entire pipeline. Additionally, Databricks Cloud reaps the benefit of the rapid pace of innovation in Spark, driven by the 200+ contributors that have made it the most active project in the Hadoop ecosystem.

The cloud comes with several built in apps. Notebooks is the interface for data discovery, interactive plotting, and collaboration. Dashboards can be created on the fly and refreshed at will. A WISIWYG editor makes it possibly to publish the dashboard. Plus, a job launcher makes building data products simple by allowing users to run Apache Spark jobs.

Is There Life After Death of an API? PeopleLinx Shows Path to Reincarnation

It was only yesterday that I penned a piece about how Netflix shutting down its API heralded a chilling trend away from public APIs. But Nathan Egan, founder of PeopleLinx, says getting cut off from an API can be a good thing. In his case, the API from LinkedIn is still very much available. But out of the blue he got an email from LinkedIn saying access had been terminated because their use violated terms of use. PeopleLinx helps companies turn employees into brand ambassadors by using social media, including LinkedIn. He describes four key steps to getting over the crisis. Transparency was key: First, level with the team. Second, level with customers—by doing so he unleashed a lot of good will that came to the company's aid and protested LinkedIn's decision, arguing that PeopleLinx made LinkedIn more valuable. Third, they engineered around the road block. And lastly, they were forced to identify their company's real value, their expertise, and capitalize on that, not someone else's API.


As Egan writes, those who are worried about companies hitching their stars to someone else's API are right to be nervous:

In today’s “API economy”, many entrepreneurs found startups on APIs and access to someone else’s data. That can be a great way to get a business off the ground quickly. But as we learned the hard way, you’re putting yourself and your business at risk if you depend too heavily on the kindness of strangers. Sooner or later you have to look deep inside your company to find that thing—the proprietary data, expertise, technology, relationships, whatever it may be—that is your company’s unique contribution to the ecosystem.

PeopleLinx has landed on its feet. One measure of health besides retaining and growing the customer base, they are hiring again.

API News You Shouldn't Miss

Be sure to read the next Cloud article: Today in APIs: ShepHertz Launches APIs for Time and Gift Management in Apps