Twitter Ending Firehose Access for DataSift, Other Data Resellers

On Friday, DataSift, a provider of data from a number of popular social networks, announced that Twitter is ending its relationship with the company effective August 13, 2015.

After that date, DataSift customers who currently access Twitter data through DataSift’s APIs and want continued access to the Twitter firehose will need to establish a direct relationship with Gnip, the social data provider Twitter acquired in 2014 for $134 million. 

According to DataSift CEO Nick Halstead, Twitter’s move will affect tens of thousands of individuals and companies using its APIs. “Today we power an ecosystem of 1,000 tech companies, enterprises, and marketing agencies, many of whom embed DataSift into commercial applications that they sell to brands,” he explained in a blog post. “The ‘customers of our customers’ span into tens of thousands. This does not only benefit those brands, but also the social networks themselves.”

Halstead says that even with the ability to switch to Gnip, a process that DataSift will help facilitate with a Gnip connector, customers will lose out because 80% of DataSift’s customers rely on advanced features it offers that Gnip doesn’t. For instance, DataSift’s Platform provides automatic categorization of content, advanced filtering, and wildcard searching, functionality Gnip apparently lacks.

“Without these advanced capabilities, companies would be left with simple keyword searches which return large volumes of noisy and irrelevant data. This represents a step backwards for them, a step backwards for the industry, and a step backwards for Twitter,” Halstead lamented.

Differing views on how best to serve Twitter’s ecosystem

The shuttering or restriction of an API platform is nothing new. Twitter has done it numerous times before, and companies like ESPN and Netflix have also made changes to their API offerings after reevaluating their strategies. By now, developers who build on top of third-party APIs know there are risks. But Twitter’s decision to end its relationship with DataSift stands out because DataSift was a formal partner of Twitter’s. No doubt some DataSift customers will have assumed that their access to Twitter data was secure because they were paying for it.

In an update to his post, Halstead suggested that in ending its relationship with DataSift, Twitter is making a grave mistake:

The bottom line: Twitter has seriously damaged the ecosystem this week. 80% of our customers use technology that can’t be replaced by Twitter. At the end of the day, Twitter is providing data licensing, not processing data to enable analysis.

Twitter also demonstrated that it doesn’t understand the basic rules of this market: social networks make money from engagement and advertising. Revenue from data should be a secondary concern to distribution and it should occur only in a privacy-safe way. Better understanding of their audiences means more engagement and more ad spend from brands. More noise = less ad spend.

Halstead says that Twitter’s decision, however disappointing, won’t cripple DataSift. “Our business model has never relied on access to Twitter data,” he explained. “We’ve built a robust Big Data processing platform, which is data source-agnostic, capable of dealing with billions of interactions a day from 20 other social and news networks we’re working with.”

Halstead pointed to the world’s largest social network as a Twitter contrast, revealing that Facebook and DataSift are working together on a new Facebook topic data offering. “This data has never been available before and use cases for its application are practically limitless. It’s also a stepping stone for us in providing unprecedented access to social data, only possible now, because it’s delivered in a privacy-first model with aggregated and anonymized results,” he detailed.

Twitter, however, believes that its new, direct approach will ultimately produce better offerings for those paying for access to its treasure trove of data. In a post on the Gnip blog, Zach Hofer-Shall, Senior Manager of Twitter Ecosystem, explained:

One of the reasons Twitter acquired Gnip was because Twitter believes the best way to support the distribution of Twitter data is to have direct data relationships with its data customers – the companies building analytic solutions using Twitter’s data and platform. Direct relationships help Twitter develop a deeper understanding of customer needs, get direct feedback for the product roadmap, and work more closely with data customers to enable the best possible solutions for the brands that rely on Twitter data to make better decisions. 

To assist customers transitioning from resellers like DataSift, Twitter has prepared technical Documentation and webinars. It is also holding weekly office hours and has assembled a dedicated transition team.

Will companies using Twitter data ultimately lose out because of Twitter’s decision, or will Twitter prove that it is better able to serve them than partners can? Only time will tell. In the meantime, developers have yet another powerful reminder that as important and wonderful as third-party APIs are, uncertainty is part of the package.

Be sure to read the next Data article: 3taps, Craigslist Settle Scraping Legal Battle