At the beginning of 2011 we reported that Collect had decided to drop it's API in order to change their offering to something more profitable. But now ReadWriteWeb have reported the disappointing demise of Collecta. This has the potential of being the first big failure of a well funded real-time web focused company, so questions need to be asked about why this happened and why Collecta weren't successful. Back in January of this year we asked "Is It Finally the End for Real-time Search Engines?" and it now looks like that very question is being raised again.
The first problem Collecta was solving was search, and the problem with building a search product in general is that the market is massively dominated by Google. Even Microsoft's Bing and Yahoo! can't really make a significant dent in the search market, so how can anybody compete? Collecta tried to compete by using real-time search results as their unique selling point - Google now has a real-time search option, and if a topic is trending real-time search results can form part of the general search results too.
Another problem that Collecta was solving was making real-time data accessible. So many companies that have real-time data are now opening up their own APIs and increasing the ease at which anybody can access that data. This means individuals and companies can now got direct to source and build their own search offering, or just extract just the data they need themselves. I'm quite sure that Collecta has some very clever technology that delivered their best guess (algorithmically computed) search results in real-time. But again, it doesn't sound like that was enough.
Creating real-time technology components capable of consuming, processing and distributing large amounts of data was another problem that Collecta had to answer. Initially building the technology capable of doing this was difficult but now, a few years later, real-time technology components are much more widely accessible. So whilst the difficulty level is still high the problem isn't as difficult as it once was. This in itself has meant that data companies can build their own APIs.
Then there's the problem of creating a compelling application and products. A website showing real-time search results in a intuitive and informative way. Widgets showing search results in an easy to digest and embeddable format. The post on ReadWriteWeb provides a comment by one of their own writers, Jared Smith, on Collecta's applications:
Collecta's emphasis on a search experience that went beyond Twitter into photos and videos made it a great tool to truly watch a story unfold in real time. Their embeddable widget, which I used regularly on ReadWriteWeb's event sites, was far more powerful than what Twitter provided and is still unmatched in my mind.
So, even though they were building a useful products it wasn't enough. Did Jared pay for the widget, and if not would he have been willing to? Would enough users have been willing to pay?
The good news out of all of this is that it sounds like the Collecta technology is going to be open sourced. A quote on the ReadWriteWeb post states:
Collecta now says it will open source its software and is working with a variety of organizations to do so - including United Nations crisis relief projects.
Is the real-time search problem hard enough? We should keep our eyes on Topsy and Social Mention to see how they get on. What we can be sure of though is that with Collecta open sourcing their technology the real-time search technology question will now be easier than ever to answer.