A Contest To See If Your Search Idea Has Legs

Adam DuVander
Nov. 12 2009, 12:39AM EST

What could you do if you were spidering two billion web pages per day? Whatever your answer is to that hypothetical, you might as well do it now. The 80legs platform lets you write your own web crawler and sell your apps to users (our 80legs API profile). And if you can build it fast, there's an 80legs contest looking for your entry.

The company wants to fill out its 80apps store, which is launching later this fall. As an incentive they're offering some pretty geeky prizes: Herman Miller chair, gadgets and an Atari 2600, for example. To qualify you'll need to submit your app by December 11th.

Of course, there's not a whole lot of code to write, since the crawling is done for you. Using Java, you'll need to implement processDocument() and parseLinks() functions. Then you'll need to export a JAR file and upload it to 80legs. More technical details can be found on the 80Apps Wiki.

80legs service comparison table

As you can see from the table above, the platform is not free. However, it's never been cheaper or easier to crawl the web. Any apps you create, whether or not they win the contest, will be available for sale on the 80apps store. You keep 100% of the revenue and just pay 80legs for the page crawling and CPU-hours.

Unlike most APIs, which provide data, 80legs is providing a service, one that usually takes much more cost and effort to set up. Also, it is enabling developers a means to make money directly, something missing from many paid APIs. Expect tools like 80legs to gain in numbers, as developers start expecting platforms that give them a boost.

Adam DuVander -- Adam heads developer relations at Orchestrate, a database-as-a-service company. He's spent many years analyzing APIs and developer tools. Previously he worked at SendGrid, edited ProgrammableWeb and wrote for Wired and Webmonkey. Adam is also the author of mapping API cookbook Map Scripting 101.

Comments

Comments(2)

[...] of new APIs added to our API directory including a commercial search API (which we reported on in A Contest To See If Your Search Idea Has Legs), a mobile location positioning API from Ericsson Labs, and a new service from Google: their Issue [...]