While there’s generally a lot of excitement these days for all things Hadoop the simple fact is that there’s not a whole lot of expertise out there in terms of IT people that know how to build, roll out and then manage an instance of Hadoop. That creates a frustrating conundrum for developers looking to build Big Data applications on top of Hadoop at a time when interest in those applications is staring to reach a frenzy in the enterprise.
In the early days of Twitter, occasionally when you tried to log in you got a “fail whale” – a picture of a whale held up by a flock of birds — as a way of telling you the system was overcapacity, and you should simply try again later. We tolerated it then because, after all, it was only social media. But there are certain types of services, for example, financial exchanges, where you never want a fail whale. So if you have thoughts of building a Bitcoin exchange, where people can buy and trade the popular virtual currency in real-time, you want an infrastructure that can handle high volume trading spikes without going down.
A little over a year ago, Google released a new product called Google Fusion Tables. Fusion Tables allows you to upload, merge, share, and visualize large tables of tabular data. Basically, we wanted to answer the question “I have a spreadsheet. I want to share it with people and create a map/bar chart/storyline/other visualization.” Google Spreadsheets, while powerful, couldn’t handle the 100 meg upload we allow for Fusion Tables. And, being Google, we also provided something for developers.