Google BigQuery API Makes Big Data Analytics Easy

Ajay Ohri
Aug. 07 2012, 01:00PM EDT

On the Amazon cloud you could create many options for running databases in the cloud. A brief look at "Running databases on AWS" suggests comprehensive cloud as a database service resourcesoffered by Amazon. While Amazon RDS offers you the choice of MySQL, Oracle or SQL Server, Amazon DynamoDB offers you NoSQL storage while Amazon SimpleDB offers schemaless database. Relational Databases can also be chosen as per choice from the Amazon Machine Images. Most of this is accessible via APIs though you can sense Amazon's efforts to drive you towards their own management console. The DynamoDB has APIs and there are tools for  APIs for Amazon RDS, and Amazon Simple DB

For Google you have a MySQL database in the cloud called Google Cloud SQL . However what differentiates and even exceeds Amazon's offerings is the availability of Google BigQuery API .

Google BigQuery is a web service that lets you do interactive analysis of massive datasets—up to billions of rows. Using SQL commands via a RESTful API, you can quickly explore and understand your massive historical data. BigQuery can help you analyze your massive  network logs, identify sales trends across various time series, or help with detecting fraud and outliers. It thus aims to disrupt traditional BI using a very traditional language, SQL!

The BigQuery API has been around since May 2010. Some recent enhancements were made by- developers are able to run many queries at a single time, as long as the total amount of data processed is 200GB or less. After this threshold is reached, an additional query of any processed data size can be run as well. For example, if you run a set of queries that consume 20 Gb, you will be able to run 10 at a time, plus one more query of any size

A lucid list of commands for the API are available at  and you can also see the release notes . So you want to work on analyzing Terabytes of Data but all your IT team knows is SQL , and you are unable to find skilled Big Data Analysts. You can use this very handy tool available at.

BigQuery pricing is quite nice. First 100gb of data processed is free. Storage (upto 2TB) is priced at $0.12 per GB/month while Queries are priced at 3.5 cents per GB Processed .There is a limit of 20,000 Queries per Day and 20TB processed per day. The best use case for BigQuery API is of course  to build web-based BI dashboards to help with business insights.

Big Data Analytics- It is just another API call away!

Ajay Ohri is the author of R for Business Analytics and likes to write on Enterprise ,Cloud and Statistical APIs with an emphasis on interviews. Follow Ajay on Google+ and connect on LinkedIn