Google Updates Cloud Platform to Include Hadoop Storage Connector

Romin Irani
Jan. 17 2014, 11:30AM EST

Last year Google Cloud Platform emerged as a viable and strong alternative to Amazon Web Services and in the first few weeks of 2014 the platform has received various updates. These updates include: a new storage connector for Hadoop, App Engine 1.8.9 release, and improved landing page documentation for its Cloud Platform sample projects repository in GitHub.

Apache Hadoop is popular among developers for processing large data sets across clusters. While Google Compute Engine Virtual Machines can be used to run Hadoop, there is still considerable time that is spent moving data out of Google Cloud Storage and into HDFS. To improve this significantly, a new Google Cloud Storage connector for Hadoop has been announced. This connector makes it possible to run Hadoop directly against Google Cloud Storage data. The object store that powers this behind the scenes in Google Cloud Storage is Collossus. The Connector brings in greater stability, lower costs and better performance by eliminate a lot of overhead. Accenture Technology Labs has published a post that describes how they achieved performance improvements over local HDFS by instead using the GCS connector for Hadoop.

App Engine also saw an update that addressed various fixes and improvements to the PHP runtime. App Engine 1.8.9 is the latest release and you can check out the release notes for more information.

Google Cloud Platform contains a variety of products ranging from Compute Engine to Cloud Storage. Often, it becomes difficult to sift through the various sample projects in the GitHub repository. Going through the current list of 123 project is no easy task and keeping that in mind, a brand new landing page is now available at googlecloudplatform.github.io. The page allows you to select a particular technology, language or even a solution and narrow down the project results to your choices.

2014 has kicked off on a strong note for Google Cloud platform with the focus clearly on regular releases and improving developer experience. GigaOm has reported on a possible Google Channel Partner program for its Cloud Platform in the near future, that is bound to bring it a lot more traction.

Romin Irani Romin loves learning about new technologies and teaching it to others. Follow me on Google+

Comments