Rather than continuing to rely on traditional extract, transform, and load (ETL) tools to get data into Big Data applications running on Hadoop, Paxata at the Strata + Hadoop 2015 conference today unfurled an update to its software that lets organizations use a REST API to load data.
Nenshad Bardoiwalla, vice president of products for Paxata, says the spring 2015 release of Adaptive Data Preparation platform from Paxata leverages a RESTful API to enable data scientists and other users to load data into the application environment without the intervention usually required by an internal IT organization.
In effect, the RESTful API extends the reach and scope of the self-service capabilities of the Adaptive Data Preparation platform, says Bardoiwalla. Via the REST API, for example, organizations can use the Apache Oozie Workflow Scheduler to automate everything from the ingestion of data to the actual preparation of the data itself, says Bardoiwalla.
Designed to eliminate much of the low level coding associated with setting up a Big Data analytics application, Bardoiwalla says the Adaptive Data Preparation platform is specifically designed to be deployed on premise or in the cloud.
In addition to supporting data formats such as XML and JSON, the latest release of the Adaptive Data Preparation platform now also supports data generated by Hadoop and a variety of NoSQL databases.
The Paxata platform itself provides a layer of data management that is embedded inside the Hadoop Distributed File System (HDFS). Combined with an Intellifusion data prep engine developed by Paxata, the Adaptive Data Preparation platform can support both batch and real-time applications running in memory on Apache Spark using advanced caching software, says Bardoiwalla.
In general, Big Data has been a source of both optimism and frustration for many organizations. While there is a general consensus that access to more data should lead to better business decisions, aggregating massive amounts of data and then building applications that can make sense of it has tended to be a complex endeavor.
Bardoiwalla says the Adaptive Data Preparation platform eliminates the manual tasks associated with Big Data projects; without it, highly paid data scientists initially spent most of their time addressing Hadoop plumbing issues.
Obviously, the value of a Big Data project varies widely by organization. But one thing is certain: most organizations never know how much time they waste simply collecting data.