Overcoming Our Inner Data Integration Conflicts

As more organizations begin to think about data as a business asset versus a burden then needs to carried, many of them are become more cognizant of that fact that data only has real value when it’s shared. After all, nobody is going to spend money on data they can’t access. The trouble is everybody seems a little conflicted about how to best go about doing that.

The primary way data has been shared between applications has been via APIs that connected fairly rigid sources of data, such as a SQL database. But more recently we’ve seen not only the rise of alternative sources of data that are just as static, there are now any number of smaller sources of data that developers need to access. What that is creating is a need for a more flexible approach to share data between applications.

By way of example, VMware recently announced Cloud Foundry Integration for Eclipse, which adds the ability to open a tunnel to any Cloud Foundry data service. According to Dekel Tankel, director of product marketing for the Cloud Foundry Platform-as-a-service (PaaS) offering developed by VMware, data tunneling is most often used to integrate relatively small amounts of data within the same application environment. But as cloud computing evolves it is clear there will be a need for more use of data tunneling between applications.

James Governor, co-founder of the industry analyst firm RedMonk, says data tunneling could provide an interesting alternative to traditional APIs. Although both approaches are still fairly brittle, Governor notes that data tunneling provides a more flexible approach that could be more easily used across a wide variety of data sources. That approach, adds Governor, would be consistent with VMware’s desire to make sure the Cloud Foundry community doesn’t become overly dependent on a relatively small number of data sources that are dominated by one or two vendors at most.

Many of those data sources are emerging because many developers are trying to do an end run around database administrators (DBAs) that many of them view as having become bottlenecks. Of course, that bottleneck exists because some form of data governance policy needs to be applied to data Integration. Who ultimately performs that Function remains to be seen. But in perhaps one of the greater ironies of application development, it’s likely the developers will wind up enforcing data governance policies of their own creation if for no other reason than motivated self-interest. After all, if data has any value at all, somebody has to control who has access to it when and where. Of course, as Governor notes, there is no greater hate than “self-hate” when it comes time to put those data governance policies in place, which may lead to a lot of self-loathing among developers that in many ways are already a pretty conflicted bunch.

Regardless of how developers or DBAs may feel about the governance issues involved, however, there is be no stopping of the federation of data services across the Web. The only thing that really needs to be resolved in the absence of some formal data linking standards is finding a way to do that in a way that not only everybody can actually live with, but eventually even come to like.

Be sure to read the next Best Practices article: In Defense of OAuth 2.0