Inside Analysis

Cloud Computing: Changing the Data Integration Landscape

Editor’s Note: Dell Boomi’s CTO Rick Nucci briefs author Dave Menninger in a recent Briefing Room entitled, “How the Cloud Enhances Traditional Integration Practices.” Click here to view the lively conversation.

By Dave Menninger

Cloud computing is having an increasingly large influence over the IT landscape. It is likely that, whether you realize it or not, corporate data exists and/or is migrating outside the walls of your organization. Recent research by Ventana Research shows that in areas such as customer services, sales, and workforce or human capital management, software as a service (SaaS) or cloud-based applications increasingly are being accepted and adopted. In our benchmark research on business intelligence and performance management, for example, only 53 percent of organizations prefer their systems on-premises, and we expect that percentage to decline in the next 12 to 24 months, in which more than one-third of organizations plan to begin using cloud-based or SaaS applications.

However, cloud-based applications and services raise information management challenges that don’t necessarily exist in on-premises deployments. The new silos of applications and software that enable doing business “in the cloud” also are new corporate data repositories that must be integrated with other enterprise data and must be managed as a whole. Among the many challenges lurking inside the cloud are data accessibility, data consistency, data integration, data quality, and data governance.

In many cases, the advocates and buyers for using cloud-based services are line-of-business managers who see such solutions as addressing their immediate concerns for rapid deployment with minimal capital outlays. But these business folks may not be aware of the data challenges associated with moving to the cloud. For instance, as more and more data resides in applications managed by third parties, how will the organization bring it all together for analysis, reporting, and other necessary uses? Without a capable data integration infrastructure, will users be forced to cut and paste data from reports or export it to spreadsheets, encountering the issues of consistency and accuracy that practice raises?

The proliferation of cloud-based data and applications has implications for the movement of data into and out of the enterprise. When a line of business goes to a cloud-based application, it must migrate its data about employees, customers, products, and other essentials to the associated cloud-based application. Since some applications are likely to reside on the premises, some portion of the cloud-based data may need to be brought into the enterprise as well. Once data resides in multiple locations, challenges arise in data synchronization from system to system in different locations – in the cloud and locally. In this case, the consistency of data used by various people at various times may come into question, causing costly errors in business decisions or inefficiencies as personnel spend time reconciling different sets of information.

When their data resides outside the firewall organizations also must rely, at least in part, on the cloud providers for adequate data governance and security. As business advances its use of software as a service and managed services in areas such as CRM, ERP, BI, and others outside of IT, our research shows only 12 percent have deployed data governance to this area; an additional 18 percent had plans to do so within 12 months. That leaves more than two-thirds who have not tackled data governance in the cloud. When it comes to relying on the third-party providers, only one-third of organizations are very satisfied or satisfied with current efforts to support data governance. Clearly a significant gap exists where organizations need to embrace data governance no matter where data is created, stored, and used. Master data management systems would be similarly impacted as well.

Some vendors have begun to adapt their on-premises information management products for cloud-based data. Other vendors have emerged to focus purely on cloud-related data issues. The market for cloud data integration products and services is still emerging. We have learned some lessons in the on-premises past that will be applicable to the cloud, and established vendors are aggressively pursuing these market opportunities through a combination of development efforts and acquisitions. We also see new vendors entering the space, each with their own unique capabilities. One common theme I hear repeated by large and small companies alike is that cloud data integration is about frequent, smaller transfers of data rather than large bulk operations. I’ve also observed that none of the cloud-only vendors offer all the functionality of fully established on-premises solutions.

Marketplaces, another phenomenon associated with cloud-based applications, if successful can help level the playing field between new and established vendors. A marketplace provides a venue where third-party applications can be shared with the user community for a particular product or vendor, effectively expanding the vendor-provided capabilities. With the proliferation of cloud-based applications and data sources, both the need and the opportunity exist for connectors to these different sources. Our research indicates that organizations are already integrating on average five or more different data sources. As more applications are delivered as services, this number is likely to rise.

Even the largest vendors will struggle to support all the different data sources organizations need to access and integrate. So in addition to evaluating the technical merits of different cloud-based solutions, it’s also important to understand how active the marketplace is surrounding the product you are evaluating.

Your organization will need to address another key question specific to the cloud: public versus private. Most of the discussion of cloud-based products and services centers around the public cloud platforms such as Amazon’s Web Services. However, private clouds offer an alternative with many of the benefits of public clouds while minimizing some of the concerns about security and service levels. In my experience, while private clouds may address these key concerns they lag behind public clouds in delivering core benefits such as automatic provisioning and elasticity.

As an industry we’re just beginning to understand the challenges and opportunities that are unique to the cloud. It’s important to understand the issues that are driving purchases of cloud-based solutions. Combined with an understanding of IT’s concerns about cloud-based deployments, we can develop strategies that align these two sets of requirements.

Author Dave Menninger is VP and Research Director for Ventana Research.

One Response to "Cloud Computing: Changing the Data Integration Landscape"

  • Darren Cunningham
    May 28, 2011 - 1:17 am Reply

    Great post Dave! It’s a hybrid world and having the right data integration strategy is increasingly recognized as critical to cloud success – especially as it relates to SaaS applications and platforms. These days IT is involved more than ever and governance is always going to be top of mind.

    I do believe that the data replication use case in particular requires both real-time (only replicate changed data as it happens) and batch data integration. Data migation also requires the ability to manage the heavy lifting. At Informatica we see 4 primary use cases today and we have developed a multi-tenant set of cloud services to address them:

    1) Data Migration – load data into a SaaS app.
    2) Data Synchronization – this is the process-centric, event-based integration you highlight. Primary use case is CRM – ERP integration today.
    3) Data Cleansing – insight into data quality, address verification and correction, etc.
    4) Data Replication – integration SaaS app data with a data warehouse or general compliance purposes.

    Cloud integration vendors should truly be able to deliver data integration as a service and their architecture should lend itself to the transparency of a trust site that allows customers and prospects to see what kinds of data volumes the on-demand service can manage. (for example: There should also be flexible, hybrid deployment options.

    Informatica Cloud

Leave a Reply

Your email address will not be published. Required fields are marked *