Inside Analysis

Data Ingestion for the Modern Enterprise: New Technologies for Multi-Modal Data Pipelines

Hosted by Eric Kavanagh (The Bloor Group).
Presentation by Wayne Eckerson (The Eckerson Group).
Sponsored by Equalum.

One of the biggest challenges facing organizations looking to modernize their data infrastructures and pipelines is how to ingest data from dozens, if not hundreds, of new and legacy systems and applications. Most want to adopt a streaming-first architecture, but many legacy systems and target applications still work in batch mode. Also, with large volumes of fast-moving data, it’s mandatory to update targets with changes, not refresh the entire database. But not all ingestion methods support change data capture. Finally, reconciling source data and preparing it for consumption requires a significant amount of transformation, aggregation, and integration, which is often done in a separate environment. Ditto for security management, performance tuning, and systems and application monitoring and management.

This webcast will highlight the challenges organizations face when building modern big data and analytic environments. It will emphasize the need for an integrated data platform that handles ingestion, change data capture, transformation, security, and performance management. It will underscore the need for such platforms to span both premises and cloud environments and support both traditional ETL developers with GUI-based tools and modern data engineers who write Python, Scala, and other code. It will propose that the industry needs a “bridge” platform that spans the “old” and “new” worlds of data processing and provides an end-to-end environment that supports all data analytic use cases.



Leave a Reply

Your email address will not be published. Required fields are marked *