ETL is dead; long-live streams Neha Narkhede, Co-founder & CTO, Confluent “ Data and data systems have really changed in the past decade Old world: Two popular locations for data DB DB DWH DB DB Operational databases Relational data warehouse “ Several recent data trends are driving a dramatic change in the ETL architecture #1: Single-server databases are replaced “ by a myriad of distributed data platforms that operate at company-wide scale #2: Many more ty“pes of data sources beyond transactional data - logs, sensors, metrics... #3: Stream da“ta is increasingly ubiquitous; need for faster processing than daily The end result?“ This is what data integration ends up looking like in practice App App App App cache cache MQ MQ monitoring security search DWH Hadoop App App App App cache cache MQ MQ monitoring security search DWH Hadoop A giant mess