))
From Data Mesh to Lake House: Revolutionizing Metadata with Lakekeeper
Manage episode 472606309 series 3594857
Kostas Pardalis, Nitay Joffe에서 제공하는 콘텐츠입니다. 에피소드, 그래픽, 팟캐스트 설명을 포함한 모든 팟캐스트 콘텐츠는 Kostas Pardalis, Nitay Joffe 또는 해당 팟캐스트 플랫폼 파트너가 직접 업로드하고 제공합니다. 누군가가 귀하의 허락 없이 귀하의 저작물을 사용하고 있다고 생각되는 경우 여기에 설명된 절차를 따르실 수 있습니다 https://ko.player.fm/legal.
Summary
In this episode, Viktor Kessler shares his journey and insights from his extensive experience in data management—from building risk management systems and data warehouses to working as a solutions architect at MongoDB and Dremio, and now co-founding a startup.
Initially exploring data mesh concepts, Viktor explains how real-world challenges—such as the disconnect between technical data models and business needs, inconsistent definitions across departments, and the difficulty in managing actionable metadata—led him and his co-founder to pivot toward building a lake house solution.
His startup is developing Lakekeeper, an open source REST catalog for Apache Iceberg, which aims to bridge the gap between decentralized data production and centralized metadata management.
The conversation also delves into the evolution of data catalogs, the necessity for self-service analytics, and how creating consumption-ready data products can transform data functions from cost centers into profit centers.
Finally, Viktor outlines ways for interested listeners to get involved with the Lakekeeper community through GitHub, upcoming meetups, and a dedicated Discord channel.
Chapters
00:00 Introduction to Viktor Kessler and His Journey
04:57 Transitioning from Data Mesh to Lake House
09:15 Understanding Data Mesh: Pain Points and Solutions
13:47 The Role of Metadata in Data Management
18:16 The Evolution of Catalogs and Metadata Management
28:14 Stabilizing the Consumption Pipeline
31:18 Centralizing Metadata for Decentralized Organizations
37:09 Bridging the Gap: Technical and Business Perspectives
43:17 Rethinking Data Products and Consumption
50:45 Finding Balance: Control and Flexibility in Data Management
22 에피소드
공유
