Making Analytical APIs Fast With Tinybird

Data Engineering Podcast

Tobias Macey에서 제공하는 콘텐츠입니다. 에피소드, 그래픽, 팟캐스트 설명을 포함한 모든 팟캐스트 콘텐츠는 Tobias Macey 또는 해당 팟캐스트 플랫폼 파트너가 직접 업로드하고 제공합니다. 누군가가 귀하의 허락 없이 귀하의 저작물을 사용하고 있다고 생각되는 경우 여기에 설명된 절차를 따르실 수 있습니다 https://ko.player.fm/legal.

3y ago 54:23

MP3•에피소드 홈

저장한 시리즈 ("피드 비활성화" status)

When? This feed was archived on January 17, 2023 09:15 (1+ y ago). Last successful fetch was on December 12, 2022 17:37 (1+ y ago)

Why? 피드 비활성화 status. 잠시 서버에 문제가 발생해 팟캐스트를 불러오지 못합니다.

What now? You might be able to find a more up-to-date version using the search function. This series will no longer be checked for updates. If you believe this to be in error, please check if the publisher's feed link below is valid and contact support to request the feed be restored or if you have any other concerns about this.

Summary

Building an API for real-time data is a challenging project. Making it robust, scalable, and fast is a full time job. The team at Tinybird wants to make it easy to turn a continuous stream of data into a production ready API or data product. In this episode CEO Jorge Sancha explains how they have architected their system to handle high data throughput and fast response times, and why they have invested heavily in Clickhouse as the core of their platform. This is a great conversation about the challenges of building a maintainable business from a technical and product perspective.

Announcements

Hello and welcome to the Data Engineering Podcast, the show about modern data management
When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their managed Kubernetes platform it’s now even easier to deploy and scale your workflows, or try out the latest Helm charts from tools like Pulsar and Pachyderm. With simple pricing, fast networking, object storage, and worldwide data centers, you’ve got everything you need to run a bulletproof data platform. Go to dataengineeringpodcast.com/linode today and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
RudderStack’s smart customer data pipeline is warehouse-first. It builds your customer data warehouse and your identity graph on your data warehouse, with support for Snowflake, Google BigQuery, Amazon Redshift, and more. Their SDKs and plugins make event streaming easy, and their integrations with cloud applications like Salesforce and ZenDesk help you go beyond event streaming. With RudderStack you can use all of your customer data to answer more difficult questions and then send those insights to your whole customer data stack. Sign up free at dataengineeringpodcast.com/rudder today.
Ascend.io — recognized as a 2021 Gartner Cool Vendor in Enterprise AI Operationalization and Engineering—empowers data teams to to build, scale, and operate declarative data pipelines with 95% less code and zero maintenance. Connect to any data source using Ascend’s new flex code data connectors, rapidly iterate on transformations and send data to any destination in a fraction of the time it traditionally takes—just ask companies like Harry’s, HNI, and Mayvenn. Sound exciting? Come join the team! We’re hiring data engineers, so head on over to dataengineeringpodcast.com/ascend and check out our careers page to learn more.
Your host is Tobias Macey and today I’m interviewing Jorge Sancha about Tinybird, a platform to easily build analytical APIs for real-time data

Interview

Introduction
How did you get involved in the area of data management?
Can you start by describing what you are building at Tinybird and the story behind it?
What are some of the types of use cases that your customers are focused on?
What are the areas of complexity that come up when building analytical APIs that are often overlooked when first designing a system to operate on and expose real-time data?
- What are the supporting systems that are necessary and useful for operating this kind of system which contribute to the overall time and engineering cost beyond the baseline functionality?
How is the Tinybird platform architected?
- How have the goals and implementation of Tinybird changed or evolved since you first began building it?
What was your criteria for selecting the core building block of your platform, and how did that lead to your choice to build on top of Clickhouse?
What are some of the sharp edges that you have run into while operating Clickhouse?
- What are some of the custom tools or systems that you have built to help deal with them?
What are some of the performance challenges that an API built with Tinybird might run into?
- What are the considerations that users should be aware of to avoid introducing performance issues?
How do you handle multi-tenancy in your platform? (e.g. separate clusters, in-database quotas, etc.)
For users of Tinybird, can you talk through the workflow of getting it integrated into their platform and designing an API from their data?
What are some of the most interesting, innovative, or unexpected ways that you have seen Tinybird used?
What are the most interesting, unexpected, or challenging lessons that you have learned while building and growing Tinybird?
When is Tinybird the wrong choice?
What do you have planned for the future of the product and business?

Contact Info

@jorgesancha on Twitter
LinkedIn
jorgesancha on GitHub

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Links

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

351 에피소드

#Tech #Tobias Macey #Machine Learning