))
Ep2. Data Engineering at Uber and Lyft
Manage episode 283145520 series 2858756
Thomas Wang에서 제공하는 콘텐츠입니다. 에피소드, 그래픽, 팟캐스트 설명을 포함한 모든 팟캐스트 콘텐츠는 Thomas Wang 또는 해당 팟캐스트 플랫폼 파트너가 직접 업로드하고 제공합니다. 누군가가 귀하의 허락 없이 귀하의 저작물을 사용하고 있다고 생각되는 경우 여기에 설명된 절차를 따르실 수 있습니다 https://ko.player.fm/legal.
这一期节目我们和嘉宾泉来聊一聊Data Engineering，在Uber和Lyft的应用和工作体验，对于行业发展的反思与展望，以及对于入行Data Engineering的建议。只要你对Large Scale Data Engineering感兴趣，本期节目都会对你有帮助 :)
Timestamps:
- 00:00:00 Intro
- 00:00:40 Data Infrastructure vs Data Engineer
- 00:02:45 Data Engineering 领域介绍
- 00:09:00 Pipeline权限设置
- 00:11:00 Pipeline在Uber的使用场景
- 00:11:45 如何追踪Data Owner
- 00:11:52 如何保护用户数据隐私
- 00:17:16 Data Infra转到Data Engineer的日常工作
- 00:21:44 Data Quality Tier
- 00:32:52 Uber vs Lyft 数据量级和迭代的区别
- 00:35:46 COVID影响
- 00:38:40 对颠覆性产业和商业模式的反思
- 00:40:27 规模效应和数据
- 00:41:26 ML Infra vs Data Infra
- 00:44:47 对新入行Data Engineering的建议
- 00:47:28 联系方式
Links:
- Airflow: https://airflow.apache.org/
- Piper: https://eng.uber.com/no-code-workflow-orchestrator/
- uWorc: https://eng.uber.com/no-code-workflow-orchestrator/
- Amundsen: https://eng.lyft.com/open-sourcing-amundsen-a-data-discovery-and-metadata-platform-2282bb436234
- PII: https://en.wikipedia.org/wiki/Personal_data
- GDPR: https://en.wikipedia.org/wiki/General_Data_Protection_Regulation
- ETL: https://en.wikipedia.org/wiki/Extract,_transform,_load
- 联系泉来：mail@quanlai.li
联系方式：
- 官网: eng.cafe/
- 微信公众号: Eng Cafe
- Twitter: @engcafefm
- Youtube: Eng Cafe
- Email: [email protected]
收听渠道:
16 에피소드
공유
