Developing Feast, the Leading Open Source Feature Store, with Willem Pienaar (Gojek, Tecton)

Machine Learning Engineered

Willem Pienaar is the co-creator of Feast, the leading open source feature store, which he leads the development of as a tech lead at Tecton. Previously, he led the ML platform team at Gojek, a super-app in Southeast Asia.

Learn more:

https://twitter.com/willpienaar

https://feast.dev/

Every Thursday I send out the most useful things I’ve learned, curated specifically for the busy machine learning engineer. Sign up here: https://www.cyou.ai/newsletter

Follow Charlie on Twitter: https://twitter.com/CharlieYouAI

Subscribe to ML Engineered: https://mlengineered.com/listen

Comments? Questions? Submit them here: http://bit.ly/mle-survey

Take the Giving What We Can Pledge: https://www.givingwhatwecan.org/

Timestamps:

02:15 How Willem got started in computer science

03:40 Paying for college by starting an ISP

05:25 Willem's experience creating Gojek's ML platform

21:45 Issues faced that led to the creation of Feast

26:45 Lessons learned building Feast

33:45 Integrating Feast with data quality monitoring tools

40:10 What it looks like for a team to adopt Feast

44:20 Feast's current integrations and future roadmap

46:05 How a data scientist would use Feast when creating a model

49:40 How the feature store pattern handles DAGs of models

52:00 Priorities for a startup's data infrastructure

55:00 Integrating with Amundsen, Lyft's data catalog

57:15 The evolution of data and MLOps tool standards for interoperability

01:01:35 Other tools in the modern data stack

01:04:30 The interplay between open and closed source offerings

Links:

Feast's Github

Gojek Data Science Blog

Data Build Tool (DBT)

Tensorflow Data Validation (TFDV)

A State of Feast

Google BigQuery

Lyft Amundsen

Cortex

Kubeflow

MLFlow