Player FM 앱으로 오프라인으로 전환하세요!
Mixture-of-Experts and Trends in Large-Scale Language Modeling with Irwan Bello - #569
Manage episode 326563385 series 2355587
Today we’re joined by Irwan Bello, formerly a research scientist at Google Brain, and now on the founding team at a stealth AI startup. We begin our conversation with an exploration of Irwan’s recent paper, Designing Effective Sparse Expert Models, which acts as a design guide for building sparse large language model architectures. We discuss mixture of experts as a technique, the scalability of this method, and it's applicability beyond NLP tasks the data sets this experiment was benchmarked against. We also explore Irwan’s interest in the research areas of alignment and retrieval, talking through interesting lines of work for each area including instruction tuning and direct alignment.
The complete show notes for this episode can be found at twimlai.com/go/569
699 에피소드
Mixture-of-Experts and Trends in Large-Scale Language Modeling with Irwan Bello - #569
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Manage episode 326563385 series 2355587
Today we’re joined by Irwan Bello, formerly a research scientist at Google Brain, and now on the founding team at a stealth AI startup. We begin our conversation with an exploration of Irwan’s recent paper, Designing Effective Sparse Expert Models, which acts as a design guide for building sparse large language model architectures. We discuss mixture of experts as a technique, the scalability of this method, and it's applicability beyond NLP tasks the data sets this experiment was benchmarked against. We also explore Irwan’s interest in the research areas of alignment and retrieval, talking through interesting lines of work for each area including instruction tuning and direct alignment.
The complete show notes for this episode can be found at twimlai.com/go/569
699 에피소드
すべてのエピソード
×플레이어 FM에 오신것을 환영합니다!
플레이어 FM은 웹에서 고품질 팟캐스트를 검색하여 지금 바로 즐길 수 있도록 합니다. 최고의 팟캐스트 앱이며 Android, iPhone 및 웹에서도 작동합니다. 장치 간 구독 동기화를 위해 가입하세요.