Artwork

Machine Learning Street Talk (MLST)에서 제공하는 콘텐츠입니다. 에피소드, 그래픽, 팟캐스트 설명을 포함한 모든 팟캐스트 콘텐츠는 Machine Learning Street Talk (MLST) 또는 해당 팟캐스트 플랫폼 파트너가 직접 업로드하고 제공합니다. 누군가가 귀하의 허락 없이 귀하의 저작물을 사용하고 있다고 생각되는 경우 여기에 설명된 절차를 따르실 수 있습니다 https://ko.player.fm/legal.
Player FM -팟 캐스트 앱
Player FM 앱으로 오프라인으로 전환하세요!

Jonas Hübotter (ETH) - Test Time Inference

1:45:56
 
공유
 

Manage episode 453005807 series 2803422
Machine Learning Street Talk (MLST)에서 제공하는 콘텐츠입니다. 에피소드, 그래픽, 팟캐스트 설명을 포함한 모든 팟캐스트 콘텐츠는 Machine Learning Street Talk (MLST) 또는 해당 팟캐스트 플랫폼 파트너가 직접 업로드하고 제공합니다. 누군가가 귀하의 허락 없이 귀하의 저작물을 사용하고 있다고 생각되는 경우 여기에 설명된 절차를 따르실 수 있습니다 https://ko.player.fm/legal.

Jonas Hübotter, PhD student at ETH Zurich's Institute for Machine Learning, discusses his groundbreaking research on test-time computation and local learning. He demonstrates how smaller models can outperform larger ones by 30x through strategic test-time computation and introduces a novel paradigm combining inductive and transductive learning approaches.

Using Bayesian linear regression as a surrogate model for uncertainty estimation, Jonas explains how models can efficiently adapt to specific tasks without massive pre-training. He draws an analogy to Google Earth's variable resolution system to illustrate dynamic resource allocation based on task complexity.

The conversation explores the future of AI architecture, envisioning systems that continuously learn and adapt beyond current monolithic models. Jonas concludes by proposing hybrid deployment strategies combining local and cloud computation, suggesting a future where compute resources are allocated based on task complexity rather than fixed model size.

This research represents a significant shift in machine learning, prioritizing intelligent resource allocation and adaptive learning over traditional scaling approaches.

SPONSOR MESSAGES:

CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.

https://centml.ai/pricing/

Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/

Transcription, references and show notes PDF download:

https://www.dropbox.com/scl/fi/cxg80p388snwt6qbp4m52/JonasFinal.pdf?rlkey=glk9mhpzjvesanlc14rtpvk4r&st=6qwi8n3x&dl=0

Jonas Hübotter

https://jonhue.github.io/

https://scholar.google.com/citations?user=pxi_RkwAAAAJ

Transductive Active Learning: Theory and Applications (NeurIPS 2024)

https://arxiv.org/pdf/2402.15898

EFFICIENTLY LEARNING AT TEST-TIME: ACTIVE FINE-TUNING OF LLMS (SIFT)

https://arxiv.org/pdf/2410.08020

TOC:

1. Test-Time Computation Fundamentals

[00:00:00] Intro

[00:03:10] 1.1 Test-Time Computation and Model Performance Comparison

[00:05:52] 1.2 Retrieval Augmentation and Machine Teaching Strategies

[00:09:40] 1.3 In-Context Learning vs Fine-Tuning Trade-offs

2. System Architecture and Intelligence

[00:15:58] 2.1 System Architecture and Intelligence Emergence

[00:23:22] 2.2 Active Inference and Constrained Agency in AI

[00:29:52] 2.3 Evolution of Local Learning Methods

[00:32:05] 2.4 Vapnik's Contributions to Transductive Learning

3. Resource Optimization and Local Learning

[00:34:35] 3.1 Computational Resource Allocation in ML Models

[00:35:30] 3.2 Historical Context and Traditional ML Optimization

[00:37:55] 3.3 Variable Resolution Processing and Active Inference in ML

[00:43:01] 3.4 Local Learning and Base Model Capacity Trade-offs

[00:48:04] 3.5 Active Learning vs Local Learning Approaches

4. Information Retrieval and Model Interpretability

[00:51:08] 4.1 Information Retrieval and Nearest Neighbor Limitations

[01:03:07] 4.2 Model Interpretability and Surrogate Models

[01:15:03] 4.3 Bayesian Uncertainty Estimation and Surrogate Models

5. Distributed Systems and Deployment

[01:23:56] 5.1 Memory Architecture and Controller Systems

[01:28:14] 5.2 Evolution from Static to Distributed Learning Systems

[01:38:03] 5.3 Transductive Learning and Model Specialization

[01:41:58] 5.4 Hybrid Local-Cloud Deployment Strategies

  continue reading

233 에피소드

Artwork
icon공유
 
Manage episode 453005807 series 2803422
Machine Learning Street Talk (MLST)에서 제공하는 콘텐츠입니다. 에피소드, 그래픽, 팟캐스트 설명을 포함한 모든 팟캐스트 콘텐츠는 Machine Learning Street Talk (MLST) 또는 해당 팟캐스트 플랫폼 파트너가 직접 업로드하고 제공합니다. 누군가가 귀하의 허락 없이 귀하의 저작물을 사용하고 있다고 생각되는 경우 여기에 설명된 절차를 따르실 수 있습니다 https://ko.player.fm/legal.

Jonas Hübotter, PhD student at ETH Zurich's Institute for Machine Learning, discusses his groundbreaking research on test-time computation and local learning. He demonstrates how smaller models can outperform larger ones by 30x through strategic test-time computation and introduces a novel paradigm combining inductive and transductive learning approaches.

Using Bayesian linear regression as a surrogate model for uncertainty estimation, Jonas explains how models can efficiently adapt to specific tasks without massive pre-training. He draws an analogy to Google Earth's variable resolution system to illustrate dynamic resource allocation based on task complexity.

The conversation explores the future of AI architecture, envisioning systems that continuously learn and adapt beyond current monolithic models. Jonas concludes by proposing hybrid deployment strategies combining local and cloud computation, suggesting a future where compute resources are allocated based on task complexity rather than fixed model size.

This research represents a significant shift in machine learning, prioritizing intelligent resource allocation and adaptive learning over traditional scaling approaches.

SPONSOR MESSAGES:

CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.

https://centml.ai/pricing/

Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/

Transcription, references and show notes PDF download:

https://www.dropbox.com/scl/fi/cxg80p388snwt6qbp4m52/JonasFinal.pdf?rlkey=glk9mhpzjvesanlc14rtpvk4r&st=6qwi8n3x&dl=0

Jonas Hübotter

https://jonhue.github.io/

https://scholar.google.com/citations?user=pxi_RkwAAAAJ

Transductive Active Learning: Theory and Applications (NeurIPS 2024)

https://arxiv.org/pdf/2402.15898

EFFICIENTLY LEARNING AT TEST-TIME: ACTIVE FINE-TUNING OF LLMS (SIFT)

https://arxiv.org/pdf/2410.08020

TOC:

1. Test-Time Computation Fundamentals

[00:00:00] Intro

[00:03:10] 1.1 Test-Time Computation and Model Performance Comparison

[00:05:52] 1.2 Retrieval Augmentation and Machine Teaching Strategies

[00:09:40] 1.3 In-Context Learning vs Fine-Tuning Trade-offs

2. System Architecture and Intelligence

[00:15:58] 2.1 System Architecture and Intelligence Emergence

[00:23:22] 2.2 Active Inference and Constrained Agency in AI

[00:29:52] 2.3 Evolution of Local Learning Methods

[00:32:05] 2.4 Vapnik's Contributions to Transductive Learning

3. Resource Optimization and Local Learning

[00:34:35] 3.1 Computational Resource Allocation in ML Models

[00:35:30] 3.2 Historical Context and Traditional ML Optimization

[00:37:55] 3.3 Variable Resolution Processing and Active Inference in ML

[00:43:01] 3.4 Local Learning and Base Model Capacity Trade-offs

[00:48:04] 3.5 Active Learning vs Local Learning Approaches

4. Information Retrieval and Model Interpretability

[00:51:08] 4.1 Information Retrieval and Nearest Neighbor Limitations

[01:03:07] 4.2 Model Interpretability and Surrogate Models

[01:15:03] 4.3 Bayesian Uncertainty Estimation and Surrogate Models

5. Distributed Systems and Deployment

[01:23:56] 5.1 Memory Architecture and Controller Systems

[01:28:14] 5.2 Evolution from Static to Distributed Learning Systems

[01:38:03] 5.3 Transductive Learning and Model Specialization

[01:41:58] 5.4 Hybrid Local-Cloud Deployment Strategies

  continue reading

233 에피소드

Tous les épisodes

×
 
Loading …

플레이어 FM에 오신것을 환영합니다!

플레이어 FM은 웹에서 고품질 팟캐스트를 검색하여 지금 바로 즐길 수 있도록 합니다. 최고의 팟캐스트 앱이며 Android, iPhone 및 웹에서도 작동합니다. 장치 간 구독 동기화를 위해 가입하세요.

 

빠른 참조 가이드

탐색하는 동안 이 프로그램을 들어보세요.
재생