최고 Arx 팟캐스트 (2024)

1
[QA] Add-it: Training-Free Object Insertion in Images With Pretrained Diffusion Models 6:57

34m ago6:57

6:57

Add-it is a training-free approach for semantic image editing that seamlessly integrates objects into images using a weighted extended-attention mechanism, achieving state-of-the-art results without fine-tuning. https://arxiv.org/abs//2411.07232 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcast…

1
Add-it: Training-Free Object Insertion in Images With Pretrained Diffusion Models 19:23

34m ago19:23

19:23

Add-it is a training-free approach for semantic image editing that seamlessly integrates objects into images using a weighted extended-attention mechanism, achieving state-of-the-art results without fine-tuning. https://arxiv.org/abs//2411.07232 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcast…

1
[QA] Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models 7:19

41m ago7:19

7:19

The SPA framework enhances user experience by generating diverse, high-quality responses from foundation models using synthetic data and data attribution methods, improving performance in code generation and natural language tasks. https://arxiv.org/abs//2411.06722 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_…

1
Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models 16:56

42m ago16:56

16:56

The SPA framework enhances user experience by generating diverse, high-quality responses from foundation models using synthetic data and data attribution methods, improving performance in code generation and natural language tasks. https://arxiv.org/abs//2411.06722 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_…

1
[QA] Aioli: A unified optimization framework for language model data mixing 6:40

1d ago6:40

6:40

1
Aioli: A unified optimization framework for language model data mixing 29:42

1d ago29:42

29:42

1
[QA] BALANCING PIPELINE PARALLELISM WITH VOCABULARY PARALLELISM 8:21

1d ago8:21

8:21

This paper addresses imbalanced computation and memory in pipeline parallelism for large language models by partitioning vocabulary layers, reducing communication barriers, and achieving improved throughput and memory balance. https://arxiv.org/abs//2411.05288 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_paper…

1
BALANCING PIPELINE PARALLELISM WITH VOCABULARY PARALLELISM 20:22

1d ago20:22

20:22

This paper addresses imbalanced computation and memory in pipeline parallelism for large language models by partitioning vocabulary layers, reducing communication barriers, and achieving improved throughput and memory balance. https://arxiv.org/abs//2411.05288 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_paper…

1
[QA] Can Transformers Smell Like Humans? 7:34

19h ago7:34

7:34

This study explores whether pre-trained transformer models of chemical structures align with human olfactory perception, demonstrating their ability to predict expert labels and human ratings of odorants. https://arxiv.org/abs//2411.03038 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: http…

1
Can Transformers Smell Like Humans? 18:01

19h ago18:01

18:01

This study explores whether pre-trained transformer models of chemical structures align with human olfactory perception, demonstrating their ability to predict expert labels and human ratings of odorants. https://arxiv.org/abs//2411.03038 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: http…

1
[QA] Mixtures of In-Context Learners 7:11

19h ago7:11

7:11

The paper introduces Mixtures of In-Context Learners (MOICL), enhancing in-context learning by optimizing demonstration subsets, improving performance, and reducing memory usage in Transformer LLMs. https://arxiv.org/abs//2411.02830 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://po…

1
Mixtures of In-Context Learners 15:37

20h ago15:37

15:37

The paper introduces Mixtures of In-Context Learners (MOICL), enhancing in-context learning by optimizing demonstration subsets, improving performance, and reducing memory usage in Transformer LLMs. https://arxiv.org/abs//2411.02830 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://po…

1
[QA] How Far Is Video Generation from World Model: A Physical Law Perspective 8:58

2d ago8:58

8:58

OpenAI's Sora evaluates video generation models' ability to learn physical laws, revealing limitations in generalization and suggesting scaling alone isn't enough for uncovering fundamental principles. https://arxiv.org/abs//2411.02385 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https:/…

1
How Far Is Video Generation from World Model: A Physical Law Perspective 27:51

2d ago27:51

27:51

OpenAI's Sora evaluates video generation models' ability to learn physical laws, revealing limitations in generalization and suggesting scaling alone isn't enough for uncovering fundamental principles. https://arxiv.org/abs//2411.02385 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https:/…

1
[QA] ADOPT: Modified Adam Can Converge with Any with the Optimal Rate 7:47

2d ago7:47

7:47

The paper introduces ADOPT, a new adaptive gradient method that resolves Adam's non-convergence issue without bounded noise assumptions, demonstrating superior performance across various deep learning tasks. https://arxiv.org/abs//2411.02853 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: h…

1
ADOPT: Modified Adam Can Converge with Any with the Optimal Rate 15:16

2d ago15:16

15:16

The paper introduces ADOPT, a new adaptive gradient method that resolves Adam's non-convergence issue without bounded noise assumptions, demonstrating superior performance across various deep learning tasks. https://arxiv.org/abs//2411.02853 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: h…

1
[QA] Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks? 7:24

3d ago7:24

7:24

This study evaluates 17 leading Large Language Models' abilities in complex information retrieval, revealing many are thread-safe but have shorter effective context limits than supported lengths. https://arxiv.org/abs//2411.05000 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podca…

1
Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks? 14:03

3d ago14:03

14:03

This study evaluates 17 leading Large Language Models' abilities in complex information retrieval, revealing many are thread-safe but have shorter effective context limits than supported lengths. https://arxiv.org/abs//2411.05000 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podca…

1
[QA] Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models 7:53

3d ago7:53

7:53

https://arxiv.org/abs//2411.04996 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers --- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/supp…

1
Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models 41:18

3d ago41:18

41:18

https://arxiv.org/abs//2411.04996 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016 Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers --- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/supp…

1
[QA] Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex 10:52

4d ago10:52

10:52

The study reveals that task-specific representation learning continues in mice's piriform cortex during overtraining, enhancing classification accuracy despite behavior plateauing, suggesting hidden learning mechanisms at play. https://arxiv.org/abs//2411.03541 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_pape…

1
Do Mice Grok? Glimpses of Hidden Progress During Overtraining in Sensory Cortex 15:09

4d ago15:09

15:09

The study reveals that task-specific representation learning continues in mice's piriform cortex during overtraining, enhancing classification accuracy despite behavior plateauing, suggesting hidden learning mechanisms at play. https://arxiv.org/abs//2411.03541 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_pape…

1
[QA] How Transformers Solve Propositional Logic Problems: A Mechanistic Analysis 7:22

4d ago7:22

7:22

This study explores how transformers, both small and large, perform complex logical reasoning, identifying key circuits and mechanisms involved in planning and reasoning through a synthetic propositional logic problem. https://arxiv.org/abs//2411.04105 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple …

1
How Transformers Solve Propositional Logic Problems: A Mechanistic Analysis 22:34

4d ago22:34

22:34

This study explores how transformers, both small and large, perform complex logical reasoning, identifying key circuits and mechanisms involved in planning and reasoning through a synthetic propositional logic problem. https://arxiv.org/abs//2411.04105 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple …

1
[QA] Discovering Data Structures: Nearest Neighbor Search and Beyond 7:59

5d ago7:59

7:59

We present a framework for end-to-end learning of data structures, optimizing query and space complexity, applied to nearest neighbor search and frequency estimation in data streams. https://arxiv.org/abs//2411.03253 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com…

1
Discovering Data Structures: Nearest Neighbor Search and Beyond 28:18

5d ago28:18

28:18

We present a framework for end-to-end learning of data structures, optimizing query and space complexity, applied to nearest neighbor search and frequency estimation in data streams. https://arxiv.org/abs//2411.03253 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com…

1
[QA] BrainBits: How Much of the Brain are Generative Reconstruction Methods Using? 7:36

5d ago7:36

7:36

The paper examines factors influencing stimulus reconstruction fidelity, revealing that powerful generative models can mislead interpretations of neural signal extraction effectiveness. It proposes improved evaluation metrics for reconstruction methods. https://arxiv.org/abs//2411.02783 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://…

1
BrainBits: How Much of the Brain are Generative Reconstruction Methods Using? 15:29

5d ago15:29

15:29

The paper examines factors influencing stimulus reconstruction fidelity, revealing that powerful generative models can mislead interpretations of neural signal extraction effectiveness. It proposes improved evaluation metrics for reconstruction methods. https://arxiv.org/abs//2411.02783 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://…

1
[QA] Adapting Language Models via Token Translation 8:13

8d ago8:13

8:13

Sparse Sinkhorn Token Translation (S2T2) improves text compression and inference in new domains by training tailored tokenizers and enabling effective token translation, enhancing performance in language models. https://arxiv.org/abs//2411.00593 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcast…

1
Adapting Language Models via Token Translation 9:33

7d ago9:33

9:33

Sparse Sinkhorn Token Translation (S2T2) improves text compression and inference in new domains by training tailored tokenizers and enabling effective token translation, enhancing performance in language models. https://arxiv.org/abs//2411.00593 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcast…

1
[QA] Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models 8:29

7d ago8:29

8:29

Specialized Sparse Autoencoders (SSAEs) enhance interpretability of foundation models by effectively capturing rare concepts, improving classification accuracy, and revealing insights into subdomain representations. https://arxiv.org/abs//2411.00743 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Pod…

1
Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models 26:54

7d ago26:54

26:54

Specialized Sparse Autoencoders (SSAEs) enhance interpretability of foundation models by effectively capturing rare concepts, improving classification accuracy, and revealing insights into subdomain representations. https://arxiv.org/abs//2411.00743 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Pod…

1
[QA] Tokenformer: Rethinking Transformer Scaling with Tokenized Model Parameters 7:51

9d ago7:51

7:51

Tokenformer introduces a scalable architecture that enhances Transformers' efficiency by using token-parameter attention, allowing for incremental scaling without retraining, thus reducing computational costs significantly. https://arxiv.org/abs//2410.23168 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers A…

1
Tokenformer: Rethinking Transformer Scaling with Tokenized Model Parameters 19:10

9d ago19:10

19:10

Tokenformer introduces a scalable architecture that enhances Transformers' efficiency by using token-parameter attention, allowing for incremental scaling without retraining, thus reducing computational costs significantly. https://arxiv.org/abs//2410.23168 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers A…

1
[QA] $100K or 100 Days: Trade-offs when Pre-Training with Academic Resources 7:22

9d ago7:22

7:22

This paper challenges the assumption that academic researchers can't pre-train models, providing benchmarks and insights on optimizing GPU resources for efficient model training. https://arxiv.org/abs//2410.23261 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/…

1
$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources 16:51

9d ago16:51

16:51

This paper challenges the assumption that academic researchers can't pre-train models, providing benchmarks and insights on optimizing GPU resources for efficient model training. https://arxiv.org/abs//2410.23261 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple.com/us/…

1
[QA] What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective 7:59

12d ago7:59

7:59

This study analyzes layer-wise gradients in LLMs, revealing that slow thinking enhances learning stability and response correctness, while fast thinking shows larger gradient variations. https://arxiv.org/abs//2410.23743 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple…

1
What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective 15:27

12d ago15:27

15:27

This study analyzes layer-wise gradients in LLMs, revealing that slow thinking enhances learning stability and response correctness, while fast thinking shows larger gradient variations. https://arxiv.org/abs//2410.23743 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.apple…

1
[QA] Tokenformer: Rethinking Transformer Scaling with Tokenized Model Parameters 7:28

13d ago7:28

7:28

Tokenformer introduces a scalable architecture that enhances Transformers' efficiency by treating model parameters as tokens, allowing for flexible scaling without retraining, significantly reducing computational costs. https://arxiv.org/abs//2410.23168 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple…

1
Tokenformer: Rethinking Transformer Scaling with Tokenized Model Parameters 19:38

13d ago19:38

19:38

Tokenformer introduces a scalable architecture that enhances Transformers' efficiency by treating model parameters as tokens, allowing for flexible scaling without retraining, significantly reducing computational costs. https://arxiv.org/abs//2410.23168 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple…

1
[QA] Where Do Large Learning Rates Lead Us? 8:30

14d ago8:30

8:30

This study investigates optimal initial learning rates for neural networks, finding a narrow range enhances generalization by locating high-quality minima and focusing on relevant features, unlike extreme rates. https://arxiv.org/abs//2410.22113 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcast…

1
Where Do Large Learning Rates Lead Us? 28:43

14d ago28:43

28:43

This study investigates optimal initial learning rates for neural networks, finding a narrow range enhances generalization by locating high-quality minima and focusing on relevant features, unlike extreme rates. https://arxiv.org/abs//2410.22113 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcast…

1
[QA] Fourier Head: Helping Large Language Models Learn Complex Probability Distributions 7:10

14d ago7:10

7:10

The paper introduces a Fourier series-based neural network layer to improve continuous token modeling in decision-making and time series tasks, enhancing performance in various benchmarks. https://arxiv.org/abs//2410.22269 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app…

1
Fourier Head: Helping Large Language Models Learn Complex Probability Distributions 13:56

14d ago13:56

13:56

The paper introduces a Fourier series-based neural network layer to improve continuous token modeling in decision-making and time series tasks, enhancing performance in various benchmarks. https://arxiv.org/abs//2410.22269 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts.app…

1
[QA] LoRA vs Full Fine-tuning: An Illusion of Equivalence 7:47

15d ago7:47

7:47

This study analyzes the differences between full fine-tuning and LoRA in large language models, revealing distinct weight matrix structures and generalization behaviors despite similar performance on tasks. https://arxiv.org/abs//2410.21228 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: ht…

1
LoRA vs Full Fine-tuning: An Illusion of Equivalence 13:44

15d ago13:44

13:44

This study analyzes the differences between full fine-tuning and LoRA in large language models, revealing distinct weight matrix structures and generalization behaviors despite similar performance on tasks. https://arxiv.org/abs//2410.21228 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: ht…

1
[QA] Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad? 6:57

16d ago6:57

6:57

Vision-Language Models show promise in reasoning across text and images but struggle with basic visual concepts, revealing significant gaps in their understanding and generalization abilities. https://arxiv.org/abs//2410.19546 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts…

1
Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad? 8:44

16d ago8:44

8:44

Vision-Language Models show promise in reasoning across text and images but struggle with basic visual concepts, revealing significant gaps in their understanding and generalization abilities. https://arxiv.org/abs//2410.19546 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_papers Apple Podcasts: https://podcasts…

1
[QA] Computational Bottlenecks of Training Small-scale Large Language Models 8:10

16d ago8:10

8:10

This study investigates the training behavior and computational requirements of Small-scale Large Language Models (SLMs), focusing on hyperparameters and configurations to enhance efficiency and support low-resource AI research. https://arxiv.org/abs//2410.19456 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_pap…

1
Computational Bottlenecks of Training Small-scale Large Language Models 9:57

16d ago9:57

9:57

This study investigates the training behavior and computational requirements of Small-scale Large Language Models (SLMs), focusing on hyperparameters and configurations to enhance efficiency and support low-resource AI research. https://arxiv.org/abs//2410.19456 YouTube: https://www.youtube.com/@ArxivPapers TikTok: https://www.tiktok.com/@arxiv_pap…

들어볼 가치가 있는 팟캐스트

Arx 팟 캐스트

들어볼 가치가 있는 팟캐스트

빠른 참조 가이드