Episode 8: The New Parser

core.py

The suspense was killing us! OK, the old parser was then... but what about NOW? We're finally answering this question... in more detail than you dared to ask for. PEG, memoization, funky secrets, and how a certain auto-formatter self-inflicted an existential crisis on itself. It's all there, told in barely 100 minutes! Can you believe it?

# Timestamps

(00:00:00)  INTRO

(00:00:54)  PART 1: What even is PEG?

(00:04:02)  You can't prove anything!

(00:05:03)  What's a "parsing expression"?

(00:08:23)  Our old LL1 parser wasn't doing its job

(00:09:37)  "Soft keywords" in LL1: A Horror Story

(00:13:16)  PART 2: How PEG was adopted by Python

(00:17:10)  Why not LALR?

(00:22:11)  The PEG paper wasn't enough either, if we're honest

(00:26:26)  Less obvious advantages of the new parser

(00:31:28)  Black is stuck with LL1, can it cope?

(00:36:24)  Hedging against Łukasz, the bringer of doom

(00:41:14)  PART 3: How does the PEG parser of CPython work?

(00:44:30)  Pedantic Pablo on "exponential"

(00:45:14)  Fresh news from literally yesterday last week

(00:46:39)  Pedantic Pablo on "infinite"

(00:47:32)  Memoization in the PEG parser

(00:50:41)  Parse once, and if it fails, try again!

(00:52:14)  How to model a grammar of programming mistakes?

(00:56:36)  Why is there C code in my grammar file?

(00:59:57)  Bro, do you even lift?

(01:01:45)  How soft keywords work today: it's not free lunch

(01:04:29)  Funky grammar secrets

(01:09:07)  PART 4: PR OF THE WEEK

(01:09:15)  audioop.c license shenanigans

(01:14:56)  The secret profiler inside CPython (tests)

(01:22:45)  PART 5: WHAT'S GOING ON IN CPYTHON?

(01:23:30)  Free-threading changes

(01:28:15)  Faster Python changes

(01:35:39)  End of an era: docs get rid of Python 2 migration info

(01:36:45)  Python --help output is now nicer

(01:38:43) SQLite as a dbm backend

(01:41:08)  OUTRO