This post opens a series on reinforcement learning. The plan is to build up the foundations along David Silver’s course, reach PPO, and continue toward more recent methods from there. First, where to get the materials, free.
The book (free)
Sutton & Barto’s Reinforcement Learning: An Introduction (2nd edition, MIT Press 2018) is the field’s standard text, and the authors keep the full PDF free.
- Free PDF: incompleteideas.net/book/RLbook2020.pdf
- Book page, with code, solutions, and errata: incompleteideas.net/book/the-book-2nd.html
The lectures (free)
David Silver’s UCL course (2015) is the classic video introduction: ten lectures, and this series follows its chapter structure.
- Course page and slides: davidsilver.uk/teaching
Lecture 1:
Where it is headed
From the textbook fundamentals, through policy-gradient methods, to PPO (Schulman et al. 2017), pairing the reading with the lectures. PPO is not the newest idea in the field. It is the workhorse much of today’s RL still leans on, which makes it a natural checkpoint before the more recent methods, not the finish line.