How machines learn
to play, win, repeat.

From spotting cats in photos to mastering Go — we'll trace the leap from supervised learning to reinforcement learning, and play a tiny RL game ourselves.

Two ways a machine can learn.

Like studying with flashcards.

Show the model an example, tell it the right answer, and let it adjust. Repeat millions of times with new examples — until it gets the pattern right on cards it has never seen.

Each card = one labeled example. The model's job: predict the label of a new card.

Cat or dog?

Even ChatGPT is just
predicting the next word.

Same recipe as cat-vs-dog. Show the model billions of sentences, hide the next word, ask it to guess, shrink the loss. The only twist — instead of one label, it produces something one piece at a time.

Same \(\min \, \ell(f(x), y)\) — but now \(y\) is a piece of the output (a word, a note, a slice of noise), not a label.

You're the language model.

An LLM picks the next word from a probability distribution — over and over. Tap a word to add it. Watch the sentence grow.

Where's the answer key?

Try to imagine a flashcard set for any of these. You can't — not because nobody tried, but because no human knows the right answer at every moment.

No teacher. No labels. Just try things, see what happens, get rewarded. Sound familiar? That's how you learn most things — and it's where RL lives.

Two characters, talking in a loop.

To do math on bikes, games, and puppies, we need a picture that fits all of them. RL boils every one of those situations down to two characters — an agent and an environment — passing messages back and forth.

The agent's policy π — its strategy — picks an action. The world updates and hands back a new state + a reward, which the agent uses to nudge its policy. Repeat forever. Mathematicians call this a Markov Decision Process — but you can just call it a loop.

A number every step.
Add them up.

After every action, the world hands back one number — the score for that step. Sometimes it's a big jackpot at the end (you won the game). Sometimes it's a little something every step (a tasty scoop of ice cream). Either way, the agent's only job: make the total as big as possible.

Explore or exploit?

You walk into an ice cream shop with 31 flavors. Mint chip is your reliable favorite. Do you order it again — or finally try the one with the weird name?

Every RL agent has to balance these. Too much exploring = waste. Too much exploiting = miss the best.

The game everyone said
computers couldn't win.

\(10^{170}\)

POSSIBLE BOARD POSITIONS

2,500+

YEARS OF HUMAN STUDY

2016

YEAR ALPHAGO BEAT WORLD CHAMP LEE SEDOL

More positions than atoms in the universe. You can't memorize Go — you have to understand it.

How AlphaGo learned, in three stages.

Stage 2 is the magic. Without an answer key, RL let the system surpass every human teacher it had.

The move
no human would have made.

It's not a human move. I've never seen a human play this move. So beautiful.

— Fan Hui, professional Go player, watching live

Estimated probability of a human playing it: 1 in 10,000. AlphaGo discovered it through self-play — no teacher could have shown it this move. Lee Sedol stared at the board for 12 minutes.

Twelve minutes.

How can a machine play a move
no human ever played?

AlphaGo started by copying 30 million pro moves. So where did Move 37 come from? Talk to a neighbor — guess before we answer.

Meet ε-greedy.

How would a computer play that game? Here's the simplest RL algorithm that works — it's just a coin flip and a running average. (ε is the Greek letter "epsilon" — think of it as "how curious am I?")

ε-greedy plays your game.

Same three boxes. Same hidden means. Same 20 pulls. Each turn we roll a dice 🎲 — if it lands on 1, ε-greedy explores (random box). On 2–6, it exploits (best so far). That's ε = 1/6.

What do you think
of ε-greedy?

You watched it play. You compared it to your own strategy. Now turn to a partner — or shout it out:

Match each game word
to its RL name.

Tap a phrase on the left, then tap its match on the right. Get all 6 right and you've spoken the language of RL.

Every part of the loop has a hard problem.

The same loop you saw earlier — agent, policy, action, state, reward, environment. Now tap any piece to see what the next 7 lectures will tackle. For each, ask yourself: what would YOU try?

Same recipe.
Now for language.

Remember predict the next word from earlier? That's just step one. Modern LLMs follow the same two-stage recipe as AlphaGo — first imitate, then RL.

Same idea, different game. RL is what turns a fluent imitator into something that can surpass its teacher.

Four ideas to take with you.