Skip to content

The curriculum

Fifty lessons · Ten phases · About thirty-seven hours · Nothing installed

Back to the course Glossary


Every lesson on one page, in order, with what you do in each. Each one is a directory holding a README.md, a solve.py you write, a check.py that marks it, and the data it needs. Nothing is installed at any point.

Tick them off as you go. On the website the boxes remember where you are, in your own browser and nowhere else.


Phase 0. Your machine

Five lessons. About two hours. Nothing installed beforehand.

  • 1. The terminal  ·  open a terminal, run three commands, read what comes back, send the answer to a file, and install Python.
  • 2. Files and paths  ·  read a path the way the computer reads it, move around with cd, and make a folder inside a folder.
  • 3. Your first program  ·  write a file, make the computer run it, break it on purpose, and read the error properly.
  • 4. An isolated environment  ·  understand why installing a library can break a project you were not touching, and build the wall that prevents it.
  • 5. Git and your copy  ·  record a version of a folder, see the record, and understand what forking this course actually does.


Phase 1. Teach it to tell two things apart

Five lessons. About four hours. Nothing installed.

  • 1. Words into numbers  ·  read a data file, turn sentences into counts, and find out where the signal in a problem actually lives.
  • 2. A rule by hand  ·  write a rule, measure how often it is right, and learn why that number on its own can be a lie.
  • 3. Let it learn  ·  write a learning loop, watch the mistakes fall, and see the program rediscover what you worked out by hand.
  • 4. Tell the truth  ·  hold data back, score on it, and watch a model you were proud of lose a quarter of its accuracy.
  • 5. When it breaks  ·  read every mistake your model makes, one at a time, and find that they are not all the same mistake.


Phase 2. A network you wrote yourself

Five lessons. About four hours. Still nothing installed.

  • 1. The thing one line cannot do  ·  prove, by exhausting every possibility, that there is a problem no single weighted sum can solve.
  • 2. A second layer  ·  build a two layer network by hand, solve the problem lesson 1 proved was impossible, and see exactly why it works.
  • 3. How wrong, and in which direction  ·  replace the threshold with a curve, measure wrongness as one number, and find the slope of that number against every weight in the network.
  • 4. Backpropagation  ·  compute every gradient in one backward sweep, and check each one against the nudging you already trust.
  • 5. Train it on the messages  ·  train the network on the phase 1 messages, score it honestly, and compare it against everything you have built.


Phase 3. Words that know what they mean

Five lessons. About four hours. Still nothing installed.

  • 1. The vocabulary wall  ·  measure how much of the vocabulary is missing, and then prove that the representation has no idea any two words are related.
  • 2. The company a word keeps  ·  count which words appear near which, use those counts as meaning, and watch it fail in an instructive way.
  • 3. Surprise is the signal  ·  replace every count with how surprising it is, and watch meaning appear out of the same numbers.
  • 4. Fewer numbers that say more  ·  squeeze every profile down to 24 numbers, and find that the words become closer to more meaningful than they were.
  • 5. Does it help?  ·  put the vectors on the real problem, measure honestly, and diagnose a failure properly.


Phase 4. A language model you wrote yourself

Five lessons. About five hours. Still nothing installed.

  • 1. Predicting the next word  ·  build a language model, watch it guess the next word, and measure how often it is right on sentences it has never seen.
  • 2. Never say never  ·  measure a language model properly, watch the measurement come out infinite, and fix it two different ways.
  • 3. Making it talk  ·  generate sentences three ways, and see exactly what a memory of one word can and cannot do.
  • 4. A model that generalises  ·  train a language model on word vectors instead of counts, and get the first win of this entire course.
  • 5. What it knows  ·  look inside the vectors a language model invented, and find structure nobody put there.


Phase 5. Finding the right thing to say

Five lessons. About four hours. Still nothing installed.

  • 1. It has read it and cannot recall it  ·  ask a model twenty questions it should be able to answer, watch it fail all twenty, and see the shape of the fix.
  • 2. Finding it by the words  ·  build the simplest search that works, compare it against four more sophisticated ones, and find out which actually wins.
  • 3. Finding it by meaning  ·  search by meaning instead of by words, on two sets of questions designed to tell the two approaches apart.
  • 4. Measuring search properly  ·  score the same three searches several ways, and find that which one is best depends on the question you asked.
  • 5. Retrieve, then answer  ·  build the full pipeline, measure it against each half alone, and find that adding the model makes the system strictly worse.


Phase 6. Tools and agents

Five lessons. About four hours. Still nothing installed.

  • 1. What a model cannot do  ·  measure a gap the model cannot close, then close it with three small programs, and notice that two of them are a different species from the third.
  • 2. Choosing the tool  ·  write the rule anyone writes first, measure it, find the rule that fixes it, and then discover that your fix scored 100% because you wrote it with the answers in front of you.
  • 3. When it chooses wrong  ·  give each tool the power to refuse work, measure what that buys, and find that it rescues one failure out of five and cannot touch the other four.
  • 4. A loop that can stop  ·  add ten questions that have no answer, give the system a loop with three ways to stop, find that the retrying is worthless for a reason you could have proved in advance, and find that lesson 3 reached the wrong conclusion from a correct measurement.
  • 5. Measuring an agent  ·  assemble the whole phase into one system, score three versions of it on six numbers instead of one, and work out the exact price at which the ranking flips.


Phase 7. Measuring whether any of it works

Five lessons. About three hours. Still nothing installed.

  • 1. The number has error bars  ·  compute exactly what each score in this course establishes, find that the course's own comparisons rest on nothing, and work out how many examples a claim actually needs.
  • 2. Is the difference real?  ·  compare the two agents question by question instead of total by total, find they disagree about eleven times, and watch the conclusion reverse depending on which outcome you count.
  • 3. The baseline you forgot  ·  build the systems nobody builds, measure them against the agent five phases went into, and find that most of that agent cannot be shown to be doing anything.
  • 4. The test set you keep looking at  ·  repeat that sweep against twenty questions written afterwards, find that the warning does not land the way you expect, and find the thing that actually broke.
  • 5. A test you can actually run  ·  watch a score-based test miss a change that moves a quarter of the questions, then write the test that catches it, and find that the whole thing rests on the system being deterministic.


Phase 8. Shipping: cost, latency, failure, safety

Five lessons. About three hours. Still nothing installed.

  • 1. Counting the work  ·  meter the system in units that do not depend on your laptop, find that its average describes a question that does not exist, and find that refusing to answer costs more than answering.
  • 2. When the corpus grows  ·  build an inverted index, confirm it changes nothing, then take the obvious next step and watch a threshold in another phase silently change meaning.
  • 3. When something fails  ·  take pieces out of the running agent, measure what each costs, and find that the catastrophic failure is the one no ordinary alarm can see.
  • 4. What it must not say  ·  put eight sentences in the corpus that must never come out, measure how many leak, and measure the two defences, one of which is the one everybody builds and does not work.
  • 5. The thing you ship  ·  assemble everything the course built, produce the page you would actually give somebody, and write down what the numbers do not establish.


Phase 9. Attention, and the block

Five lessons. About four hours. Still nothing installed.

  • 1. The thing a bigram cannot see  ·  train a bigram on the whole corpus, score it on the same corpus, and find it capped at exactly half. Not badly trained. Capped.
  • 2. Looking at every word at once  ·  write one attention head and its backward pass, clear the ceiling, and then find that the head is not doing what the story says it does.
  • 3. It works without looking  ·  find the signal by taking things away, and finish with a model that has no query and no key in it and scores within one question of the head.
  • 4. Where a word sits  ·  add a learned vector per position, then discover that the same model on the same number of examples either learns the order of ten things or learns nothing, depending on which examples it was shown.
  • 5. The block  ·  build a transformer block by hand, prove its backward pass against finite differences, count what it costs, and discover that three seeds cannot tell any of these architectures apart.




Fifty lessons. Ten phases. About thirty-seven hours.

Nothing installed, no account, no API key, no GPU, and no network at any point.

Start at phase 0  ·  Read the FAQ  ·  Read the glossary