Learn machine learning by writing it.

Follow the roadmap from a matrix product to multi-head attention, first in numpy and then in real PyTorch. You write each algorithm yourself, and hidden tests check your code.

Start with Matrix Multiplication See the roadmap

Easy0 / 59
Medium0 / 96
Hard0 / 33

No account needed to start. numpy runs in your browser.

21 topics

Roadmap

Each topic builds on the topics above it. Select a topic to see its problems.

0 / 188 solved

IIIIIIIVV

Courses

Articles and videos for each subject, split into the ideas the problems need.

All courses

The PyTorch track

Real PyTorch, graded like everything else

The roadmap does not stop at numpy. Once you have built the pieces by hand, you write them again in the framework people actually train with: autograd, a training loop, then a full classifier. Hidden tests check the result, the same way they check everything else. Every other topic has PyTorch problems too, 45 in all.

8 sections / 21 resources / 13 videos

Evaluation MetricsScore a model honestly, especially when the classes are imbalanced.7 sections / 17 resources / 3 videos

Unsupervised LearningFind structure without labels: clustering, projection, and scoring both.6 sections / 14 resources / 4 videos

Deep LearningActivations, backprop, optimisers, normalisation, recurrence, attention.8 sections / 20 resources / 5 videos

Natural Language ProcessingTokenisation, term weighting, position, and translation quality.7 sections / 15 resources / 2 videos

Computer VisionConvolution, pooling, box overlap and duplicate suppression.7 sections / 15 resources / 1 videos

PyTorchAutograd, optimisers and training loops in a real framework.6 sections / 13 resources

Information TheoryEntropy at work: conditional entropy, mutual information, divergences, codes.6 sections / 15 resources / 1 videos

OptimisationMomentum, adaptive steps, schedules and line search: how training moves downhill.6 sections / 15 resources

Ensembles and BoostingCombine weak models: voting, bagging, AdaBoost and gradient boosting.6 sections / 14 resources / 5 videos

Probabilistic ModelsMarkov chains, hidden Markov models, mixtures and Bayesian regression.6 sections / 13 resources / 4 videos

Reinforcement LearningReturns, Bellman updates, Q-learning, advantages and bandits.7 sections / 17 resources / 1 videos

Recommender SystemsCollaborative filtering, matrix factorisation and ranking metrics.6 sections / 13 resources

Time SeriesSmoothing, autocorrelation, autoregression and sequence alignment.6 sections / 17 resources / 2 videos

MLOps and Data ProcessingClean, split and encode data, watch it drift, and ship models small.7 sections / 16 resources

Generative ModelsVAEs, GANs, normalising flows and diffusion, one equation at a time.6 sections / 15 resources / 2 videos

Graph Neural NetworksMessage passing on graphs: GCN, GraphSAGE, graph attention and PageRank.7 sections / 16 resources / 1 videos

Transformers and LLMsThe parts of a modern language model: masks, RoPE, RMSNorm, caches, sampling.7 sections / 15 resources / 2 videos

Hard

04Mini-batch Training with DataLoaderMedium

05Gradient AccumulationMedium

Start the PyTorch track

rms_norm.pyPyTorchMedium

import torch
from torch import nn

class RMSNorm(nn.Module):
    def __init__(self, dim, eps=1e-6):
        super().__init__()
        self.eps = eps
        self.weight = nn.Parameter(torch.ones(dim))

    def forward(self, x):
        ms = x.pow(2).mean(-1, keepdim=True)
        return x * torch.rsqrt(ms + self.eps) * self.weight

Accepted5/5 cases · graded in Colab

Where your code runs

numpy problems run in your browser. PyTorch has no build for the browser, so PyTorch problems run in Google Colab or on your own machine.

In your browser
numpy
numpy problems run in a Python built for the browser. Nothing to install.
Google Colab
PyTorch + GPU
PyTorch and a free GPU with nothing to install. Take a code, open the notebook, and the result comes back here.
Your machine
PyTorch
One command installs a small local runtime. After that, this site finds it on its own.