lecture + exercises + lab
Machine learning from first principles: every model is derived on paper and implemented from
scratch in numpy before a framework takes over — from linear regression via the bias–variance
trade-off and model evaluation to neural networks, language models, attention and Transformers.
Automatic differentiation (PyTorch) arrives right before the neural networks.
Prerequisites: Mathematics (calculus, linear algebra, probability) and programming in Python.
Status: Alpha - Work in progress
| Lecture | Topic | Slides | Exercises | Lab | |
|---|---|---|---|---|---|
| 1 | L-LinReg |
Univariate Linear Regression and Gradient Descent | |||
| 2 | L-MultiReg |
Multivariate Linear Regression and Feature Scaling | |||
| 3 | L-LogReg |
Logistic Regression and the Cross-Entropy |
| Lecture | Topic | Slides | Exercises | Lab | |
|---|---|---|---|---|---|
| 4 | L-BiasVar |
The Bias–Variance Trade-off | lecture | exercises | |
| 5 | L-Eval |
Regularisation, Validation and Evaluation | |||
| 6 | L-Fair |
Fairness Metrics and the Impossibility Theorem |
| Lecture | Topic | Slides | Exercises | Lab | |
|---|---|---|---|---|---|
| 7 | L-Autodiff |
Automatic Differentiation | |||
| 8 | L-FFNN |
Feed-Forward Neural Networks | |||
| 9 | L-Train |
Training Deep Networks: Adam, Normalisation, Residuals |
| Lecture | Topic | Slides | Exercises | Lab | |
|---|---|---|---|---|---|
| 10 | L-LM |
Language Modelling and Information Theory | |||
| 11 | L-Embed |
Embeddings and the Neural Language Model | |||
| 12 | L-Att |
Attention | |||
| 13 | L-Transformer |
Transformers |
| Lecture | Topic | Slides | Exercises | Lab | |
|---|---|---|---|---|---|
| 14 | L-Classical |
Classical ML: Entropy, Decision Trees, Ensembles |
A fundamentals notebook recaps the mathematics the lectures need, with its own pen-and-paper exercises and lab.
| Topic | Content | Used in | Notes | Exercises | Lab |
|---|---|---|---|---|---|
| Probability | sample space and axioms, random variables, joint, marginal and conditional distributions, Bayes' rule, conditional independence, expectation, variance and covariance, the binomial and the Gaussian | L-BiasVar, L-Eval, L-Fair, L-LM |
notes | exercises | lab |
| Estimators | statistic and estimator, bias, variance and mean squared error, sufficient statistics | L-BiasVar |
notes | exercises | lab |
| Uncertainty | standard error, 95 % confidence interval, maximum likelihood, error bars for proportions, the bootstrap, comparing two estimates | L-Eval |
notes | exercises | lab |
| Linear Algebra | vectors and matrices, dot and matrix product, the shapes a network layer needs | L-MultiReg, L-FFNN |
Machine learning
L-BiasVar, L-Eval).L-LinReg, L-MultiReg, L-LogReg).L-BiasVar, L-Eval, L-Classical).L-Autodiff, L-FFNN, L-Train).Probability & statistics
L-LM).Language models
L-LM, L-Embed).L-Att, L-Transformer).Fairness
L-Fair).