Machine Learning · L-BiasVar
Pen-and-paper exercises on the bias–variance trade-off. The programming exercises are in the lab
notebook lecture-BiasVar-lab.ipynb. References point back to the lecture; the
estimator exercises are part of the recap Estimators.
Solutions are in skip cells (tag solution), stripped for the handout.
Task 1. State the bias–variance decomposition of $\mathbb{E}_{\mathcal{D}}[E_{out}(h^{\mathcal{D}})]$, and define the average hypothesis $\tilde h$ (lecture).
Task 2. Explain the correspondence with Exercise 3 of the Estimators exercises: what plays the role of $\theta$, of $\hat\theta$, and of the expectation?
Task 3. For each situation, say whether bias or variance dominates:
Task 4. With noisy data a third term appears — the irreducible (Bayes) error.
What is it, and why can no model, however good, reduce it?
Task. Answer briefly (lecture):