AI & ML Engineering · Unit 8 · Machine Learning

Model Evaluation

Measuring whether a model is good, and whether it will stay good on new data.

  1. 8.1

    Train, Validation and Test Sets

    Splitting data correctly and avoiding data leakage.

    Video
  2. 8.2

    Cross-Validation

    K-fold validation and getting a reliable estimate of performance.

    Video
  3. 8.3

    Classification Metrics

    Accuracy, precision, recall, F1, ROC and the confusion matrix.

    Video
  4. 8.4

    Regression Metrics

    MAE, RMSE and R-squared, and when each one misleads.

    Video
  5. 8.5

    Bias, Variance and Overfitting

    Diagnosing underfitting and overfitting with learning curves.

    Video
  6. 8.6

    Class Imbalance

    Resampling, class weights and choosing the right metric.

    Video
  7. 8.7

    Hyperparameter Tuning

    Grid search, random search and Bayesian optimisation.

    Video

Unit project

An evaluation report that compares three models on the same problem and recommends one, with the trade-offs stated.

Paid learners get the full brief, submit it from their dashboard and have it reviewed. It goes in your portfolio.

Open every unit

Enrol once to open every unit, its videos and a reviewed project for your portfolio.