🌀 Overfitting Trap – Will Your Model Survive New Data?

The Challenge

You will see two models trained on the same 8 students:

Simple model – a straight line (may leave some error)
Complex model – a very flexible curve that almost perfectly hits every training point

Then we reveal 4 new students the models have never seen. Your job: decide which model will predict the new students better. Think about generalisation, not just training performance.

PHASE 1 – Training data only

Look at both models on the data they were trained on

Simple (straight line)

Training MSE: –

Complex (wiggly)

Training MSE: –

Which model do you trust more for new students?

I trust the Simple model

I trust the Complex model

PHASE 2 – New unseen students

Here are 4 new students. Watch what happens.

Simple model predictions

Test MSE: –

Complex model predictions

Test MSE: –