Loss Landscape Lab

Training Data

y = 2.5 * x + 1 + noise

30 training examples. Each target receives independent Gaussian noise.

Resampling creates a new training dataset, which changes the actual loss landscape.

Data Noise

noise std 0.45

0 2

std means standard deviation.

Each training target receives random Gaussian noise with this standard deviation.

Change this value and press Resample Data to generate a new dataset.

Model

y_hat = w1 * x + w2

parameters:
w1, w2

Every point on the landscape represents a different pair of model parameters.

Regularization

lambda 0.00

0 10

norm power p 2.00

0.5 6

Regularization term:

abs(w1)^p + abs(w2)^p

Total objective:

MSE + lambda * regularization

Landscape

Click directly on the plot to choose the starting point.

Optimizer

learning rate 0.010

0.001 0.1

Starting Point

Click anywhere on the plot.

The green point is the current optimizer position.

The yellow line shows its path.