How does batch size affect gradient descent?

Fit y_pred = theta0 + theta1 * x to 40 fixed training points. Click the contour to select a starting point.

Updates: 0
Completed epochs: 0
Samples processed: 0
Full-data MSE: -
theta0: -
theta1: -

Full-data loss contour

Blue lines: equal full-data MSE | Orange path: parameter updates
Square: starting point | Dot: current parameters | Plus: least-squares minimum

Training data and current fitted line

Orange circles: last batch used | Solid line: current fit | Dashed line: least-squares fit

Full-data loss after each update