CS365A

HomeWork1

Overfitting and Complex Models

Coefficient Values:

Table of Coefficients for training on small dataset

M=0

M=1

M=2

M=3

M=4

M=5

M=6

M=7

M = 8

M = 9

w0

156.45

50.37

5.40

1.26

1.29

3.88

-0.75

-0.18

1.03

-2.53

w1

-82.82

-0.93

-12.54

-23.22

-90.91

25.18

5.32

-39.19

109.27

w2

27.89

79.68

155.25

827.59

-338.12

-59.71

644.12

-2070.44

w3

-85.14

-409.84

-3647.90

2320.93

308.94

-5943.95

22595.02

w4

425.57

7804.80

-8591.79

-578.30

33632.40

-156416.60

w5

-6420.10

16298.20

-963.06

-119305.40

711845.77

w6

-12329.26

5392.12

258762.97

-2128283.18

w7

-5798.99

-313378.84

4028322.86

w8

162172.84

-4377092.04

w9

2079030.47

Table of Coefficients for training on large dataset

M = 0

M = 1

M = 2

M = 3

M = 4

M = 5

M = 6

M = 7

M = 8

M = 9

w0

256.38

67.63

4.29

-0.25

0.04

-0.05

0.01

0.01

0.00

0.00

w1

-166.28

14.06

8.93

-1.37

1.51

-0.39

-0.28

-0.07

0.01

w2

-19.50

-12.82

18.89

-18.91

6.65

4.92

1.25

-0.36

w3

27.56

-49.72

119.17

-58.82

-45.41

-13.25

5.36

w4

75.17

-321.41

286.13

237.34

85.55

-49.14

w5

353.08

-677.64

-688.34

-348.59

289.25

w6

655.07

1036.04

897.80

-1098.09

w7

-585.58

-1335.85

2619.87

w8

913.09

-3575.61

w9

2168.68

Part A

Plots for different polynomial degree which is fitted to the small training data:

Degree = 0

Degree = 1

Degree = 2

Degree = 3

Degree = 4

Degree = 5

Degree = 6

Degree =7

Degree = 8

Degree = 9

Part B

Plots for different polynomial degree which is fitted to the large training data:

Degree = 0

Degree = 1

Degree = 2

Degree = 3

Degree = 4

Degree = 5

Degree = 6

Degree = 7

Degree = 8

Degree = 9

Part C

Various Error plots:

The Erms value is smallest for polynomial of degree n = 2 for the small training set. This corresponds to the polynomial:

The overfitted curve attains an Erms value of zero for n = 9 but has very large Erms for test data. This is primarily due to very large coefficients of the the 9th order polynomial and alternating signs between coefficients which makes it oscillate wildly between training values.

Noticeable is the fact that for the large dataset the 9th order polynomial performs well as expected.

The code is attached here.