Shalabh
 shalab@iitk.ac.in
shalabh1@yahoo.com
Department of Mathematics & Statistics
Indian Institute of Technology Kanpur, Kanpur - 208016 (India)

HOME PAGE

MTH 441 : Linear Regression and ANOVA

Credits 3-0-1-0 (10 credits)

Prerequisites MTH207A or MSO201A

Objectives This is a fundamental course on statistical modelling - principles and methodologies of which can be extensively applied to social sciences, basic sciences, engineering sciences, medical sciences, etc., for the purpose of model building. The course not only provides a theoretical foundation to the vastly popular statistical tools used in linear regression analysis and Analysis of Variance (ANOVA), but also introduces various concepts and modelling techniques to effectively analyse and model real datasets that do not conform to a set of basic underlying assumptions.

Course Contents

  1. Introduction to Simple and Multiple Linear Regression Models [3 Lectures]

    1. Estimation of the parameters using least squares and maximum likelihood estimation methods and their properties.

  2. Topics on Multivariate Normal Distribution [2 lectures]

    1. Introduction to Multivariate Normal distribution, and basic properties,

    2. Distribution and independence of quadratic forms,

    3. Cochran's theorem.

  3. Testing of Hypotheses and Confidence Intervals [6 lectures]

    1. General testing of H : Rβ = r all possible special cases.

    2. Goodness of fit test with introduction to R-square, etc.

    3. Interval estimation: Confidence band and prediction intervals, simultaneous confidence intervals/ellipsoid, Bonferroni's correction.

  4. Residual Analysis and Regression Diagnostics [6 lectures]

    1. Detecting outliers: Different types of residuals, Hat matrix diagonals (in connection with leverage points), Detecting and dealing with outliers.

    2. Departures from underlying assumptions: diagnosis and remedies. (i) Dealing with curvatures, (ii) non-constant variance and serial correlation,

    3. Departures from normality.

  5. Multicollinearity [4 lectures]

    1. Implication of multicollinearity.

    2. Diagnostics: VIF and Variance Decomposition Methods.

    3. Remedial measures: Canonical regression and principal component regression, Ridge Regression.

  6. Variable Selection [7 lectures]

    1. Introduction to variable selection, under and over-fitting problems.

    2. Model selection: Adjusted R-square, Mallows Cp,  Cross-validation.

    3. Variable selection: Forward selection, Backward elimination, Stepwise variable selection.

    4. Penalized regression: AIC, BIC criteria.

  7. Use of Categorical Explanatory or Dummy Variables [4 lectures]

    1. Indicator variables.

    2. Introduction to ANOVA and ANCOVA models.

    3. ANOVA table, splitting of sum of squares.

    4. Estimation and testing in the above setups.

  8. Generalized Linear Models [4 Lectures]

    1. Introduction to GLM: systematic and random components, link functions,

    2. Maximum Likelihood Estimation: iteratively re-weighted least squares,

    3. Applications: Logistic regression for binary data, Poisson regression for count data.

  9. Design preliminaries, CRD, RBD and LSD [6 Lectures]

    1. Introduction and Principles of designs.

    2. CRD- Methodology and development of analysis of variance.

    3. RBD- Methodology and development of analysis of variance.

    4. LSD- Methodology and development of analysis of variance.

References

Books to be followed:

Montgomery, D. C., Peck, E. A. and Vining, G. G. (2012): Introductionto Linear Regression Analysis, Wiley Series in Probability and Statistics, Wiley.

Draper, N. R. and Smith, H. (1998):  Applied Regression Analysis, Wiley.

Rao, C.R., Toutenburg H., Shalabh, and Heumann C. (2008) : Linear Models and Generalizations - Least Squares and Alternatives, Springer.

 

Other References:

Monahan, John F.  (2008):   A Primer on Linear Models, CRC Press,

Khuri, Andre I. (2010) : Linear Model Methodology, CRC Press.

Seber G. A. F. and Lee, A. J. (2003) Linear Regression Analysis, Wiley

Sengupta, D. and Jammalamadaka, S. R. (2003) Linear Models: An Integrated Approach, World Scientific, Singapore.

Vinod, H. D. and Ullah, A. (1981) Recent Advances in Regression Methods, M. Dekker.

 

Course Policy: Earn your marks and grades. I will be the happiest instructor to award the best grades to all the students.

Class schedule: Mon, Tue, Thu, Fri 11-11:50 AM in L8. Any deviation from the schedule will be announced in the class.

Grading Scheme: Quiz- 30 %,   Mid Sem.- 30%   End Sem.- 40 %

TA:  Miss Arham Kaiyoom (Email: arhamk24@iitk.ac.in), Mr. Sayan Bhowmik (Email: sayanb22@iitk.ac.in)

Contact hours: 24 X 7, by email, phone, what's app. (If possible and not so urgent, avoid calling between 12-7 AM.)

Announcements:

Assignments:

Assignment 1

Assignment 2

Assignment 3

Assignment 4

Assignment 5

Assignment 6

Assignment 7

Assignment 8

 

Lecture notes for your help (If you find any typo, please let me know)

Lecture Notes 1 : Useful Results on Linear Algebra, Distributions and Introduction to Regression Analysis

Lecture Notes 2 : Simple Linear Regression Analysis

Lecture Notes 3 : Multiple Linear Regression Model

Lecture Notes 4 : Model Adequacy Checking

Lecture Notes 5 : Diagnostic for Leverage and Influence

Lecture Notes 6 : Multicollinearity

Lecture Notes 7 : Variable Selection and Model Building

Lecture Notes 8 :  Logistic and Poisson Regression Models

Lecture Notes 9 :  Generalized Linear Models

Lecture Notes 10 : Experimental Design Models

Lecture Notes 11 : Experimental Designs and Their Analysis