Shalabh
shalab@iitk.ac.in
shalabh1@yahoo.com
Department of Mathematics & Statistics
Indian
Multivariate Data Mining - Methods and Applications
by
Allahabad University
Swayam Prabha Course
The forty hours course is for the students in Bachelor's and Master's programmes and covers the topics of data mining
Suggested books:
(i) Izenman, A.J., (2008), Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning, Springer.
(ii) James, G., Witten D., Hastie T., Tibshirani R., (2013), An Introduction to Statistical Learning with applications to R, Springer.
(iii) Everitt B.S., Landau S., Leese M., Stahl D. (2011), Cluster Analysis, 5th Edition, Wiley.
(iv) Han, J. and Kamber, M (2006). Data Mining: Concepts and Techniques, 2nd edition, Morgan Kaufmann.
(v) Dunham, M. H. (2003). Data Mining: Introductory and Advanced Topics, Pearson Education.
Language of the course: English
Duration of the course: 40 Hours
Swayam Prabha DTH Channel 16 Youtube link: The telecasted lectures are available at YouTube (Click here).
Slides and Videos used in the lectures:
|
Lecture No. |
Lecture videos download links |
Lecture slides download links |
Brief Description |
Lecture Title |
|
1 |
Introduction |
Introduction to data mining, its applications in various fields, Outline of the course |
||
|
2 |
Data Mining, Machine Learning and Artificial Intelligence |
Basics of Data mining, Data mining and knowledge discovery, Artificial Intelligence, |
||
|
3 |
Machine Learning Rules |
Machine Learning Rules, Supervised, unsupervised learning, Batch Learning and Online learning, Reinforcement learning, resubstitution Estimate, Generalizations for improving resubstitution estimates, Training, learning and test sets, Bootstrap, Ockham’s (or Occam’s) razor principle, methods for reducing the effects of overfitting, Sampling Design for obtaining data, |
||
|
4 |
Matrix Algebra |
Introduction to vectors, Operations of vectors, Different types of vectors, different types of matrices, matrix operations, Eigen values and eigen vectors, different results related to orthogonal matrices, idempotent matrices, quadratic forms, Matric norms |
||
|
5 |
Multivariate Analysis |
Multivariate probability distributions, Multivariate normal distribution, marginal and conditional distributions, Expectation of some quadratic forms |
||
|
6 |
Multiple Regression Model: Introduction |
General structure of regression problem, Multiple linear models, Estimation of parameters, model in deviation form |
||
|
7 |
Properties of Estimators and Model Selection Criterion |
Properties of estimators and model selection criterion, R square, adjusted R square, AIC, BIC |
||
|
8 |
Model Assessment for Multiple Regression |
Model Assessment for random and fixed X, Prediction error, apparent error rate or resubstitution error rate, resampling methods, V-fold cross validation, Optimism corrected bootstrap estimate of PE |
||
|
9 |
Multicollinearity and Variables Selection |
Multicollinearity problem and its implications and measures, stepwise variable selection regression, backward, and forward methods, hybris stepwise method |
||
|
10 |
Shrinkage Estimation |
Shrinkage estimation, penalized regression estimators, LASSO and Ridge regression |
||
|
11 |
Principal Component and Least Angle Regression |
Principal Component regression and Least Angle Regression methods |
||
|
12 |
Regression Methods for Classification |
Formulation of probability models, LOGIT and PROBIT Models for classification |
||
|
13 |
Data Mining Methods for High Dimensional data: Principal Component Analysis |
The Curse of Dimensionality, Basics and objectives of Principal Component Analysis for linear feature space, Advantages and Disadvantages |
||
|
14 |
Statistical Analysis of PCA |
Population PCA, Least-Squares Optimality of PCA, Eckart-Young Theorem, Courant–Fischer Min-Max theorem, PCA as a Variance-Maximization Technique |
||
|
15 |
Sample PCA and Ap16plications |
Sample PCA, Tools for selecting the number of principal components, and Real data applications of PCA. Principal Component Analysis for Data Visualization |
||
|
16 |
Sparse PCA and Nonlinear Dimensionality Reduction |
Sparse and robust methods for PCA, PCA for outlier detection, nonlinear dimensionality reduction, polynomial PCA, Basic elements of Nonparametric Density Estimation |
||
|
17 |
Kernel Principal Component Analysis |
PCA for non-linear feature space, Kernel PCA, |
||
|
18 |
Latent Variable Model for Blind Source Separation |
Latent variable models for blind source separation: cocktail party problem, independent component analysis (ICA) and its applications, linear mixing, and noiseless ICA |
||
|
19 |
ICA Algorithms and Exploratory Factory Analysis |
FastICA algorithm for determining single source component, deflation, and parallel FastICA algorithm for extracting multiple independent source components, Applications to the real dataset, Exploratory factor analysis model |
||
|
20 |
Introduction to Artificial Neural Network |
Basics and Structure of ANN, its various applications, ANN design and brain activity |
||
|
21 |
McCulloch- Pitts Neuron and Single-Layer Perceptron |
Threshold logic unit, McCulloch-Pitts Neuron and its limitations, Hebb learning rule, Different types of neural networks |
||
|
22 |
Rosenblatt’s single-layer perceptron |
Feedforward single layer network, Rosenblatt’s Single layer perceptron, single unit perceptron, Algorithm for implementing Rosenblatt’s single layer perceptron, perceptron convergence theorem |
||
|
23 |
Multi-layer perceptron |
Multilayer perceptron, Learning networks, Multiclass classification rule |
||
|
24 |
Backpropagation of Errors Algorithm |
Backpropagation of Errors Algorithm-Single hidden layer, Online learning mode, Stochastic learning mode, Batch learning mode |
||
|
25 |
Convolutional Neural Networks |
Convolution Neural Network (CNN), its architecture, and applications |
||
|
26 |
Recurrent neural network and Projection Pursuit |
Recurrent neural network (RNN) and CNN, Basics of RNN, Elman and Jordan networks, Projection pursuit regression, generalized additive model |
||
|
27 |
Cluster Analysis: An Introduction |
Basic elements and objectives of cluster analysis, various similarity and distance measures, |
||
|
28 |
Hierarchical Clustering Techniques |
Distance measures for quantitative variables, Hierarchical clustering method, Agglomerative Hierarchical Clustering, Single linkage Clustering, Complete Linkage Clustering, Average Linkage Clustering, |
||
|
29 |
Centroid and Non-hierarchical Clustering Methods |
Centroid Linkage Clustering, Steps for implementation of Agglomerative Hierarchical Clustering, Ward’s Hierarchical Clustering method, Partitioning Clustering: K-means clustering |
||
|
30 |
Partition around medoids (PAM) Clustering Algorithm |
PAM Clustering Algorithm, K-medoid and PAM clustering algorithm, Fuzzy analysis, Selecting number of clusters, Silhouette and average Silhouette Method |
||
|
31 |
Self-Organizing Map |
Self-organizing maps (SOM) or Kohonen neural network, on-line and batch versions of SOM algorithm, distance weight version, U-matrix, Hierarchical SOM, Quality measures |
||
|
32 |
Clustering based upon Mixture Models |
Density-Based Clustering Methods, Clustering based on Gaussian Mixture Models, Expectation-Maximization Clustering algorithm |
||
|
33 |
Recursive Partitioning: Decision Trees |
Components of decision tree classification and basic terminology, Attribute Selection Measures: information gain and entropy, Gini index and node impurity function, choosing the best split, pruning algorithm for classification trees., Recursive partitioning to grow a tree, mating |
||
|
34 |
Training and Pruning Decision Trees |
Overfitting and pruning the tree, cost complexity pruning measure, Choosing the best pruned tree, Cross validation for selecting best subtree |
||
|
35 |
Regression Trees |
Regression trees: Background, Basic terminology, Recursive partitioning for regression data, terminal node value and splitting strategy, pruning the tree and best pruned subtree |
||
|
36 |
Committee Machine and Random Forests |
Committee Machine: Bagging tree-based classifiers and regression tree predictors, Boosting, ADABOOST algorithm for binary classification. Random Forests algorithm for regression or classification |
||
|
37 |
Support Vector Machine for Linear Separable Cases |
Support vector machine (SVM) with linear separable case, obtaining optimal separating hyperplane for linear separable case, Karuh, Kuhn, Tucker conditions, Multiclass SVM as a series of binary problems |
||
|
38 |
Support Vector Machine for Linearly Non-Separable Cases |
SVM for nonlinearly separable datasets, nonlinear SVM, kernel trick for nonlinearly separable datasets, SVM for regression, e-insensitive loss function and its optimization |
||
|
39 |
Block Clustering |
Basics of Block clustering, Hartigan’s block-clustering algorithm, Bi-clustering, two-way ANOVA model for bi-clustering |
||
|
40 |
Plaid Models for Block Clustering |
Plaid models for bi-clustering with examples |