CSE

Source: https://cse.iitk.ac.in/pages/CS771.html Parent: https://cse.iitk.ac.in/pages/ResearchAreasNew.html

CS 771: Introduction to Machine Learning

Pre-requisites

Instructor's consent (no course prerequisites).

Desirable

MSO201A/equivalent, ESO207A, familiarity with programming in MATLAB/Octave, Python, or R, or instructor’s consent.

About the course

Machine Learning is the discipline of designing algorithms that allow machines (e.g., a computer) to learn patterns and concepts from data without being explicitly programmed.  This course will be an introduction to the design (and some analysis) of machine learning algorithms, with a modern outlook focusing on recent advances, and examples of real-world applications of machine learning algorithms.

List of Topics

1. Preliminaries
2. Multivariate calculus:  gradient, Hessian, Jacobian, chain rule
3. Linear algebra:  determinants, eigenvalues/vectors, SVD
4. Probability theory:  conditional probability, marginal probability, Bayes rule
5. Supervised Learning
6. Local/proximity-based methods:  nearest-neighbors, decision trees
7. Learning by function approximation
   1. Linear models:  (multiclass) support vector machines, ridge regression
   2. Non-linear models:  kernel methods, neural networks (feedforward)
8. Learning by probabilistic modeling
   1. Discriminative methods:  (multiclass) logistic regression, generalized linear models
   2. Generative methods:  naive Bayes
9. Unsupervised Learning
10. Discriminative Models:k-means (clustering), PCA (dimensionality reduction)
11. Generative Models
    1. Latent variable models:  expectation-maximization for learning latent variable models
    2. Applications:  Gaussian mixture models, probabilistic PCA
12. Practical Aspects
13. Concepts of over-fitting and generalization, bias-variance tradeoffs
14. Model and feature selection using the above concepts
15. Optimization for machine learning:  (stochastic/mini-batch) gradient descent
16. Additional Topics (a subset to be covered depending on interest)
17. Deep learning:  CNN, RNN, LSTM, autoencoders
18. Structured output prediction:  multi-label classification, sequence tagging, ranking
19. Ensemble methods:  boosting, bagging, random forests
20. Recommendation systems:  ranking methods, collaborative filtering via matrix completion
21. Reinforcement learning and applications
22. Kernel extensions for PCA, clustering, spectral clustering, manifold learning
23. Probability density estimation and anomaly detection
24. Time-series analysis and modeling sequence data
25. Sparse modeling and estimation
26. Online learning algorithms:  perceptron, Widrow-Hoff, explore-exploit
27. Statistical learning theory:  PAC learning, VC dimension, generalization bounds
28. A selection from some other advanced topics such as semi-supervised learning, active learning, inference in graphical models, Bayesian learning and inference

Reference

There will not be any dedicated textbook for this course. In lieu of that, we will have lecture slides/notes and monographs, tutorials, and papers for the topics that will be covered in this course. Some recommended (although not required) books are:

1. Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2007
2. Hal Daume III, A Course in Machine Learning, 2015 (freely available online)
3. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, Springer, 2009
4. John Hopcroft, Ravindran Kannan, Foundations of Data Science, 2014 (freely available online)
5. Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning, The MIT Press, 2012
6. Kevin Murphy, Machine Learning: A Probabilistic Perspective, The MIT Press, 2012