- ES Español (Latinoamérica)

- EN English

7.61. Statistical Learning Theory (Elective)
- Semester: 9th Sem. Credits: 4
- Hour of this course: Theory: 2 hours; Practice: 4 hours;
- Syllabus:
- htmlonly

Español (Latinoamérica)

English - Prerrequisites:
- MA352 Stochastic processes (8th Sem)
- MA372 Functional analysis II (8th Sem)
7.61.1. Justification ↑ Back to top
This course provides the theoretical foundations of statistical learning, covering the mathematical principles that support machine learning algorithms. It focuses on the theoretical analysis of generalization capability, model complexity, and performance guarantees for various learning methods. The course connects theory with practice through the study of generalization bounds, bias-variance tradeoff, and foundations of computational learning theory.
7.61.2. Generales Goals ↑ Back to top
- Understand the fundamentals of statistical learning theory and its probabilistic foundations.
- Analyze the bias-variance tradeoff in the selection and evaluation of machine learning models.
- Apply Vapnik-Chervonenkis (VC) theory to analyze model complexity and generalization capability.
- Develop and demonstrate generalization bounds for different learning algorithms.
- Connect theoretical principles with the practical implementation of statistical learning algorithms.
7.61.3. Contribution to Outcomes ↑ Back to top
- ABET-1) An ability to identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline. (Usage)
- ABET-3) An ability to develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions. (Usage)
7.61.4. Content ↑ Back to top
7.61.4.1. Foundations of Statistical Learning (20 hours) [Skills ABET-1,ABET-3] ↑ Back to top
Bibliography: (Vapnik, 1998; Hastie et al., 2009)
Topics
- Bias-variance tradeoff
- Vapnik-Chervonenkis (VC) theory
- Empirical risk minimization
- Generalization bounds
- Learning theory fundamentals
Learning Outcomes
- Analyze the bias-variance tradeoff in model selection [Assessment]
- Apply VC theory to analyze model complexity [Usage]
- Implement empirical risk minimization [Usage]
- Interpret generalization bounds [Familiarity]
7.61.4.2. Supervised Learning Methods (20 hours) [Skills ABET-1,ABET-3] ↑ Back to top
Bibliography: (Hastie et al., 2009; James et al., 2013)
Topics
- Linear models and extensions
- Support Vector Machines (SVM)
- Decision trees and ensemble methods
- Kernel methods
- Probabilistic supervised learning
Learning Outcomes
- Implement and analyze linear models [Usage]
- Apply SVM for classification and regression [Usage]
- Construct ensemble methods (bagging, boosting) [Usage]
- Use kernel methods for nonlinear problems [Usage]
7.61.4.3. Unsupervised Learning Methods (20 hours) [Skills ABET-1,ABET-3] ↑ Back to top
Bibliography: (Murphy, 2012; Bishop, 2006)
Topics
- Clustering algorithms (k-means, hierarchical)
- Dimensionality reduction (PCA, t-SNE)
- Density estimation
- Manifold learning
- Autoencoders and deep unsupervised learning
Learning Outcomes
- Apply clustering algorithms to data exploration [Usage]
- Implement PCA and other dimensionality reduction methods [Usage]
- Use density estimation for unsupervised learning [Usage]
- Analyze manifold learning techniques [Assessment]
7.61.4.4. Model Evaluation and Selection (20 hours) [Skills ABET-1,ABET-3] ↑ Back to top
Bibliography: (Hastie et al., 2009; James et al., 2013)
Topics
- Cross-validation strategies
- Performance metrics for classification and regression
- Model comparison and statistical tests
- Hyperparameter optimization
- Learning curves and diagnostic tools
Learning Outcomes
- Implement cross-validation for model assessment [Usage]
- Select appropriate performance metrics [Assessment]
- Compare models using statistical tests [Usage]
- Optimize hyperparameters using systematic methods [Usage]
7.61.5. Bibliography ↑ Back to top
Vapnik, V. N. (1998). Statistical Learning Theory. Wiley, 1st edition.
Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2nd edition.
James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An Introduction to Statistical Learning: with Applications in R. Springer, 1st edition.
Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press, 1st edition.
Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer, 1st edition.