- ES Español (Latinoamérica)

- EN English

4.8. Mathematical Foundations of Data Science (FDS)
This area covers the mathematical theory underlying modern data science and machine learning, including statistical learning, optimization, kernel methods, dimensionality reduction, topological data analysis, information theory, and the mathematics of deep learning.
| Knowledge Area (KA) | Core Tier1 | Core Tier2 |
4.8.1 Statistical Learning Theory | 1 | 1 |
4.8.2 Optimization for Machine Learning | 1 | 1 |
4.8.3 Topological Data Analysis | 1 | 1 |
4.8.4 Dimensionality Reduction | 1 | 1 |
4.8.5 Kernel Methods and Reproducing Kernel Hilbert Spaces | 1 | 1 |
4.8.6 Information Theory | 1 | 1 |
4.8.7 Mathematical Foundations of Deep Learning | 1 | 1 |
4.8.1. FDS/Statistical Learning Theory (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
PAC learning framework, VC dimension, Rademacher complexity, generalization bounds, and the bias-variance trade-off.
Topics:
Core
- PAC learning framework: sample complexity, hypothesis classes, and learnability
- VC dimension: definition, examples, and the fundamental theorem of learning
- Rademacher complexity and uniform convergence bounds
- Bias-variance decomposition and the double descent phenomenon
- Regularization: Tikhonov regularization and structural risk minimization
Learning Outcomes:
Core:
- State the PAC learning definition and determine whether a hypothesis class is PAC-learnable [Familiarity]
- Compute the VC dimension of standard hypothesis classes and derive sample complexity bounds [Usage]
- Analyze generalization error using Rademacher complexity and explain the double descent phenomenon [Assessment]
4.8.2. FDS/Optimization for Machine Learning (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Convex optimization theory, SGD and its variants, adaptive methods, and non-convex optimization landscapes.
Topics:
Core
- Convexity, strong convexity, and Fenchel duality in optimization
- Stochastic gradient descent: convergence rates for convex and non-convex objectives
- Adaptive gradient methods: AdaGrad, RMSProp, and Adam
- Non-convex optimization: saddle points, local minima, and loss landscape geometry of neural networks
- Proximal gradient methods and coordinate descent for structured problems
Learning Outcomes:
Core:
- Explain the convergence guarantees of SGD for smooth convex functions and identify the role of the learning rate [Familiarity]
- Apply Adam and proximal gradient methods to train regularized machine learning models [Usage]
- Analyze the loss landscape of an overparameterized network and explain why SGD finds good minima [Assessment]
4.8.3. FDS/Topological Data Analysis (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Persistent homology, Vietoris-Rips complexes, the mapper algorithm, and applications to data shape analysis.
Topics:
Core
- Simplicial complexes: Čech, Vietoris-Rips, and witness complexes
- Persistent homology: filtrations, persistence diagrams, and barcodes
- Stability theorem for persistence diagrams and the bottleneck distance
- Mapper algorithm for topological summarization of high-dimensional data
Non Core
- Applications of TDA to materials science, genomics, and neural network analysis (2020-2025)
Learning Outcomes:
Core:
- Construct a Vietoris-Rips filtration from a point cloud and compute its persistence diagram [Familiarity]
- Apply the stability theorem to bound the distance between persistence diagrams of similar data sets [Usage]
NonCore:
- Interpret topological features of a data set using the mapper algorithm and explain their scientific significance [Assessment]
4.8.4. FDS/Dimensionality Reduction (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Linear and nonlinear dimensionality reduction: PCA, random projections, manifold learning, and modern methods (t-SNE, UMAP).
Topics:
Core
- PCA: singular value decomposition, explained variance, and optimal low-rank approximation
- Johnson-Lindenstrauss lemma and random projections for dimensionality reduction
- Manifold learning: Isomap, locally linear embedding (LLE), and spectral embedding
- Modern nonlinear methods: t-SNE and UMAP; theory, parameters, and limitations
- Autoencoders and variational autoencoders (VAEs) as nonlinear dimensionality reduction
Learning Outcomes:
Core:
- Explain the Johnson-Lindenstrauss lemma and justify why random projections preserve distances [Familiarity]
- Apply PCA and t-SNE to high-dimensional data and interpret the resulting low-dimensional embeddings [Usage]
- Compare linear and nonlinear dimensionality reduction methods with respect to geometry preservation and computational cost [Assessment]
4.8.5. FDS/Kernel Methods and Reproducing Kernel Hilbert Spaces (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Positive definite kernels, RKHS theory, support vector machines, kernel PCA, and the kernel trick.
Topics:
Core
- Positive definite kernels: Mercer's theorem and common kernel functions (RBF, polynomial, Matérn)
- Reproducing kernel Hilbert spaces: the representer theorem and regularization
- Support vector machines: large-margin classification and the dual formulation
- Kernel PCA and kernel regression via the kernel trick
- Gaussian processes as distributions over functions: posterior inference and hyperparameter learning
Learning Outcomes:
Core:
- State Mercer's theorem and verify that a given function is a positive definite kernel [Familiarity]
- Derive the dual SVM formulation and apply the kernel trick to kernelize a linear algorithm [Usage]
- Apply Gaussian process regression to a prediction problem and calibrate uncertainty estimates [Assessment]
4.8.6. FDS/Information Theory (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Shannon entropy, mutual information, channel capacity, rate-distortion theory, and applications to machine learning.
Topics:
Core
- Shannon entropy, joint and conditional entropy, and the chain rule
- Mutual information, KL divergence, and the data processing inequality
- Channel capacity: Shannon's noisy channel coding theorem and the binary symmetric channel
- Rate-distortion theory and the trade-off between compression and fidelity
- Information-theoretic foundations of ML: MDL, variational inference (ELBO), and the information bottleneck
Learning Outcomes:
Core:
- Compute Shannon entropy and mutual information for discrete distributions and interpret their meaning [Familiarity]
- Apply Shannon's channel capacity theorem to determine the maximum reliable transmission rate [Usage]
- Analyze a learning algorithm using the information bottleneck principle and relate compression to generalization [Assessment]
4.8.7. FDS/Mathematical Foundations of Deep Learning (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Universal approximation, neural tangent kernel, implicit bias of SGD, overparameterization, and geometric deep learning.
Topics:
Core
- Universal approximation theorems: width and depth trade-offs for ReLU networks
- Neural tangent kernel (NTK) and the infinite-width limit of neural networks
- Implicit regularization and implicit bias of SGD toward minimum-norm solutions
- Benign overfitting and double descent in overparameterized models
- Geometric deep learning: equivariance, group symmetry, and graph neural networks
Learning Outcomes:
Core:
- State the universal approximation theorem and identify its assumptions and limitations [Familiarity]
- Explain the neural tangent kernel and its role in understanding training dynamics of wide networks [Usage]
- Analyze the implicit bias of gradient descent and relate it to generalization in overparameterized regimes [Assessment]