- ES Español (Latinoamérica)

- EN English

4.5. Probability and Statistics (PST)
This area covers the mathematical theory of uncertainty and inference, from the axiomatic foundations of probability to modern statistical methodology including Bayesian inference, high-dimensional statistics, and time series analysis.
| Knowledge Area (KA) | Core Tier1 | Core Tier2 |
4.5.1 Probability Axioms and Distributions | 1 | 1 |
4.5.2 Limit Theorems | 1 | 1 |
4.5.3 Stochastic Processes | 1 | 1 |
4.5.4 Estimation Theory | 1 | 1 |
4.5.5 Hypothesis Testing | 1 | 1 |
4.5.6 Bayesian Inference | 1 | 1 |
4.5.7 High-Dimensional Statistics | Elective | |
4.5.8 Time Series Analysis | 1 | 1 |
4.5.9 Nonparametric Methods | 1 | 1 |
4.5.1. PST/Probability Axioms and Distributions (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Kolmogorov's axiomatic foundation of probability, random variables, distributions, and expectations.
Topics:
Core
- Kolmogorov axioms, sigma-algebras, and probability spaces
- Random variables: discrete and continuous distributions, CDF, PDF, and PMF
- Expectation, variance, moments, and moment generating functions
- Joint, marginal, and conditional distributions; independence and covariance
- Standard families: Binomial, Poisson, Normal, Exponential, Gamma, and Beta
Learning Outcomes:
Core:
- State Kolmogorov's axioms and construct a probability space for a standard experiment [Familiarity]
- Compute expectations, variances, and moment generating functions for standard distributions [Usage]
- Determine joint and conditional distributions and assess independence of random variables [Assessment]
4.5.2. PST/Limit Theorems (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Laws of large numbers, central limit theorem, modes of convergence, and large deviations.
Topics:
Core
- Modes of convergence: almost sure, in probability, in \(L^p\), and in distribution
- Weak and strong laws of large numbers and their applications
- Central Limit Theorem: statement, proof via characteristic functions, and Berry-Esseen bound
- Characteristic functions and the continuity theorem (Lévy)
- Large deviations: Cramér's theorem and the Gärtner-Ellis theorem
Learning Outcomes:
Core:
- Distinguish between modes of convergence of random variables and provide counterexamples showing their differences [Familiarity]
- Apply the Central Limit Theorem to approximate distributions of sums and means [Usage]
- Prove the strong law of large numbers and estimate large deviation probabilities using Cramér's theorem [Assessment]
4.5.3. PST/Stochastic Processes (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Markov chains, Poisson processes, Brownian motion, martingales, and stochastic calculus.
Topics:
Core
- Discrete-time Markov chains: classification of states, stationary distributions, and ergodic theorem
- Poisson processes: definition, properties, and generalizations (compound, inhomogeneous)
- Brownian motion: definition, path properties, and the reflection principle
- Martingales, stopping times, optional sampling theorem, and Doob's inequalities
- Itô integral, Itô's formula, and stochastic differential equations
Learning Outcomes:
Core:
- Classify states of a Markov chain and determine its stationary distribution [Familiarity]
- Apply martingale theory and the optional sampling theorem to solve stopping time problems [Usage]
- Formulate and solve a stochastic differential equation using Itô's formula [Assessment]
4.5.4. PST/Estimation Theory (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Point and interval estimation: sufficiency, unbiasedness, efficiency, maximum likelihood, and asymptotic theory.
Topics:
Core
- Sufficient statistics, the factorization theorem, and completeness
- Unbiased estimators, the Cramér-Rao lower bound, and UMVUE
- Maximum likelihood estimation: properties, consistency, and asymptotic normality
- Confidence intervals: pivotal quantities, likelihood ratio, and bootstrap methods
- Asymptotic theory: delta method, M-estimators, and the Fisher information matrix
Learning Outcomes:
Core:
- Explain the Cramér-Rao bound and identify when a UMVUE exists using the Lehmann-Scheffé theorem [Familiarity]
- Derive maximum likelihood estimators and compute their asymptotic variance [Usage]
- Construct confidence intervals using pivotal quantities and bootstrap resampling [Assessment]
4.5.5. PST/Hypothesis Testing (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Neyman-Pearson framework, uniformly most powerful tests, likelihood ratio tests, and multiple testing.
Topics:
Core
- Neyman-Pearson lemma: most powerful tests for simple hypotheses
- Uniformly most powerful (UMP) tests and monotone likelihood ratio families
- Generalized likelihood ratio tests and Wilks' theorem
- \(p\)-values, multiple testing, and false discovery rate (FDR) control
- Nonparametric tests: sign test, Wilcoxon rank-sum, and Kolmogorov-Smirnov
Learning Outcomes:
Core:
- State the Neyman-Pearson lemma and explain the trade-off between type I and type II errors [Familiarity]
- Apply the likelihood ratio test and Wilks' theorem to composite hypothesis problems [Usage]
- Control the false discovery rate in a multiple testing scenario using the Benjamini-Hochberg procedure [Assessment]
4.5.6. PST/Bayesian Inference (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Bayesian paradigm: prior and posterior distributions, conjugate families, hierarchical models, and computational methods.
Topics:
Core
- Bayes' theorem in continuous settings; prior and posterior distributions
- Conjugate prior families: Beta-Binomial, Dirichlet-Multinomial, Normal-Normal
- Hierarchical Bayesian models and empirical Bayes
- Markov chain Monte Carlo (MCMC): Metropolis-Hastings and Gibbs sampling
- Variational inference and the evidence lower bound (ELBO)
Learning Outcomes:
Core:
- Apply Bayes' theorem to update a prior with observed data and obtain a posterior distribution [Familiarity]
- Construct a hierarchical Bayesian model and derive its joint and conditional distributions [Usage]
- Implement Gibbs sampling or Metropolis-Hastings to approximate a posterior distribution [Assessment]
4.5.7. PST/High-Dimensional Statistics ↑ Back to top
Statistical theory in high dimensions: concentration inequalities, sparsity, LASSO, and random matrix theory.
Topics:
Non Core
- Concentration inequalities: Markov, Chebyshev, Chernoff bounds, and sub-Gaussian variables
- Sparse estimation: LASSO, ridge regression, and the restricted isometry property (RIP)
- Random matrix theory: Wigner semicircle law and the Marchenko-Pastur distribution
- Compressed sensing: exact recovery conditions and convex relaxation
- Minimax lower bounds and the bias-variance trade-off in high dimensions
Learning Outcomes:
NonCore:
- Apply concentration inequalities to bound the deviation of sums of random variables [Familiarity]
- Analyze the statistical properties of LASSO estimators under sparsity and RIP conditions [Usage]
- Derive minimax lower bounds for estimation problems using information-theoretic methods [Assessment]
4.5.8. PST/Time Series Analysis (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Stationary processes, ARMA models, spectral analysis, state-space models, and forecasting.
Topics:
Core
- Stationarity, autocovariance function, and the autocorrelation function (ACF)
- ARMA models: identification, estimation (Yule-Walker, MLE), and diagnostics
- Spectral density, the periodogram, and Wiener-Khinchin theorem
- State-space models and the Kalman filter
- ARIMA models, seasonal adjustment, and multi-step forecasting
Learning Outcomes:
Core:
- Identify stationarity and determine the order of an ARMA model from ACF and PACF plots [Familiarity]
- Fit ARIMA models to time series data, validate residuals, and produce forecasts [Usage]
- Apply the Kalman filter to estimate hidden states in a linear Gaussian state-space model [Assessment]
4.5.9. PST/Nonparametric Methods (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Distribution-free inference, rank-based tests, kernel density estimation, and nonparametric regression.
Topics:
Core
- Empirical distribution function, the Glivenko-Cantelli theorem, and the bootstrap
- Rank-based tests: Wilcoxon, Kruskal-Wallis, and Spearman correlation
- Kernel density estimation: bandwidth selection, bias-variance trade-off
- Nonparametric regression: Nadaraya-Watson estimator and smoothing splines
- Permutation tests and randomization inference
Learning Outcomes:
Core:
- Explain the advantages and limitations of nonparametric methods compared to parametric alternatives [Familiarity]
- Apply kernel density estimation and select bandwidth using cross-validation [Usage]
- Conduct permutation and rank-based tests and interpret their results for inference without distributional assumptions [Assessment]