- ES Español (Latinoamérica)

- EN English

2.6. Optimization and Machine Learning (OML)
This area covers the mathematical foundations and computational methods for optimization, with special emphasis on techniques relevant to machine learning. It includes convex and non-convex optimization, constrained optimization, large-scale optimization methods, and metaheuristics. The area bridges traditional optimization theory with modern machine learning applications, providing the tools needed to train complex models and solve large-scale optimization problems efficiently.
| Knowledge Area (KA) | Core Tier1 | Core Tier2 |
2.6.1 Convex and Non-Convex Optimization | 1 | 1 |
2.6.2 Constrained Optimization | 1 | 1 |
2.6.3 Optimization for Machine Learning | 1 | 1 |
2.6.4 Large-Scale and Distributed Optimization | 1 | 1 |
2.6.5 Metaheuristics and Bio-Inspired Optimization | 1 | 1 |
2.6.1. OML/Convex and Non-Convex Optimization (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Topics:
Core
- Convex sets and functions
- Optimality conditions
- Gradient methods for convex optimization
- Non-convex optimization challenges
- Convergence analysis
Learning Outcomes:
Core:
- Characterize convex optimization problems [Assessment]
- Apply optimality conditions to verify solutions [Usage]
- Implement gradient-based methods for convex problems [Usage]
- Analyze challenges in non-convex optimization [Assessment]
2.6.2. OML/Constrained Optimization (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Topics:
Core
- Linear programming (simplex method)
- Integer programming
- Lagrange multipliers and KKT conditions
- Penalty and barrier methods
- Duality theory
Learning Outcomes:
Core:
- Solve linear programming problems [Usage]
- Formulate and solve integer programming problems [Usage]
- Apply KKT conditions to constrained problems [Usage]
- Use penalty methods for constrained optimization [Usage]
2.6.3. OML/Optimization for Machine Learning (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Topics:
Core
- Stochastic gradient descent
- Momentum and adaptive methods (Adam, RMSProp)
- Regularization and sparsity
- Distributed optimization for ML
- Second-order methods for ML
Learning Outcomes:
Core:
- Implement stochastic gradient descent variants [Usage]
- Apply adaptive optimization methods [Usage]
- Use regularization for model generalization [Usage]
- Design distributed optimization strategies [Usage]
2.6.4. OML/Large-Scale and Distributed Optimization (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Topics:
Core
- First-order methods for large-scale problems
- Coordinate descent and block methods
- Consensus and distributed optimization
- Federated learning optimization
- Communication-efficient distributed optimization
Learning Outcomes:
Core:
- Apply first-order methods to large-scale problems [Usage]
- Implement coordinate descent algorithms [Usage]
- Design consensus-based distributed algorithms [Usage]
- Optimize federated learning systems [Usage]
2.6.5. OML/Metaheuristics and Bio-Inspired Optimization (Core Tier1: 1 hr, Core Tier2: 1 hr) ↑ Back to top
Topics:
Core
- Genetic algorithms and evolutionary strategies
- Swarm intelligence (PSO, ant colony)
- Simulated annealing
- Tabu search
- Hybrid metaheuristics
Learning Outcomes:
Core:
- Implement genetic algorithms [Usage]
- Apply swarm intelligence methods [Usage]
- Use simulated annealing for global optimization [Usage]
- Design hybrid metaheuristic approaches [Usage]