- ES Español (Latinoamérica)

- EN English

7.60. Optimization I (Elective)
- Semester: 9th Sem. Credits: 4
- Hour of this course: Theory: 2 hours; Practice: 2 hours; Laboratory: 2 hours;
- Syllabus:
- htmlonly

Español (Latinoamérica)

English - Prerrequisites:
- MA221 Numerical Analysis I (5th Sem)
7.60.1. Justification ↑ Back to top
This course introduces the mathematical foundations of optimization, covering both unconstrained and constrained problems. Classical methods of continuous optimization, optimality conditions, numerical algorithms, and applications in various fields are studied. The course provides the theoretical and computational foundations for formulating and solving optimization problems in science, engineering, and economics, developing analytical skills to characterize and find optimal solutions.
7.60.2. Generales Goals ↑ Back to top
- Understand the fundamental concepts of optimization and classify different types of optimization problems.
- Apply optimality conditions to characterize solutions in continuous optimization problems.
- Implement numerical algorithms to solve unconstrained and constrained optimization problems.
- Analyze the convergence and efficiency of different optimization methods.
- Formulate and solve real-world optimization problems in various applied contexts.
7.60.3. Contribution to Outcomes ↑ Back to top
- ABET-1) An ability to identify, formulate, and solve broadly defined technical or scientific problems by applying knowledge of mathematics and science and/or technical topics to areas relevant to the discipline. (Usage)
- ABET-3) An ability to develop and conduct experiments or test hypotheses, analyze and interpret data, and use scientific judgment to draw conclusions. (Usage)
7.60.4. Content ↑ Back to top
7.60.4.1. Convex and Non-Convex Optimization (20 hours) [Skills ABET-1,ABET-3] ↑ Back to top
Bibliography: (Nocedal and Wright, 2006; Boyd and Vandenberghe, 2004)
Topics
- Convex sets and functions
- Optimality conditions
- Gradient methods for convex optimization
- Non-convex optimization challenges
- Convergence analysis
Learning Outcomes
- Characterize convex optimization problems [Assessment]
- Apply optimality conditions to verify solutions [Usage]
- Implement gradient-based methods for convex problems [Usage]
- Analyze challenges in non-convex optimization [Assessment]
7.60.4.2. Constrained Optimization (20 hours) [Skills ABET-1,ABET-3] ↑ Back to top
Bibliography: (Nocedal and Wright, 2006; Boyd and Vandenberghe, 2004)
Topics
- Linear programming (simplex method)
- Integer programming
- Lagrange multipliers and KKT conditions
- Penalty and barrier methods
- Duality theory
Learning Outcomes
- Solve linear programming problems [Usage]
- Formulate and solve integer programming problems [Usage]
- Apply KKT conditions to constrained problems [Usage]
- Use penalty methods for constrained optimization [Usage]
7.60.4.3. Optimization for Machine Learning (20 hours) [Skills ABET-1,ABET-3] ↑ Back to top
Bibliography: (Nocedal and Wright, 2006; Bertsekas, 2016)
Topics
- Stochastic gradient descent
- Momentum and adaptive methods (Adam, RMSProp)
- Regularization and sparsity
- Distributed optimization for ML
- Second-order methods for ML
Learning Outcomes
- Implement stochastic gradient descent variants [Usage]
- Apply adaptive optimization methods [Usage]
- Use regularization for model generalization [Usage]
- Design distributed optimization strategies [Usage]
7.60.4.4. Metaheuristics and Bio-Inspired Optimization (20 hours) [Skills ABET-1,ABET-3] ↑ Back to top
Bibliography: (Talbi, 2009; Mirjalili, 2019)
Topics
- Genetic algorithms and evolutionary strategies
- Swarm intelligence (PSO, ant colony)
- Simulated annealing
- Tabu search
- Hybrid metaheuristics
Learning Outcomes
- Implement genetic algorithms [Usage]
- Apply swarm intelligence methods [Usage]
- Use simulated annealing for global optimization [Usage]
- Design hybrid metaheuristic approaches [Usage]
7.60.5. Bibliography ↑ Back to top
Nocedal, J. and Wright, S. J. (2006). Numerical Optimization. Springer, 2nd edition.
Boyd, S. and Vandenberghe, L. (2004). Convex Optimization. Cambridge University Press, 1st edition.
Bertsekas, D. P. (2016). Nonlinear Programming. Athena Scientific, 3rd edition.
Talbi, E.-G. (2009). Metaheuristics: From Design to Implementation. Wiley, 1st edition.
Mirjalili, S. (2019). Evolutionary Algorithms and Neural Networks: Theory and Applications. Springer, 1st edition.