Last Updated:
28/08/2017 - 21:10

IAM566 - Numerical Optimization

Credit: 3(3-0); ECTS: 8.0
Instructor(s): Hamdullah Yücel
Prerequisites: Consent of Instructor(s)

Course Catalogue Description

Unconstrained Optimization: steepest descent, line search methods, trust-region methods, conjugate gradient methods, Newton and quasi-Newton methods, large-scale unconstrained optimization, least-square problems; Theory of Constrained Optimization; Linear Programming: simplex method, interior point method; Quadratic Programming; Active Set Methods; Interior Point Methods; Penalty, Barrier and Augmented Lagrangian Methods; Sequential Quadratic Programming.

Course Objectives

Computational science, engineering and applied mathematics face a growing need to develop algorithms, methods, and simulation codes that solve difficult and large scale problems. Solutions are desired that can provide designs, controls, and inversion results for the best choice of input parameters. Numerical optimization algorithms can provide computer scientist, engineers and mathematicians an avenue to the most desirable solution, automate the execution, and achieve efficient convergence rates.

This course is designed for graduate students majoring in mathematics as well as mathematically inclined graduate engineering students. At the end of this course, the student will:

  • learn the central ideas behind algorithms for the numerical solution of differentiable optimization problems by presenting key methods for both unconstrained and constrained optimization, as well as providing theoretical justification as to why they succeed;
  • learn the computational tools available to solving optimization problems on computers once a mathematical formulation has been found.

Course Learning Outcomes

Upon successful completion of this course, the student will be able to

  • recognize the character of an optimization problem (constrained, unconstrained, smooth, nonsmooth) and choose appropriate algorithms for their solutions;
  • understand the basic convergence analysis for the learned optimization methods;
  • solve optimization problems using Matlab, Phyton, Julia or other commercial software;
  • how to use and design efficient numerical optimization algorithms for their own research problems.

Tentative (Weekly) Outline

  1. Fundamentals of unconstrained optimization
  2. Line search methods
  3. Trust-region methods
  4. Conjugate gradient methods
  5. Newton and quasi-Newton methods
  6. Large-scale unconstrained optimization, least-square problems
  7. Theory of Constrained Optimization
  8. Linear Programming: simplex method
  9. Linear Programming: interior point method
  10. Quadratic programming
  11. Active set methods
  12. Interior point methods
  13. Penalty, barrier and augmented Lagrangian methods
  14. Sequential quadratic programming

Course Textbook(s)

  • I. Griva, S. G. Nash and A. Sofer, Linear and nonlinear programming, 2nd edition, SIAM, Philadelphia, 2009
  • J. Nocedal and S. J. Wright, Numerical Optimization, Springer, 1999

Supplementary Materials and Resources

  • Books:
    • J. F. Bannans, J. C. Gilbert, C. Lemaréchel and C. A. Sagastizábal, Numerical Optimization: Theoretical and Practical Aspects, Springer, 2006
    • W. Forst and D. Hoffmann, Optimization -Theory and Practice, Springer, 2010
    • R. Flechter, Practical Methods of Optimization, Wiley, 1987
  • Lecture Notes:
    • B. Karasözen and G.-W. Weber, Numerical Optimization: Constrained optimization, available at IAM Website (download)
  • Resources:
    • MATLAB Student Version is available to download on MathWorks website,, or METU FTP Severs (Licenced)

More Info on METU Catalogue