Last Updated:
28/08/2017 - 21:10

IAM566 - Numerical Optimization

Credit: 3(3-0); ECTS: 8.0
Instructor(s): Bülent Karasözen
Prerequisites: Consent of Instructor(s)

Course Catalogue Description

Unconstrained optimization: line search methods, steepest descent, Newton and quasi Newton methods, the conjugate gradient method constrained optimization: equality and inequality constraints, linear constraints and duality, linear programming, the simplex method, Lagrange multiplier algorithms, interior point methods, penalty methods, large scale optimization.

Course Objectives

  • The objective of this course is to introduce the central ideas behind algorithms for the numerical solution of differentiable optimization problems by presenting key methods for both unconstrained and constrained optimization, as well as providing theoretical justification as to why they succeed.

Course Learning Outcomes

  • At the end of this course students should be able to tackle optimization problems of in science, engineering and finance using state of art numerical methods. Both lectures and exercises serve for this aim of learning, deepening, applying and preparing.

Tentative (Weekly) Outline

  1. Fundamentals of unconstrained optimization
  2. Newton methods
  3. Line-search methods
  4. Trust-region methods
  5. Quasi-Newton methods
  6. Nonlinear least-squares problems
  7. Conjugate Gradient methods
  8. Theory of constrained optimization
  9. Theory of linear programming
  10. Simplex method
  11. Interior point methods
  12. Active index-set strategy
  13. Penalty and barrier methods
  14. Sequential quadratic programming

Course Textbook(s)

  • I. Griva, S. G. Nash and A. Sofer, Linear and nonlinear programming, 2nd edition, SIAM, Philadelphia, 2009

Supplementary Materials and Resources

  • Books:
    • W. Forst and D. Hoffmann, Optimization – Theory and Practice, Springer, 2010
    • J. F. Bannans, J. C. Gilbert, C. Lemaréchel and C. A. Sagastizábal, Numerical Optimization: Theoretical and Practical Aspects, 2nd edition, Springer, 2006
    • J. Nocedal and S.J. Wright, Numerical Optimization, Springer, 1999
    • R. Flechter, Practical Methods of Optimization, Wiley, 1987
  • Lecture Notes:
    • Lecture Notes are prepared by B. Karasözen and G.-W. Weber on the second part of the course - Constrained Optimization - and electronically available from IAM Lecture Notes Series (
    • Moreover, handwritten lecture notes and selected chapters of books, further lecture notes, presentations and additional exercises will be provided during the semester.
  • Resources:
    • MATLAB Student Version is available to download on MathWorks website,, or METU FTP Severs (Licenced)

More Info on METU Catalogue