INSTITUTE OF APPLIED MATHEMATICS
Last Updated:
28/08/2017 - 21:10

IAM528 - Markov Decision Processes

Credit: 3(3-0); ECTS: 8.0
Instructor(s): Consent of IAM
Prerequisites: Consent of Instructor(s)

Course Catalogue Description

The course containing a strong emphasis on applications purports to give students the skills needed in dealing with the control of Markov Chains. The outline of Topics: Discrete-time Markov chains : Ordinary and strong Markov properties, classification of states, stationary probabilities, limit theorems. Continuous-time Markov chains (a survey). Discrete-time Markovian Decision Processes: Various policies, policy-iteration algorithm, linear programming formulation, value-iteration algorithm. Semi-Markov Decision Processes. Applications to inventory problems, to portfolio optimization and to communication systems.

Course Objectives

Course Learning Outcomes

Tentative (Weekly) Outline

The course containing a strong emphasis on applications purports to give students the skills needed in dealing with the control of Markov Chains. The outline of Topics: Discrete-time Markov chains : Ordinary and strong Markov properties, classification of states, stationary probabilities, limit theorems. Continuous-time Markov chains (a survey). Discrete-time Markovian Decision Processes: Various policies, policy-iteration algorithm, linear programming formulation, value-iteration algorithm. Semi-Markov Decision Processes. Applications to inventory problems, to portfolio optimization and to communication systems.

More Info on METU Catalogue

Back