Last Updated:
28/08/2017 - 21:10

IAM557 - Statistical Learning and Simulation

Credit: 3(3-0); ECTS: 8.0
Instructor(s): Ömür Uğur
Prerequisites: Consent of Instructor(s)

Course Catalogue Description

Brief introduction to Statistical Learning: Regression versus Classification; Linear Regression: simple and multiple Linear Regression; Classification: Logistic Regression, Discriminant Analysis; Resampling Methods: Cross-Validation, the Bootstrap; Regularization: Subset Selection, Ridge Regression, the Lasso, Principle Components and Partial Least Squares Regression; Nonlinear Models: Polynomial; Splines; Generalized Additive Models; Tree-Based Models: Decision Trees, Random Forest, Boosting; Support Vector Machines; Unsupervised Learning: Principle Component Analysis, Clustering Methods.

Course Objectives

At the end of the course, the student will learn:
  • the fundamentals of Statistical Learning, regression and classification
  • linear and nonlinear regressions including splines
  • Generalised Additive Models for both regression and classification problems
  • regularisation techniques including Ridge regression and the Lasso
  • the tree-based methods for regression and classification
  • Support Vector Machine which is highly appreciated among Data Science and Machine Learning Community
  • the difference between supervised and unsupervised learning methods

Course Learning Outcomes

Student, who passed the course satisfactorily will be able to:
  • present the data and its descriptive analysis
  • distinguish between regression and classification problems
  • apply regression or classification algorithms to solve related problems
  • code their own algorithms for specific applications in Statistical and Machine Learning
  • understand the fundamentals of Support Vector Machine and be able to apply to specific problems
  • distinguish between supervised and unsupervised learning methods in related applications

Tentative (Weekly) Outline

  1. Brief introduction to Statistical Learning: a) Regression versus Classification
  2. Linear Regression: a) simple and multiple Linear Regression
  3. Classification: a) Logistic Regression b) Discriminant Analysis (Linear and Quadratic)
  4. Resampling Methods: a) Cross-Validation b) the Bootstrap
  5. Regularisation: a) Subset Selection b) Ridge Regression c) the Lasso d) Principle Components Regression e) Partial Least Squares Regression
  6. Nonlinear Models: a) Polynomial and Splines b) Generalised Additive Models
  7. Tree-Based Models: a) Decision Trees b) Random Forest c) Boosting
  8. Support Vector Machines
  9. Unsupervised Learning: a) Principle Component Analysis b) Clustering Methods

Course Textbook(s)

  • Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, An Introduction to Statistical Learning - with Applications in R, 8th ed., Springer, 2013 (Corrected at 8th printing 2017)

Supplementary Materials and Resources

  • Books:
    • Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., Springer, 2009 (Corrected at 12th printing 2017)
    • Kevin P. Murphy, Machine Learning: A Probabilistic Perspective, The MIT Press, 2012
    • Peter Harrington, Machine Learning in Action, Manning Publications Co., 2012
    • Charu C. Aggarwal, Neural Networks and Deep Learning: A Textbook, Springer, 2018
    • G. Jay Kerns, Introduction to Probability and Statistics Using R, 1st ed., 2015
    • Robert V. Hogg, Elliot A. Tanis, Dale Zimmerman, Probability and Statistical Inference, 9th ed., 2015
    • Larry Wasserman, All of Statistics - A Concise Course in Statistical Inference, 2004
    • W. N. Venables, D. M. Smith, and the R Core Team, An Introduction to R - Notes on R: A Programming Environment for Data Analysis and Graphics, Version 3.4.2 (2017-09-28)
  • Resources:
    • The R Project for Statistical Computing:
    • python:
    • RStudio:
    • Anaconda:

More Info on METU Catalogue