Skip to main content Skip to navigation

ST411: Dynamic Stochastic Control

Lecturer(s): Dr Gechun Liang

Commitment: 3 x 1-hour lectures per week and one revision class in Term 3. This module runs in Term 1.

Prerequisite(s): ST318 Probability Theory; ST333 Applied Stochastic Processes.

An example of a stochastic control problem is the ‘Red and Black’ problem. Essentially, this asks what the best betting strategy is if you want to maximise your chance of winning £1000 playing roulette.

Syllabus: This module will cover:

  • Recapitulation of the theory of stochastic processes.
  • Introduction to finite horizon control problems and optimal stopping.
  • The Hamilton-Jacobi-Bellman equation.
  • Infinite horizon discounted problems.
  • Applications to finance, clinical trials, planning production processes and insurance, and, time permitting
  • Discussion of long-run average problems.

Aims: This module is designed to cover the important area of stochastic control within applied probability. The taught material will prepare students for careers in business, industry or government and will also lead up to the boundaries of research.

Learning Outcomes: Students who have successfully completed this module will be able to:

  • Identify and deal with stochastic control and optimal stopping problems.
  • Solve simple Hamilton-Jacobi-Bellman equations.
  • Apply the above techniques to finance, to clinical trials and to the planning of production processes.

Books: Ross, S.M., Introduction to Stochastic Dynamic Programming, 1983, Academic Press

Assessment: 100% by 2 hour examination.

You may also wish to see:

ST411: Resources for current students (restricted access)