MAS275 Probability Modelling

Note: Information for future academic years is provisional. Timetable information and teaching staff are especially likely to change, but other details may also be altered, some courses may not run at all, and other courses may be added.

Semester 2, 2022/23 10 Credits
Lecturer: Dr Robin Nicholas Stephenson Home page Timetable Reading List
Aims Outcomes Assessment Full Syllabus

The course introduces a number of general models for processes where the state of a system is fluctuating randomly over time. Examples might be the length of a queue, the size of a reproducing population, or the quantity of water in a reservoir. The aim is to familiarize students with an important area of probability modelling.

Prerequisites: MAS113 (Introduction to Probability and Statistics); MAS211 (Advanced Calculus and Linear Algebra)

The following modules have this module as a prerequisite:

MAS352Stochastic Processes and Finance
MAS371Applied Probability
MAS452Stochastic Processes and Finance


Outline syllabus

  • Introduction to Markov chains
  • Discrete time renewal theory
  • Limiting behaviour of Markov chains
  • Applications of Markov chains
  • Hitting times and probabilities
  • Poisson processes



Aims

  • To introduce and study a number of general models for processes where the state of a system is fluctuating over a period of time according to some random mechanism.
  • To illustrate the above models by example and by simulation.
  • To familiarise students with an important area of probability modelling.

Learning outcomes

  • model a range of situations by writing down transition matrices for suitable Markov chains.
  • calculate and interpret equilibrium probabilities and distributions of Markov chains.
  • calculate and interpret absorption probabilities and expected times to absorption in Markov chains.
  • understand the special properties of the simple Poisson process, perform calculations with them and interpret the results.
  • understand the spatial and inhomogeneous extensions of the Poisson process, and apply them as models of real phenomena.

22 lectures, 5 tutorials

Assessment

One formal 2 hour closed book examination.

Full syllabus

Introduction to Markov chains
Definition, transition probabilities, examples including random walks and gambler's ruin, Chapman-Kolmogorov, stationary distributions.

Discrete time renewal processes
Definition, generating functions, delayed renewal processes.
Limiting behaviour of Markov chains
Classification of states, link to renewal theory, limit theorem, periodic and non-irreducible chains, the renewal theorem.
Applications of Markov chains
Google PageRank.
Hitting times and probabilities
Hitting probabilities, expected hitting times.
Poisson processes
Poisson process: superposition, thinning, conditioning on number of events in an interval. Inhomogeneous and spatial generalisations.

Reading list

Type Author(s) Title Library Blackwells Amazon
C E. Parzen Stochastic Processes 519.23 (P) Blackwells Amazon
C G.R. Grimmett, D.R.Stirzaker Probability and Random Processes 519.2 (G) Blackwells Amazon
C S.M. Ross Introduction to Probability Models 519.2 (R) Blackwells Amazon
C W. Feller An Introduction to Probability Theory and its Applications

(A = essential, B = recommended, C = background.)

Most books on reading lists should also be available from the Blackwells shop at Jessop West.