School of Mathematics and Statistics (SoMaS)

MAS6004 Inference

This unit is largely concerned with practical statistical inference. Modern computational tools for the implementation of the frequentist and likelihood-based approaches to inference are explored, with strong emphasis placed on the use of simulation and Monte Carlo methods. Statistical theory is also developed with an introduction to the Bayesian approach to inference and decision making. Computational methods for practical Bayesian inference will also be covered.

There are no prerequisites for this module.
No other modules have this module as a prerequisite.

Outline syllabus

• Semester 1: Bayesian Statistics
  • Subjective probability.
  • Inference using Bayes Theorem. Prior distributions. Exponential families. Conjugacy. Exchangeability.
  • Predictive inference.
  • Utility and decisions. Tests and interval estimation from a decision-theoretic perspective.
  • Hierarchical models.
  • Computation. Gibbs sampling. Metropolis-Hastings. Case studies.
  • Linear regression
• Semester 2: Computational inference
  • Computational methods for likelihoods. Profile likelihood.
  • Simulation. Generating techniques. Monte Carlo integration and variance reduction.
  • Bootstrapping.
  • Simulation and Monte Carlo testing. Randomization tests.

Aims

• To extend understanding of the practice of statistical inference.
• To familiarize the student with ideas, techniques and some uses of statistical simulation.
• To describe computational implementation of likelihood-based analyses.
• To introduce examples of modern computer-intensive statistical techniques.
• To familiarize the student with the Bayesian approach to inference.
• To describe computational implementation of Bayesian analyses.

Learning outcomes

• appreciate the versatility of simulation methods in statistical inference,
• be able to use simulation methods in hypothesis/significance testing and interval estimation,
• understand and apply Bayesian ideas of prior-posterior updating,
• understand the concepts of utility and maximization of expected utility in decision making,
• understand the idea of Gibbs sampling and apply it to practical problems in Bayesian inference.

Teaching methods

Lectures, with a complete set of printed notes, plus task and exercise sheets. Practical sessions using R.

Assessment

One project (S1, 15%), and a three-hour examination (85%). NB: The exam is NOT restricted open book.

Full syllabus

Bayesian theory

• The subjective interpretation of probability. Constructing subjective probabilities.
• Independence and exchangeability.
• Inference using Bayes Theorem. Discrete examples.
• Prior distributions. Exponential families. Conjugacy.
• Continuous examples: normally distributed data with known variance, binomial data.
• Continuous examples: poisson and normal distributions with unknown variance.
• Predictive inference.

• Utility and decisions. Maximising expected utility.
• Point estimation, interval estimation and hypothesis testing from a decision-theoretic perspective.

• Hierarchical models
• Model checking. Robustness. Sensitivity.

• Gibbs sampling.
• MCMC using R
• R practicals: case studies. (practical sessions)

• Monte Carlo integration (3 sessions, 1 practical)

• Randomization tests, confidence intervals based on randomization (1 session)
• Monte Carlo tests (1 session)
• Bootstrap methods; significance tests and bootstrap confidence intervals (3 sessions, 2 practicals)
• Cross validation (1 session)

• Congruential generators, the inversion method (1 session)
• The rejection method (1 session)
• Importance sampling (2 sessions, 1 practical)

• Profile likelihood (2 sessions, 1 practical)

Reading list