Computational statistics, spring 2013

Lecturer

Petri Koistinen

Scope

8 cu.

Type

Advanced studies

Prerequisites

Basic probability theory at the level of the course Todennäköisyyslaskenta (i.e., familiarity of working with more than one random variables is a prerequisite, but measure theory is not). Basic knowledge of vector and matrix operations (e.g., courses Lineearialgebra ja matriisilaskenta I and Lineaarialgebra ja matriisilaskenta II) as well as of differential and integral calculus. Having had some exposure to Bayesian inference is highly recommended (e.g., course Bayes-päättely). Some experience in computer programming would be helpful.

Lectures

Weeks 3-9 and 11-18, Monday 10-12 in room C122 and Wednesday 12-14 in room B120.

Easter holiday 28.3.-3.4.

News

The course has a moodle page. The exercise problems and their suggested solutions will appear there.

Description

This course gives an overview of computational methods which are useful especially in Bayesian statistics (but some of the methods are also used widely in frequentist inference)

• Review of probability and Bayesian inference.
• Methods for generating independent samples from distributions.
• Classical Monte Carlo integration and importance sampling.
• Approximating the posterior distribution using numerical quadrature or Laplace expansion.
• MCMC methods: Gibbs and Metropolis-Hastings sampling.
• Auxiliary variable methods in MCMC.
• EM algorithm.
• Multi-model inference.
• MCMC theory.

There will be several examples in the lectures and in the exercise sessions which show how the methods can be implemented using the R system for statistical computing. R is convenient for us since it is freely available and widely used and it enables easy visualization of results and contains simulation functions for lots of distributions. However, the methods are in no way tied to the R environment, and the methods can as easily be used in many other environments (such as Matlab together with the Statistics toolbox).

Exams

Two course exams at the end of each of the periods III and IV. Alternatively, a separate exam.

Bibliography

The course is based on lecture notes which will appear here.

• Batch 1: Chapters 1–4 and appendices A and B.
• Batch 2: Chapters 5 and 6.
• Batch 3: Chapters 7--11.

Useful links

• Wikipedia's List of probability distributions is a treasure trove of information on familiar distributions. NB: Wikipedia's parametrization differs in some cases from the parametrization used in the lecture notes.
• Wikipedia: English pronunciation of Greek letters.
• R: R project homepage; download R from CRAN.
• BUGS: OpenBUGS, WinBUGS.
• JAGS, Just Another Gibbs Sampler, a system not wholly unlike BUGS.
• R for Matlab users. 
• MATLAB / R Reference, by David Hiebeler.

Registration

Did you forget to register? What to do.

Exercise group

No exercises on the first week of period III or period IV.

You will get additional points from solving the exercises, which will be added to your points from the two course exams.

The exercise problems and their suggested solutions are distributed on the moodle page.