Loose Notions

Here are slides that have a good discussion of Markov random fields for image analysis. The most useful part of the slides is a fairly detailed discussion of Bayesian inference, including a clear explanation of the use of conjugate priors (something I have found merely glossed over in many other places).

Summary: when doing Bayesian inference, sometimes the prior knowledge is vague enough for tractability concerns to come into play, so we use a prior distribution on parameters that is compatible with our likelihood function but also leads to a tractable posterior distribution after applying Bayes rule. Conjugate priors let you do this. Formally, if you have a family of likelihood functions L = { p(g|f) | f in F }, then a family of priors P = { p(f|θ) | θ in Θ } is a conjugate family for L if p(g|f) in L and p(f|θ) in P implies p(f|g) in P. The point is that in the Bayesian “posterior ~ likelihood * prior” relationship, you generally have some model form specified for the likelihood function, so you have some wiggle room in choosing the prior. If you choose a prior from the conjugate family for the likelihood function, the math becomes much nicer. Some of the most used examples: the Gaussian family is self-conjugate, and the Dirichlet distribution is conjugate to the multinomial distribution. More detailed examples are worked out in the slides.