In this blog post I hope to introduce you to the powerful and simple Metropolis-Hastings algorithm. This is a common algorithm for generating samples from a complicated distribution using Markov chain Monte Carlo, or MCMC. By way of motivation, remember that Bayes' theorem says that given a prior \(\pi(\theta)\) and a likelihood that depends on the data, \(f(\theta | x)\), we can calculate $$ \pi(\theta | x) = \frac{f(\theta | x) \pi(\theta)}{\int f(\theta | x) \pi(\theta) \; \mathrm{d}\theta}. $$ We call \(\pi(\theta | x)\) the posterior distribution. It's this distribution that we use to make estimates and inferences about the parameter \(\theta\). Often, however, this distribution is intractable; it can't be calculated directly. This usually happens because of a nasty integral in the denominator that isn't easily solvable. In cases like this, the best solution is usually to approximate the posterior using a Monte Carlo method.