Markov Chain Monte Carlo method (MCMC): Transition from
uncertainty to stationary state.
The emergence of MCMC approach in the 1990s led to a rapid evolution of
methods to simulate posterior distributions. It can be regarded as MCI
through Markov chains. Contrary to other methods that have a static
mechanism, MCMC follows a dynamic mechanism whereby samples are
generated via a gentle transition through a target distribution function
by considering a proposal, eventually converging on a stationary
distribution. There are two popular algorithms: Metropolis-Hastings
firstly introduced by Metropolis in 1953 and its special case the Gibbs
sampler, introduced by Geman and Geman in 1984. Recent developments in
this era have provided huge literature, Armitage provided a neat
catalogue of the references and summaries (Armitage, Berry, and
Matthews, 2001; Armitage et al., 2005). Let us review some technical
jargon in MCMC. A chain with the property of being Markov is applied to
generate samples which in consequence are dependent. A transition matrix
illustrates the probability of movement from one state in the chain to
another. It is worth mentioning that the Markov chain should have some
properties that guarantee to produce samples from a stationary
distribution. By stationary, we meant that, if a sample is generated
from a distribution, the next is from the same distribution, as well.
Firstly, it should be Irreducible meaning that the Markov chain
can get to any state from any state within a finite number of
iterations, Persistent or recurrent means returning to the state
at least once and Non-Null means finite mean number of
transitions. These three conditions had led to a property referred toergodicity by which one can ensure the generation of samples from
a stationary distribution. Therefore, the variance of the generated
samples is estimable otherwise the chain may behave badly and be
effectively useless. It also proves the consistency of estimates.
Failure of this method occurs when there are convergence issues. For
instance, in the case of a non-persistent chain, convergence to a
stationary distribution never occurs. Being Symmetric is another
property which influences the acceptance probability of sampling. It
means that\(g\left(\theta^{\prime\prime}\middle|\theta^{{}^{\prime}}\right)=g\left(\theta^{{}^{\prime}}\middle|\theta^{\prime\prime}\right)\),
(Supplementary figure 3).
The Metropolis-Hastings algorithm to produce a chain of samples by
iterative mechanism is defined as the following steps;
- Generate a candidate \(\theta^{\left({}^{\prime}\right)}\ \)from proposal
distribution\(\text{\ g}\left(.\middle|\theta^{\left(i\right)}\right)\)on which \(\theta^{\left(i\right)}\) is an initial point. The next
step is to see whether it is an acceptable sample with a high
probability of occurrences and accept it as\(\theta^{\left(i+1\right)}\) or not.
- An acceptance rule akin to rejection sampling should be considered
here. In this way, at first acceptance probability of the candidate
should be estimated. It is the minimum of one and the value of the
proportion of multiplication of proposal and target distribution in
candidate sample condition on previous point (initial) and vice versa.
When the proposal is symmetric, it reduces to the proportion of the
values of target on the candidate and initial points (proposal
eliminated from nominator and denominator (Supplementary Formula-6).
- Now proceed to decide whether to accept the candidate as the next
sample or not. So, generate a random number u from uniform (0,
1) distribution.
- If u was greater than acceptance probability, choose the
candidate as the next sample, otherwise put initial as the next sample
and continue the process.