Markov Chain Monte Carlo method (MCMC): Transition from uncertainty to stationary state.
The emergence of MCMC approach in the 1990s led to a rapid evolution of methods to simulate posterior distributions. It can be regarded as MCI through Markov chains. Contrary to other methods that have a static mechanism, MCMC follows a dynamic mechanism whereby samples are generated via a gentle transition through a target distribution function by considering a proposal, eventually converging on a stationary distribution. There are two popular algorithms: Metropolis-Hastings firstly introduced by Metropolis in 1953 and its special case the Gibbs sampler, introduced by Geman and Geman in 1984. Recent developments in this era have provided huge literature, Armitage provided a neat catalogue of the references and summaries (Armitage, Berry, and Matthews, 2001; Armitage et al., 2005). Let us review some technical jargon in MCMC. A chain with the property of being Markov is applied to generate samples which in consequence are dependent. A transition matrix illustrates the probability of movement from one state in the chain to another. It is worth mentioning that the Markov chain should have some properties that guarantee to produce samples from a stationary distribution. By stationary, we meant that, if a sample is generated from a distribution, the next is from the same distribution, as well. Firstly, it should be Irreducible meaning that the Markov chain can get to any state from any state within a finite number of iterations, Persistent or recurrent means returning to the state at least once and Non-Null means finite mean number of transitions. These three conditions had led to a property referred toergodicity by which one can ensure the generation of samples from a stationary distribution. Therefore, the variance of the generated samples is estimable otherwise the chain may behave badly and be effectively useless. It also proves the consistency of estimates. Failure of this method occurs when there are convergence issues. For instance, in the case of a non-persistent chain, convergence to a stationary distribution never occurs. Being Symmetric is another property which influences the acceptance probability of sampling. It means that\(g\left(\theta^{\prime\prime}\middle|\theta^{{}^{\prime}}\right)=g\left(\theta^{{}^{\prime}}\middle|\theta^{\prime\prime}\right)\), (Supplementary figure 3).
The Metropolis-Hastings algorithm to produce a chain of samples by iterative mechanism is defined as the following steps;
  1. Generate a candidate \(\theta^{\left({}^{\prime}\right)}\ \)from proposal distribution\(\text{\ g}\left(.\middle|\theta^{\left(i\right)}\right)\)on which \(\theta^{\left(i\right)}\) is an initial point. The next step is to see whether it is an acceptable sample with a high probability of occurrences and accept it as\(\theta^{\left(i+1\right)}\) or not.
  2. An acceptance rule akin to rejection sampling should be considered here. In this way, at first acceptance probability of the candidate should be estimated. It is the minimum of one and the value of the proportion of multiplication of proposal and target distribution in candidate sample condition on previous point (initial) and vice versa. When the proposal is symmetric, it reduces to the proportion of the values of target on the candidate and initial points (proposal eliminated from nominator and denominator (Supplementary Formula-6).
  3. Now proceed to decide whether to accept the candidate as the next sample or not. So, generate a random number u from uniform (0, 1) distribution.
  4. If u was greater than acceptance probability, choose the candidate as the next sample, otherwise put initial as the next sample and continue the process.