Rejection Sampling: Hit the
Target!
Contrary to IS which has no restriction to the choice of proposal
distribution, RS is pickier. Imagine that our objective is to generate
samples from a function. The vital condition is that proposal
distributions should cover the target. Consider\(\ g(\theta)\) as a
proposal function from which samples generated, (Supplementary 1.
Rectangle), \(f(\theta)\) as target and K as a known constant
(Supplementary Figure 1. Zigzag triangle). The algorithm is
straightforward, generate sample\(\text{\ θ}_{0}\ \)from a proposal and
calculate the likelihood of target and proposal functions at this point.
If the likelihood ratio of target to proposal was greater than a random
sample generated from uniform (0, 1), we accept\(\theta_{0}\mathbf{\ }\)as a sample from target otherwise, reject it
and continue the algorithm. In a Bayesian context, this idea could
extend to estimating the posterior distribution by considering K as the
Maximum likelihood of θ and prior as proposal, the
probability of each generated sample is compared to the value of maximum
likelihood\(\text{\ L}_{\text{Max\ }}\) so the proposal resembles
posterior distribution, (Supplementary Formula-5) and (Supplementary
figure 2). This method fails in high dimensional space parameters due to
a decrease in the acceptance rate. In addition, when \(Kg(\theta)\)\(\ \)has considerable distance compared to \(f\left(\theta\right)\),
acceptance rate would decrease drastically as well. Likewise, in
Bayesian context when \(L_{\text{Max}}\ \)is not practically possible to
calculate, this method fails. Armitage presented an example in which by
this approach posterior estimation of parameters of a linear regression
model had been estimated via various priors as the
proposals(4), that using a diffuse prior
caused high rejection rate and proved the approach futile.