Rejection Sampling: Hit the Target!

Contrary to IS which has no restriction to the choice of proposal distribution, RS is pickier. Imagine that our objective is to generate samples from a function. The vital condition is that proposal distributions should cover the target. Consider\(\ g(\theta)\) as a proposal function from which samples generated, (Supplementary 1. Rectangle), \(f(\theta)\) as target and K as a known constant (Supplementary Figure 1. Zigzag triangle). The algorithm is straightforward, generate sample\(\text{\ θ}_{0}\ \)from a proposal and calculate the likelihood of target and proposal functions at this point. If the likelihood ratio of target to proposal was greater than a random sample generated from uniform (0, 1), we accept\(\theta_{0}\mathbf{\ }\)as a sample from target otherwise, reject it and continue the algorithm. In a Bayesian context, this idea could extend to estimating the posterior distribution by considering K as the Maximum likelihood of  θ and prior as proposal, the probability of each generated sample is compared to the value of maximum likelihood\(\text{\ L}_{\text{Max\ }}\) so the proposal resembles posterior distribution, (Supplementary Formula-5) and (Supplementary figure 2). This method fails in high dimensional space parameters due to a decrease in the acceptance rate. In addition, when \(Kg(\theta)\)\(\ \)has considerable distance compared to \(f\left(\theta\right)\), acceptance rate would decrease drastically as well. Likewise, in Bayesian context when \(L_{\text{Max}}\ \)is not practically possible to calculate, this method fails. Armitage presented an example in which by this approach posterior estimation of parameters of a linear regression model had been estimated via various priors as the proposals(4), that using a diffuse prior caused high rejection rate and proved the approach futile.