Three fundamental Bayesian computational sampling algorithms implemented from scratch in R — Inverse CDF, Rejection Sampling, and Metropolis-Hastings MCMC.
This project implements and compares three core simulation methods used in Bayesian inference and statistical computing, each targeting a different probability distribution to demonstrate how the techniques work and when to apply them.
| Method | Target Distribution | Acceptance Rate |
|---|---|---|
| Inverse CDF Sampling | f(x) ∝ exp(x), x ∈ [0,1] | 100% (exact) |
| Rejection Sampling | f(x) = 4x³, x ∈ [0,1] | ~25% (M=4) |
| Metropolis-Hastings MCMC | f(x) ∝ exp(−x), x ≥ 0 | ~75% (σ=0.5) |
Each method includes density overlay plots for visual verification against the true distribution.
Analytically derives the inverse CDF and applies the probability integral transform. Exact and efficient — works when the CDF is invertible in closed form.
Proposes candidates from a Uniform(0,1) envelope and accepts with probability proportional to the target density. Demonstrates the trade-off between envelope tightness and acceptance rate.
Constructs a Markov chain whose stationary distribution matches the target. Uses a Normal proposal distribution truncated to the support x ≥ 0. Tracks acceptance rate and convergence diagnostics.
All three methods successfully recover the target distributions:
- Inverse CDF: mean ≈ 0.418 (matches theoretical E[X] = 1 − 1/(e−1))
- Rejection sampling: mean ≈ 0.800 (matches theoretical E[X] = 4/5 for f(x) = 4x³)
- MH MCMC: mean ≈ 1.000, variance ≈ 1.000 (matches Exp(1))
- Language: R (base)
- No external packages required
# No package installation needed — uses base R only
source("bayesian_sampling_methods.R")Each method prints summary statistics and generates a histogram with the true density overlaid in red.
bayesian_sampling_methods.R # Full annotated R script
README.md
Eeshal Rizwan
BSc Statistical Data Science — Heriot-Watt University Dubai
linkedin.com/in/eeshalrizwan