Bayes' Theorem is a way of calculating the probability of two competing binary hypotheses, by considering previous data. The various components of Bayes' theorem can be more easily explained with an example. Imagine there's a new disease that's emerged called Polynesian Fatigue (PF), only 1% of a population 10,000 strong has PF, but luckily our new test is 95% effective. How worried should you be if you test positive for PF?
Bayes' theorem can calculate the probability of a hypothesis (that you have PF), given an event (that you tested positive for PF), which we can write as follows.
\(P\left(PF\ |\ +\right)\)
To calculate this we need to take the prior probability that the hypothesis is true before you took the test, which is the likelihood that you have the disease at all (a good estimate for this is the frequency of PF sufferers in the population, 1%) and multiply that by the probability of a positive test given the hypothesis is true, which we call the likelihood function. Our likelihood function is that out of a population of 10,000, we would expect 100 people to have PF, and we would expect 95 of them to test positive on the test, 0.95). We then need to divide all that by the total probability of a positive test (which encompasses both the probability that you have PF, and you've tested positive; and the probability that you don't have PF and you have falsely tested positive), we call this the marginal likelihood. Which gives us:
\(P\left(PF\ |\ +\right)\ =\ \frac{P\left(+\ |\ PF\right)\cdot P\left(PF\right)}{P\left(PF\right)\cdot P\left(+\ |\ PF\right)+P\left(-PF\right)\cdot P\left(+\ \ |\ -PF\right)}\)
\(P\left(PF\ |\ +\right)\ =\ \frac{0.95\cdot0.01}{0.01\cdot0.95+0.99\cdot0.05}\)
\(P\left(PF\ |\ +\right)\ =\ 0.161\)
Meaning you only have a 16.1% chance of actually having the disease after you test positive, this is our posterior probability. However if you take another test for PF, and that also comes back positive, what's the odds you have the disease then? This time, however, we update our prior probability, it's no longer the level of PF present in the population (0.01), it's 0.161, we've adjusted our prior probability using data from our previous test.
\(P\left(PF\ |\ +\right)\ =\ \frac{0.95\cdot0.161}{0.95\cdot0.161+0.05\cdot0.839}\)