Notes 2

Bayes Theorem

In conditional probability we have a order of probabilities. That is

$$P(A|B) =\frac{P(A\cap B)}{P(B)}$$

But it may not happen that we have the probability in this order or may happen that we may not have the probability of $P(B)$, thats when Bayes Theorm comes into play.

Bayes Theorem lets you swap the order of dependence between events. So we can calculate $P(A|B)$ in terms of $P(B|A)$. This is useful as mentioned above when it is difficult to calculate any one of the term in conditinal probability.

So,

$$P(B|A) = \frac{P(A\cap B)}{P(A)} = \frac{P(A|B)P(B)}{P(A)}$$

Here, the denominator $P(A)$ can be viewed as Normalizing Constant something that ensures we always have a probability function.

So, to evaluate this denominator,

$$P(A\cap B) = P(A|B)P(B)$$ $$P(A|\cap \bar{B}) = P(A| \bar{B})P(\bar{B})$$

Therefore, by additivity rule,

$$P(A) = P(A\cap B) + P(A\cap \bar{B})$$ $$P(A)= P(A|B)P(B) + P(A| \bar{B})P(\bar{B})$$

Hence, the Bayes Theorem becomes,

$$P(B|A) = \frac{P(A|B)P(B)}{P(A|B)P(B) + P(A| \bar{B})P(\bar{B})}$$

Example:Treasure chest A contains 20 Gold and 30 Silver coins, Chest B contains 50 Gold and 50 Silver coins. You randomly choose a treasure chest to open, and then randomly choose a coin from that treasure chest. If the coin you choose is gold, then what is the probability that you chose chest A?

Solution:

Chest A = 20 Gold, 30 Silver

Chest B = 50 Gold, 50 Silver

So let $P(A)$ be the probability of selecting Chest A, and $P(B)$ be the probability of selecting Chest B, and let $P(G|A)$ be the probability of selecting a Gold coint from chest A.

$$P(A) = \frac{1}{2}$$ $$P(B) = \frac{1}{2}$$ $$P(G|A) = \frac{20}{50} = \frac{2}{5}$$ $$P(G|B) = \frac{50}{100} = \frac{1}{2}$$

So, according to Bayes Theorem,Probability of chest being A given that the coin was gold is

$$P(A|G) = \frac{P(G|A)P(A)}{P(G)}$$

Now, we can clearly see the problem here, we don’t have $P(G)$ but we can express it using the additive method explained above. So,

$$P(G) = P(A)P(G|A) + P(\bar{A})P(G|\bar{A})$$

But, $\bar{A}$ in this case is $B$, hence the above equation becomes

$$P(G) = P(A)P(G|A) + P(B)P(G|B)$$

Putting this in above Bayes formula

$$P(A|G) = \frac{P(G|A)P(A)}{P(G)}$$ $$P(A|G) = \frac{P(G|A)P(A)}{P(A)P(G|A) + P(B)P(G|B)}$$

Putting the respective values in we get,

$$P(A|G) = \frac{\frac{1}{2} . \frac{2}{5}}{\frac{1}{2} . \frac{2}{5} + \frac{1}{2} .\frac{1}{2}}$$ $$P(A|G) =\frac{\frac{1}{5}}{\frac{1}{5}+\frac{1}{4}}$$ $$P(A|G) = \frac{1}{5} .\frac{20}{9}$$ $$P(A|G) = \frac{20}{45}$$