Partition of a sample space
A partition means cutting all possibilities into pieces E1, E2, …, En that:
- do not overlap (no two can happen together),
- together cover everything (one of them must happen),
- each has a chance more than 0.
Example: a bulb comes from M1, M2 or M3, never two machines. So P(E1) + P(E2) + … + P(En) = 1.
Theorem of total probability
Event A (for example "bulb is faulty") can happen through any piece of the partition. Each path has chance P(Ei) × P(A|Ei) (multiplication rule). The paths do not overlap, so we add them:
P(A) = P(E1)P(A|E1) + P(E2)P(A|E2) + … + P(En)P(A|En)
On a tree diagram: multiply along each branch that ends in A, then add those branches.
Bayes' theorem
Now the question is turned around. We already know A happened. Which piece did it come from? By the definition of conditional probability, P(Ei|A) = P(Ei ∩ A) ÷ P(A). Put the multiplication rule on top and total probability at the bottom:
P(Ei|A) = P(Ei)·P(A|Ei) ÷ Σ P(Ej)·P(A|Ej)
- Prior P(Ei): what we believed before the news.
- Likelihood P(A|Ei): how well case Ei explains the news.
- Posterior P(Ei|A): the updated belief after the news.
All posteriors add up to 1, because A came from exactly one piece.
How to solve any Bayes question in 4 lines
- Name the cases E1, E2, … (the partition) and the news A.
- Write the priors P(Ei) and the likelihoods P(A|Ei).
- Multiply each pair and add: that sum is P(A).
- Answer = the product for the asked case ÷ P(A).
Try it
Put 2 coins in a cup: one normal, one with heads on both sides. Pick one without looking and toss it. You see heads. Predict: is it the double-headed coin? Bayes says: P = (½ × 1) ÷ (½ × 1 + ½ × ½) = 2/3. Repeat 30 times with a friend and count. In the 3D, set shares 50/50 and faults 100% and 50% to see the same 2/3.
Key formulas and definitions
- Partition: E1, E2, …, En do not overlap, cover S, and each P(Ei) > 0
- Total probability: P(A) = Σ P(Ei)·P(A|Ei)
- Bayes' theorem: P(Ei|A) = P(Ei)·P(A|Ei) ÷ Σ P(Ej)·P(A|Ej)
- P(E1|A) + P(E2|A) + … + P(En|A) = 1
- Prior = P(Ei) (before the news); posterior = P(Ei|A) (after the news)
Worked examples
1. Bag I has 3 red and 2 black balls. Bag II has 2 red and 4 black balls. A bag is chosen at random and one ball is drawn. Find the chance it is red.
Step 1: P(I) = P(II) = 1/2. Step 2: P(red|I) = 3/5, P(red|II) = 2/6 = 1/3. Step 3: P(red) = 1/2 × 3/5 + 1/2 × 1/3 = 3/10 + 1/6 = 14/30 = 7/15.
2. In the same bags, the ball drawn is red. Find the chance it came from Bag I.
Step 1: Top = P(I)·P(red|I) = 3/10. Step 2: Bottom = P(red) = 7/15. Step 3: P(I|red) = (3/10) ÷ (7/15) = 45/70 = 9/14.
3. Machines M1, M2, M3 make 50%, 30%, 20% of the bulbs. Their fault rates are 10%, 20%, 40%. A bulb is faulty. Find the chance it came from each machine.
Step 1: Paths: 0.5×0.1 = 0.05, 0.3×0.2 = 0.06, 0.2×0.4 = 0.08. Step 2: P(faulty) = 0.19. Step 3: P(M1|F) = 5/19, P(M2|F) = 6/19, P(M3|F) = 8/19. Check: 5 + 6 + 8 = 19 ✓.
4. A disease affects 1% of people. A test finds it in 95% of ill people but also gives a wrong "positive" to 5% of healthy people. Someone tests positive. Find the chance they are ill.
Step 1: Priors: P(ill) = 0.01, P(well) = 0.99. Step 2: P(+|ill) = 0.95, P(+|well) = 0.05. Step 3: P(+) = 0.0095 + 0.0495 = 0.059. Step 4: P(ill|+) = 0.0095 ÷ 0.059 ≈ 0.161, only about 16%.
5. A student knows the answer to 3/4 of MCQs. If not, she guesses from 4 options. She got a question right. Find the chance she knew it.
Step 1: P(know) = 3/4, P(guess) = 1/4. Step 2: P(right|know) = 1, P(right|guess) = 1/4. Step 3: P(right) = 3/4 + 1/16 = 13/16. Step 4: P(know|right) = (3/4) ÷ (13/16) = 12/13.
6. A man tells the truth 3 times out of 4. He throws a die and says it is a six. Find the chance it really is a six.
Step 1: Cases: six (1/6), not six (5/6). Step 2: P(says six | six) = 3/4, P(says six | not six) = 1/4. Step 3: P(says six) = 1/6 × 3/4 + 5/6 × 1/4 = 3/24 + 5/24 = 8/24. Step 4: P(six | says six) = (3/24) ÷ (8/24) = 3/8.
7. One card from a pack of 52 is lost. Two cards are then drawn from the rest and both are diamonds. Find the chance the lost card was a diamond.
Step 1: P(lost D) = 13/52 = 1/4, P(lost not D) = 3/4. Step 2: If lost D: 12 D in 51 → P = 12/51 × 11/50. If not: 13 D in 51 → 13/51 × 12/50. Step 3: Top = 1/4 × 132 = 33 (all over 2550). Bottom = 33 + 3/4 × 156 = 33 + 117 = 150. Step 4: P = 33/150 = 11/50.
Common mistakes
- Forgetting to check the cases form a partition. If they overlap or miss something, the formula breaks.
- Mixing P(A|Ei) (given in the question) with P(Ei|A) (what Bayes finds).
- Using P(A|Ei) alone as the answer and ignoring the prior P(Ei). A rare case stays fairly rare.
- Dividing by 1 instead of by the total P(A). The bottom is always the sum of all paths to A.