By Tajammal MaqboolFounder & Developer
Master the framework that doctors, intelligence analysts, and data scientists use to update beliefs rationally when new evidence arrives. These exercises build your intuition for Bayes' theorem through scenarios involving criminal investigations, medical diagnosis, A/B testing, and geopolitical analysis. You will learn when evidence should dramatically change your mind versus when it should barely shift your confidence.
Bayesian reasoning is the discipline of updating your beliefs in proportion to the evidence. Raw intuition treats new evidence as a verdict: the test was positive, so I have the disease. Bayesian thinking treats it as one input, combining with the prior odds to give an updated answer. This exercise drills the moves: spotting the prior, the evidence strength, and the update, and catching the classic failures that produce confident wrong answers.
Bayesian reasoning is the framework for updating beliefs proportionally when new evidence arrives, used by doctors, analysts, and forecasters. This exercise trains you to weigh a new piece of evidence against the prior probability, which is what prevents a striking test result from overwhelming a low base rate.
The counterintuitive part is that when the prior is low, even strong evidence leaves the answer lower than you'd expect. A 90%-accurate test for a 1-in-1,000 disease still turns up more false alarms than real cases, so a positive result is far from a diagnosis. The math is one line; the trap is trusting the vivid number and forgetting the base rate.
The most consequential failure is the prosecutor's fallacy: confusing 'the chance an innocent person would match this evidence' (tiny) with 'the chance this person is innocent given the match' (which depends on the prior). In the Sally Clark case, a mother was wrongly convicted after a 1-in-73-million figure was treated as her probability of innocence. Translate into natural frequencies to see clearly. See Types of Reasoning.
Question 1
A detective investigating a crime has three suspects. Forensic evidence arrives that is 10 times more likely if Suspect A is guilty than if innocent. The detective says, 'Case closed. Suspect A did it.' His partner says the conclusion is premature. If Suspect A initially had only a 5% chance of guilt, what is the updated probability after this evidence?
Question 2
A patient visits an emergency room with a severe headache. An intern immediately orders an MRI to rule out a brain tumor. The attending physician stops him: 'When you hear hoofbeats, think horses, not zebras. Start with tension headache and migraine workup.' The intern argues, 'But what if it IS a tumor? We'd be negligent.' Who is reasoning better?
Question 3
A tech company runs an A/B test on a new checkout button color. After 200 visitors (100 per group), the new color shows a 5.2% conversion rate versus 4.1% for the old color. The product manager says, 'The new color is 27% better, so ship it!' The data scientist says, 'We need more data.' With these sample sizes, the result has a p-value of 0.35. Who should you listen to?
Question 4
A city installs an AI camera system that identifies stolen cars with 99% accuracy (correctly flags 99% of stolen cars, correctly ignores 99% of non-stolen ones). About 1 in 10,000 cars on the road is stolen. The police chief tells officers to pursue every flagged car. A civil liberties lawyer objects. What is the actual probability that a flagged car is stolen?
Question 5
Two intelligence analysts assess whether a country is developing nuclear weapons. Both start at 20% confidence. Analyst A reads a new intelligence report and jumps to 90%. Analyst B reads the same report, notes that such reports have historically been wrong 60% of the time, and shifts to 35%. Which analyst is reasoning better?
Question 6
An election forecaster's model gives Candidate X a 30% chance of winning. Candidate X wins. A pundit says, 'The model was wrong. It said X would lose.' A statistician says the model may have been perfectly calibrated. Who is right?
Where to go after this exercise.
