Evaluating Evidence Quality

Sharpen your ability to weigh evidence like a scientist, distinguishing gold-standard research from persuasive-sounding but unreliable support. You will practice evaluating evidence from medical claims, business reports, and media stories, building the judgment to know when data actually supports a conclusion.

An argument is only as strong as the evidence under it, and most everyday evidence is weaker than it sounds. A statistic with no context, a testimonial with no comparison group, a chart with no baseline. None are lies, and none support the conclusions people hang on them. This exercise trains the judgment to know when data actually earns a claim and when it's being asked to carry too much.

Having some evidence is not the same as having strong evidence. This exercise trains you to weigh support the way a researcher does, distinguishing gold-standard studies from persuasive-sounding but unreliable backing, and to judge relevance, sample quality, and credibility rather than being satisfied that a citation exists at all.

Background

One question catches a huge share of weak arguments: compared to what? A drug where 65% of patients improve sounds great until you learn 40% improved on a sugar pill. Without a comparison, whether a placebo group, a control city, or a baseline rate, a percentage can't tell you whether anything actually worked.

The usual failures cluster: the sample isn't representative of the group the claim is about; a hidden third factor is driving both things; the causation runs backwards; or a single vivid story is standing in for real evidence. For the design side of this, see Scientific Thinking.

Questions

0 of 5 answered

Question 1

A pharmaceutical company promotes a new antidepressant, citing a study where 65% of patients improved after 8 weeks on the drug. A critical thinker should be MOST concerned about which missing piece of information?

Question 2

A news editorial argues that a city's new policing strategy reduced crime, citing a 15% drop in reported crimes over six months. Which additional piece of evidence would MOST weaken this argument?

Question 3

A wellness influencer shares a testimonial: 'I started taking this supplement and my chronic fatigue disappeared within two weeks.' As evidence for the supplement's effectiveness, this testimonial is weak primarily because:

Question 4

A business consultant presents a chart showing that companies using their consulting framework grew revenue 40% faster than industry average. Which question would be MOST important to ask before accepting this evidence?

Question 5

A researcher publishes a study finding that children who play chess score higher on math tests. The study observed 200 children over one year. A journalist writes: 'Chess makes kids better at math.' What is the strongest objection to the journalist's conclusion?

Keep going

Where to go after this exercise.

Next in this categoryIdentifying Hidden AssumptionsDevelop the ability to surface the unstated beliefs that silently hold arguments together. By practicing on arguments drawn from business strategy, policy debates, and everyday reasoning, you will learn to find the invisible load-bearing walls that, if removed, cause the entire argument to collapse.Intermediate5q →Related skillIdentifying Logical FallaciesDevelop a sharp eye for the most common reasoning errors hiding in everyday conversations, news headlines, and social media posts. You will practice distinguishing between arguments that merely sound convincing and those built on genuinely solid reasoning, building the foundational vocabulary needed to name and challenge flawed logic wherever you encounter it.Beginner6q →Read the matching lessonArgument AnalysisCore Skills module · Turn critical thinking from theory into practice: dismantle arguments, question deeply, think like a scientist, and decide well under uncertainty.Read lesson →