Source Evaluation
Advanced Source Analysis
Master the skills needed to evaluate complex information ecosystems where bias is structural, expertise is contested, and misinformation is laundered through credible-seeming intermediaries. These techniques prepare you to critically assess think tank reports, corporate disclosures, expert commentary outside its domain, and the sophisticated influence operations that shape public discourse.
Context
Why this exercise
Advanced source analysis is the discipline of treating evidence the way historians, intelligence analysts, and investigative journalists do: distinguishing primary from secondary sources, evaluating chain-of-custody and provenance, identifying the incentives that shape what gets recorded and preserved, and assigning calibrated credibility rather than binary trust judgments. This exercise drills the moves that distinguish a sophisticated information consumer from a credulous one: source triangulation, provenance tracing, recognition of manufactured 'grassroots' campaigns, and the careful evaluation of leaked or anonymously sourced material.
Before you start
The intellectual foundation here is historical source criticism, developed in the 19th century by Leopold von Ranke and the German historicist school and refined throughout the 20th century by historians of journalism, intelligence, and social movements. The core moves are deceptively simple: identify the primary source (the original document or eyewitness account), evaluate the producer's incentives and access (what would they have known, and what would they have wanted to be true), assess the chain of transmission (how has the source been preserved and copied, and what changes might have entered along the way), and triangulate against independent sources with different incentives. Intelligence analysts use a similar framework codified in CIA tradecraft documents: source reliability, information credibility, and the discipline of separating the source's reliability from the specific claim's plausibility.
Several advanced patterns deserve attention. Astroturfing — manufactured 'grassroots' campaigns funded by interested parties — exploits the credibility heuristic that says spontaneous citizen voices are more trustworthy than corporate or political ones. Documentary analysis of court records, regulatory filings, and academic affiliations reveals funding sources that op-ed bylines never disclose. The 'paper trail' approach used by investigative journalists prioritizes documents that the subject did not control over interviews where the subject controls what is said. Anonymous sourcing requires evaluation of the journalist's track record with similar sources, the specificity of the claims, and whether the anonymous account has been corroborated by independent on-record sources or documents. Leaked documents require provenance analysis: who would have had access, who benefits from the leak, and whether the document could be partially or fully fabricated.
The advanced analyst's habits include maintaining a personal source-reliability scoreboard over time (which outlets and which reporters have been right about which kinds of stories), staying alert to the difference between 'X said' (reportable as fact) and 'X is true' (requiring independent verification), giving disproportionate weight to sources whose claims would be costly for their interests if false (admissions against interest), and recognizing that the absence of expected coverage can be as informative as the presence of unexpected coverage. As you work the scenarios, practice tracing claims back to their primary sources, identifying the incentives shaping what gets reported and what gets omitted, and assigning calibrated credibility rather than binary judgments. For broader treatment of how source evaluation fits into media analysis, see Media Literacy.