I bought my mom a copy of Tim Harford’s “The Data Detective” and checked it out myself, meaning to re-read it in parallel with her. She didn’t get to reading it and my re-reading has lagged. It’s now overdue so I’ll miss doing a proper review at this moment; I’ll take this as a sign to buy my own copy. But I want to try paraphrasing his 10 statistical commandments as I at least re-read the last chapter:
- “Search your feelings.” Notice your emotional reaction to a claim but don’t let that determine whether you accept or reject it.
- “Ponder your personal experience.” Combine the abstract, general view provided by statistics with the messy, specific reality provided by our own personal experience, looking for tension and complement. (A related idea I kept in mind as an analyst was that if an effect is real, we should be able to see examples of it happening; not finding any would indicate a bug in the analysis or metrics collection; looking at examples may also cause us to change our interpretation of what the effect really means.)
- “Avoid premature enumeration.” Make sure you understand what the labels on the data actually mean.
- “Step back and enjoy the view.” Find relevant comparisons to put the claim into perspective.
- “Get the backstory.” Think about the origins for the data and what data may be missing.
- “Get the backstory.” Specifically, think about who is missing and how including data for them might change the results.
- “Demand transparency when the computer says no.” Think critically about the algorithms and big datasets. Are they trustworthy? (Even more relevant today, e.g., with the use of AI in screening resumes.)
- “Don’t take the statistical bedrock for granted.” Government collected stats are critical but not guaranteed. (Again, even more critical today.)
- “Remember that misinformation can be beautiful too.” Take care in understanding statistical graphics.
- “Keep an open mind.” Be open to the possibility that you may be wrong.
After listing these, he says: “these suggestions have a common thread — a golden rule, if you like: Be curious. Look deeper and ask questions. It’s a lot to ask, but I hope that it is not too much. At the start of this book I begged you not to abandon the idea that we can understand the world by looking at it with the help of statistics, not to give in to the cynical distrust so temptingly offered by the likes of Darrell Huff.”
The last paragraph: “If we want to make the world add up, we need to ask questions — open-minded, genuine questions. And once we start asking them, we may find it is delightfully difficult to stop.”
Harford mentions an interview with Colbert (in character). The link on Harford’s site is dead, but here’s a copy of it: https://econ.video/2017/10/15/the-colbert-report-tim-harford/.
Some other references related to the last chapter:
- “They Saw a Protest”: Cognitive Illiberalism and the Speech-Conduct Distinction (2012)
- https://www.vox.com/science-and-health/2017/2/1/14392290/partisan-bias-dan-kahan-curiosity
- Science Curiosity and Political Information Processing (2017)
- Political extremism is supported by an illusion of understanding (behind a paywall)
For that second chapter, he points to https://www.gapminder.org/dollar-street.
Leave a Reply