the data bubble

Part One · the story

One gym chain's membership file, one question at a time. Each chapter leaves you able to say something you could not say before.

1. 00

   Before the Answer

   The statisticians of the last century were not handicapped by their limited tools. They were sharpened by them. Every answer began with a question, a notebook, and the discipline to doubt the obvious.

   the question Why does more data so often produce less understanding?

   afterwards You can say what rigour is for, and why it comes before speed.

   • try it the members you can see
2. 01

   Memory

   Data began the day humans forgot.

   the question What is a dataset, and what did the act of recording leave out?

   afterwards You can read a table as a record someone made, and ask what it could not see.

3. 02

   The Shape of Data

   The average lies. The shape tells the truth.

   the question The report says one number. Is that where the data actually is?

   afterwards You can look at a distribution before a summary, and say when one number is hiding the shape.

   • figure the average nobody paid
   • figure separate the populations
   • try it the average and the tail
   • try it the ghost segment
   • try it why the average of many draws is a bell
   • reference fifteen shapes data takes
4. 03

   Relationships

   Nothing interesting happens until one variable meets another.

   the question Two things move together. What is that worth, and what is it not?

   afterwards You can read a scatter and a correlation and name the thing nobody measured.

   • try it when the line is wrong
   • reference six shapes two variables can take together
   • reference anscombe's quartet
5. 04

   Data Quality

   Garbage in, garbage out was never about computers.

   the question Some of the rows are blank or strange. Is the dataset still telling the truth?

   afterwards You can ask why a value is missing before deciding what to do about it.

   • try it a log transform straightens a tail
   • try it why it is missing
   • figure three ways to go missing
   • try it mean imputation flattens the truth
6. 05a

   When the Answer Is a Number

   Every model is a lens. Some clarify. Some distort.

   the question The answer is an amount. What has to hold before a line through the data can be trusted to predict it?

   afterwards You can say what a predicted quantity rests on, and read the places it fails as findings rather than flaws.

   • try it three questions, three kinds of answer
7. 05b

   When the Answer Is Yes or No

   A probability is not a verdict. Someone still has to decide what counts as likely enough.

   the question The model says how likely each member is to renew. Who decides what counts as likely enough?

   afterwards You can set the cut-off from what each kind of mistake costs, and say why a model that is usually right can still miss every member who leaves.

8. 05c

   When the Answer Is a Group Nobody Named

   Not every population is one population. Sometimes the data is waiting to be asked.

   the question There is nothing to predict. Is the membership one population, or several nobody has named?

   afterwards You can say what a grouping found and what it left to judgement, and why naming a group is not the same as knowing what made it.

9. 06

   Causation

   Correlation points. Causation explains.

   the question We found a pattern. Can we act on it?

   afterwards You can give three explanations for one pattern and say which one the data has ruled out.

   • try it the class effect
   • figure who was already showing up
   • try it simpson's paradox
   • figure win twice, lose once
10. 07

    The Experiment

    The coin flip is not chance surrendered. It is chance put to work.

    the question How do you learn what a change does, rather than what happened near it?

    afterwards You can say what a designed test earns that an observed difference cannot.

11. 08

    Uncertainty

    Every number is an estimate. The honest ones say so.

    the question The board wants one number. What does the number leave out?

    afterwards You can report a range, and say when the range changes the decision.

    • try it the range around the number
    • figure a hundred samples, one truth
12. 09

    Time

    Some questions live in a moment. Others only reveal themselves across many of them.

    the question The same question, asked over months instead of once. What changes?

    afterwards You can say when order matters, and when a line that bends is not a line that breaks.

    • try it the campaign that did nothing
    • figure the effect that arrived late
13. 10

    Came the Question

    Every good answer begins the same way.

    the question When it all went right, what had been done first?

    afterwards You can say what the question has to look like before any of the chapters above apply.

14. Taking the Answer Back

    The analysis is not finished when the finding is correct. It is finished when the finding connects to a decision.

    the question The result is right. Now what do you say in the room?

    afterwards You can match the precision of what you claim to the decision it feeds.

    • reference what the business says, what the analyst hears

Part Two · the reference

Shapes have names before they have stories. Read in any order.

1. 11

   Fluency

   Knowing the language is not the same as knowing what it is saying.

   the question What does a careful analyst do, in order, every time?

   afterwards You can walk the eight stages from framing to taking it back, and name the sentence for the room at each.

   • try it fits the past, fails the future
   • try it the column from the future
   • try it twenty predictors, one driver
   • reference ten assumptions, quietly load-bearing
   • reference statistical terms, plainly
2. 12

   The Right Tool for the Question

   Knowing every tool is not the same as knowing which one this question needs.

   the question Ten questions a business asks about its members. Which tool does each one need?

   afterwards You can go from a question in plain words to the family of method that answers it.

   • try it the member who hasn't left
   • reference eleven questions, eleven tools

3. Afterword

   A book can teach the discipline. It cannot make you keep it.

also on the site

Worked examples whose subject the book does not teach. No chapter to go back to; they stand on their own.