databubble

chapter 00 · before the answer

the members you can see

"The engineers examined the planes that came back." The ones you can see are never the whole population — they are the ones that survived long enough to be counted.

visits/month → satisfaction
renewal rate by visit frequency, same population as above
On the platformthe platform reports n alongside every r — a correlation on 1,307 answered members carries a different n than one on all 2,000, and both are shown, so a shrinking sample doesn't hide behind an unchanged-looking number.

chapter 02 · the shape of data

the average and the tail

"When the mean and median diverge, the distribution is skewed. The gap between them tells you how much the average is being distorted by the tail."

the average describes ~— % of members
median mean
Symmetric. Mean, median, and mode coincide. The average is an honest description of where the data sits.
On the platformunivariate.py reports skewness and flags when the mean stops being an honest summary.

chapter 02 · the shape of data

the ghost segment

"The average lands inside the majority. It says nothing about the quarter who are barely here at all."

pooled mean
0 visits/month18+ visits/month
One merged distribution. A single average describes the whole crowd.
On the platformdistribution_matching.py runs a dip test — it catches bimodality that skewness alone reports as zero.

chapter 02 · the shape of data

why the average of many draws is a bell

"The data isn't normal, so none of this applies." The data doesn't have to be. Only the average of many draws does — and it gets there fast.

The population — every transaction, unfiltered

$0$300+

300 samples of size n, each reduced to its own mean

$0$100
n=2: barely averaged at all. The sample mean's own distribution still looks almost as skewed as the population it was drawn from.
On the platformevery confidence interval the platform reports leans on this — the sample mean's own distribution, not the raw data's, is what needs to be close to bell-shaped.

chapter 03 · relationships

when the line is wrong

"A straight line through curved data is not an approximation. It is a systematic error that changes direction across the range."

0 12 24 visits / month
0 km (next door)10 km (edge of catchment)
Straight, and honestly so. A straight line is not wrong here — it is the correct description of the relationship.
On the platformnonlinearity.py tests the residual pattern itself, not just the fit — a straight line can have a decent R² and still be systematically wrong.

chapter 04 · data quality

a log transform straightens a tail

"Logs are a statistician's trick." Not a trick — a change of ruler. Slide λ from 1 (the ruler you're used to) to 0 (a ruler marked in percent) and watch the same real data read differently.

median mean
0.3 km28 km
Distance from the gym, raw kilometres. Most members live close; a long tail lives far — skewness 2.00.
On the platformtransformations.py tests a Box-Cox range and picks λ by what minimises skew, not by convention.

chapter 04 · data quality

why it is missing

"The instinct, when you see a column with missing values, is to ask how many are missing. That is the wrong first question. The right question is why."

chapter 04 · data quality

Interactive figure. An exit survey of 800 lapsed members under four conditions. With full response, ‘poor experience’ is 20% of reasons. When non-response is random the survey stays close to truth; when it follows an observable trait it distorts but can be reweighted back; when the unhappy themselves stop answering, ‘poor experience’ falls to 7% and no correction from the collected data can recover it.
Field note: "Three Ways to Go Missing" — coming soon
Grounded in the data

This isn't hypothetical. ActiveEdge's real satisfaction_score column is 34.6% missing — and the gap tracks exactly who the ghost segment already flagged as barely here.

visits < 2/month
67%
everyone else
23%
On the platformmissing_values.py diagnoses the mechanism before anything is imputed, and refuses to impute under MNAR without a stated assumption.

chapter 04 · data quality

mean imputation flattens the truth

"Just fill the blanks with the average — it doesn't change the mean." True of the mean. Watch what happens to everything else.

1 (poor)10 (excellent)
Every observed score, no gaps yet. Move the slider to start hiding some.
On the platformTest card: Detecting Missingness Mechanism — the platform checks the mechanism before choosing how (or whether) to impute.

chapter 05a · when the answer is a number

three questions, three kinds of answer

"Every model is a lens. Some clarify. Some distort."

0 months tenure36 months tenure
Both scores move together here. Fifteen ActiveEdge records worth of real signal, no wasted budget yet.
On the platformmodel_selection.py picks between these three before you do — and reports adjusted R², not raw, by default.

chapter 06 · causation

the class effect

"The gym chain saw its class-goers renew more and prepared to expand the program. It should have asked one question first: who was already choosing to show up?"

chapter 06 · causation

Animated figure. Members who attend classes renew at 67% versus 33% for those who don't — almost a 35-point gap. But class attendees are mostly already-frequent visitors, who renew more regardless. Splitting both groups by how often they already visit shrinks the gap to about 10 points: most of the difference was visit frequency, not classes.
Not wrong, just incomplete. The naive gap was real; most of it just belonged to how often they were already visiting, not to the classes themselves.
On the platformcausal_psm_ipw.py matches or reweights by the confounder before comparing groups — the same idea as this figure's visit-frequency split, done properly.

chapter 06 · causation

simpson's paradox

"Standard plan members renew better than Premium overall. Split by tenure and Premium wins in both groups. Both statements are true."

chapter 06 · causation

Interactive figure. Standard plan members renew at 67% overall versus 53% for Premium. Split by tenure, Premium renews better in both the shorter-tenure half (46% vs 38%) and the longer-tenure half (82% vs 74%). The reversal happens because 80% of Premium members are new, and new members renew less on any plan.
On the platformThis split is illustrative, not read from ActiveEdge's own membership_type column — the real data shows no such reversal on any plan split checked. Kept as a clean, worked example of the mechanism itself.
Full story: "The Numbers That Win Twice and Lose Once" — coming soon

chapter 08 · uncertainty

the range around the number

"The slide said five kilometres. It did not say between 4.2 and 5.8 kilometres, at the current level of precision."

chapter 08 · uncertainty

Animated figure. One hundred random samples of 200 members are drawn from the same 2,000. Each sample's estimated renewal rate and its 95% confidence interval are stacked as rows. The estimates scatter around the true 60%; 93 of the 100 intervals contain the truth and 7 miss it (95 is the long-run expectation). A real study is a single row, and you cannot tell which.
0 draws yet.
Every draw is one throw at the same stake. Most throws land around it — but "most" is not "all," and the width you see here is how far a throw can miss.
On the platformevery estimate the platform reports ships with its range attached — never a bare number on its own.

chapter 09 · time

the campaign that did nothing

"The November report showed nothing. AE almost pulled the campaign — three months before it actually started working."

chapter 09 · time

Animated figure. A win-back campaign launches in October of year one. The next three months show no change in renewals — the naive read says it failed. But renewals can only move when members come up for renewal: from month 13 onward the rate runs about 4.6 points above its seasonal pattern, measured against the same months a year earlier.
Judged at launch, the campaign failed. Judged on the cohorts it could actually reach, it worked — just late.
On the platformmedia_transforms.py's geometric_adstock() models exactly this: a campaign's effect spread across future periods with decay, never credited to the launch month alone.

chapter 11 · fluency

fits the past, fails the future

"The more flexible model fits better, so it is better." It fits the 60 points you gave it better. Watch what it does to the ones you didn't.

0.3 km28 km
train (60, fixed)held-out (40, redraws)
Degree 1: a straight line. It can't chase every wiggle in these 60 points — but that's not obviously a flaw.
On the platformmodel_selection.py checks held-out performance before reporting a fit, precisely so a curve like this can't pass by only looking good on the data it was trained on.

chapter 11 · fluency

the column from the future

"99% accuracy in testing" with no suspicion. A model predicting renewal, built on ActiveEdge's own real columns, including the two with an underscore in their name — the ones that look hidden, internal, suspicious.

renewal decision 9 legit features _lapsed days since last visit
known earlierknown only after
Nine honest features, all knowable before the renewal decision.
On the platforma leakage check compares every feature's timestamp to the prediction point before training starts — not a naming convention, an actual clock.

chapter 11 · fluency

twenty predictors, one driver

"We screened twenty predictors and found a significant driver." Screen twenty real columns that have nothing to do with the outcome, and by chance alone, about one of them will look significant too.

Every cell here is a real ActiveEdge column — age, distance, sessions, satisfaction, all nine of them you've already seen elsewhere on this page — reshuffled so any real relationship to renewal is destroyed. Any coral cell is a false alarm.
On the platformscreening many candidate predictors always ships with a correction — Benjamini-Hochberg by default — precisely so this chance alarm doesn't get reported as a finding.

chapter 12 · the right tool for the question

the member who hasn't left

"Still-active members are missing data — drop them." Or: "count them as churned at the cutoff." Both give you a confident number. Only one of them is honest about what you don't know yet.

80 member timelines, at this cutoff retention curves — % still active 50%
drop censored (grey), count as churned (coral), Kaplan-Meier (teal)
On the platformsurvival_model.py fits Kaplan-Meier by default and reports "not yet reached" explicitly rather than a number that implies more certainty than the data has.

also on the site

the p-value that peeked

"The p-value crossed 0.05 on day 9, so we stopped and shipped." Two arms with the real 57.25% renewal rate on both sides — genuinely no difference — checked every day until one of them looked significant.

day 1day 40
A single A/A test — same true renewal rate on both arms. The teal line is today's p-value; the dashed line is 0.05.
On the platformtest cards specify a planned sample size up front — the platform doesn't let an experiment's stopping point be decided by which day the p-value happened to look good.

also on the site

two lines that had nothing to do with each other

"These two series move together — r = 0.8 — so one must be driving the other." Two random walks, generated with no connection to each other whatsoever, routinely correlate this strongly. Persistence alone is enough to manufacture it.

One pair — sixty months, generated with no connection to each other

month 1month 60

Correlation across 200 independent pairs at this persistence

r = −1r = 0r = +1
Two series with no relationship, both generated independently.
On the platforma Durbin-Watson or stationarity check runs before any regression on time-ordered data — it's the test that catches this before "r = 0.8" reaches a dashboard.

also on the site

western electric rules

"Nothing breached the control limit, so the process is fine." A single point beyond three standard deviations is the blunt signal. A slow, sustained drift can be caught much earlier — by a pattern, not a threshold.

36 monthly renewal-rate readings — a slow drift starts at month 18
On the platformspc_control_chart.py runs all six Western Electric and Nelson run rules on every chart, not just the 3σ threshold.

also on the site

the month everyone remembers

"The manager remembered exactly why the bad month happened. The record remembered something else — that it wasn't unusual at all."

Animated figure. Twenty-four months of member lapse rates. One month spikes and gets remembered with a story attached. Against the seasonal pattern and its expected chance range, a worst month of this size is typical: in sixty simulated re-runs of the same two years, 21 produced a worse month with no story at all.
Every 24-month history has a worst month by chance alone. Remembering the spike and its story is not evidence the story is true.
On the platformoutliers_univariate.py asks the same question of any spike before anyone builds a story around it: is this extreme relative to the actual distribution, or just the most memorable point in it?