EDITION 05 / PSEUDO-RIGOR
The Directional Data Doctrine.
How to make a hunch board-ready without letting it change your mind.
- CATALOG
- AP–015–DDD
- PRINTING
- Fifth, revised upward
- FORMAT
- Landscape quarto (chart-safe)
- EXTENT
- 268 pp. (1 chart, redrawn 4×)
- READING TIME
- 7 min / 45 if you ask about the denominator
- SHELF LIFE
- Until somebody opens the dashboard
THE PROMISE
DOCTRINE / SIGNAL DENSITY MODE
“You do not need data to decide. You need just enough data-adjacent atmosphere that the decision walks into the room already dressed as inevitability — and a chart calm enough that nobody wants to be the person who interrupts it.”
ADVANCE PRAISE / UNSOLICITED
“Restores the number to its proper place: beside the argument, never inside it.”
The Quarterly of Adjacent RigorWinter list, unread
“I have never seen a single line carry so little and mean so much. Chapter six alone justifies the axis.”
Annals of Provisional ConfidencePeer review pending since March
“A landmark work. We were unable to reproduce any part of it, which the authors describe as expected at this stage.”
The DenominatorReproduction attempted once
EXHIBIT A / METRIC NUTRITION LABEL
Present a hunch with a balanced informational diet.
Nobody in a board meeting has ever asked to see the raw table. They ask to see the label. The label is a small, honest-looking object that tells the reader what the number contains, and — this is the craft — it is written by the person who wants the number to mean something.
One customer, on a Tuesday, says the new screen “feels faster.” It is not faster. She was on office wifi.
Plot it against a baseline week of your choosing. Begin the vertical axis at 41. Use one arrow. Blue.
“Early activation signal. Consistent with the thesis. We’ll keep watching it.”
A number that can embarrass you is not a metric. It is an exposure.
— Opening address to the incoming cohort, third printing
THE INSTRUMENT MAP
Not every true number belongs in the room.
Two axes govern the selection of a metric, and neither of them is accuracy. The vertical axis is narrative lift: how warmly the number speaks about the decision you have already made. The horizontal axis is decision safety: the degree to which the number, whatever it reports next quarter, cannot compel anybody to do anything differently. Mature operators instrument the upper right and visit the lower left only in a working session, with the door closed.1
The four positions.
The live wire
Unflattering and consequential. Cohort retention, unsmoothed, by signup month. It reports what happened and implies what to do about it. Your instrumentation lead runs it every Monday without being asked. It has ended a roadmap before.
The humility slide
Unflattering and harmless. One frank number about a thing nobody was going to fund anyway. Carry exactly one per deck. It establishes that you are the kind of leader who shows the bad slides, at a cost of nothing.
The committed figure
Flattering and consequential — the quadrant that ends careers. Say it out loud and somebody will check it in ninety days. If you must use it, use it in a room that has no follow-up on the calendar.
The doctrinal metric
Weekly engaged surfaces, up 34% against a baseline established the same week the definition changed. Nothing follows from it. Everything is supported by it. Put it on the title slide and never speak of it again.
- The lower left is where the instrumentation function prefers to work, and the doctrine recommends honoring that preference by scheduling those sessions at 08:00 on Fridays. ↩
SPECIMEN B / BOARD DECK
The chart and the claim do not have to be about the same thing.
A slide is a compound object. The audience reads the sentence, glances at the shape, and concludes that one produced the other. Nothing in the medium requires this to be true. Nothing in the agenda allows time to check.
The slide as delivered.
WHAT HAPPENED: The line is the count of support tickets nobody could categorize, and it rose in April because a categorization rule broke. It has no relationship to the sentence above it. Nobody asked. The sentence was in the minutes by Thursday, and two teams were re-planned around it in September.
OBJECTION HANDLING / SAMPLE SIZE
How to absorb a good question without letting it reach the decision.
The instrumentation lead is, as a rule, correct. This is what makes the conversation delicate. The objective is never to dispute the objection — disputing it would require engaging with it — but to receive it warmly, praise its rigor, and route it somewhere that has no meeting attached.2
We do not suppress the inconvenient number. We simply decline to give it an arrow.
— Field note, appended to the fourth printing
“What’s the sample size?”
“Small enough to be early. Large enough to be real.”
Answered a number with a temperament.“It’s forty-one people, and thirty-one of them are from one account.”
“Agreed — which is exactly why I’m holding this as directional rather than definitive.”
Conceded the adjective and kept the decision.“The confidence interval crosses zero.”
“This is precisely the rigor I want on this team. Can you take an action to get us a tighter read before the next board?”
Converted a finding into homework, due after the decision.“We stopped the test on day four, while it was ahead.”
“We stopped because we’d learned what we needed. Another ten days would have cost us the whole sprint.”
Priced the truth and declined it.“The definition of an active user changed in March.”
“Right — and the new definition is the one that reflects how the product is actually used now.”
Moved the line and called it product maturity.- Warmth is load-bearing here. An objection that is argued with gets remembered by the room; an objection that is thanked gets filed. ↩
FIELD EXERCISE
Three ways to keep the metric near the thesis and away from the steering wheel.
Choose a number that cannot embarrass you.
Do not measure whether the thing worked. Measure something that moves whenever the thing is touched at all — sessions opened, surfaces engaged, seats provisioned. It will never be zero, and it will never be a verdict.
Make uncertainty sound like maturity.
A contradictory week is not a contradiction. It is a reminder that the team is still learning into a non-linear system, and that seven days is a narrow window on a large change.
Keep the conclusion fixed.
If a measurement would reverse the decision, remove its power by describing it as a snapshot of a system that has not finished becoming itself.
APPENDICES / SUPPORTING APPARATUS
Material retained for completeness.
APPENDIX A Approved language for a number that went the wrong way
Each phrase below preserves the reader’s trust in the instrument while removing the instrument’s authority over the plan. Deploy one. Deploying two in the same meeting reads as anxiety.
- “Too early.” The load-bearing phrase of the discipline. Applies at any point up to and including general availability.
- “Noisy.” Use when the number is stable and you would prefer it were not.
- “A lagging read on a leading change.” Places the disappointing result in the past and the encouraging one in the future.
- “Seasonally distorted.” Every month is adjacent to a holiday somewhere.
- “An artifact of the instrumentation.” Blames the pipeline without naming anyone, which is why it survives review.
- “Directionally fine.” Reserve for results that are not fine and have no direction.
APPENDIX B The dashboard: access log, trailing two quarters
Commissioned in the January offsite as a single source of truth. Built in eleven days by two people who had other work. Referred to in three subsequent offsites as “the source of truth.”
- Opened nineteen times in six months. Fourteen of those by its author, checking whether it still loaded.
- Four of the remaining five occurred in the ninety minutes before a board meeting.
- The fifth was a mis-click from a bookmark bar and lasted eleven seconds.
- The refresh job has failed since 14 March. The failure alert routes to a channel archived during the reorg.
- It is still described as “live” in the pre-read, and this remains true in the sense that the page renders.
APPENDIX C Selecting a baseline week
The comparison week is the only real decision in most charts, and it is made by one person, alone, on a Sunday evening. Treat it with the seriousness it deserves.
- The week of the outage. Everything after it is recovery, and recovery is shaped like growth.
- The week before a public holiday, which lowers the floor without requiring comment.
- Any week containing a deploy freeze, when nothing shipped and nothing moved.
- Index the chosen week to 100 and the reader will supply the meaning unprompted.
- If no single week is suitable, use a four-week trailing average. If the four-week average is also unsuitable, use six.
APPENDIX D Questions received after publication
Reproduced without edit, in order of receipt, with the standing response.
- “Which weeks are excluded from Fig. 9?” — This is addressed in Appendix C.
- “What is the denominator on p. 63?” — This is addressed in Appendix C.
- “Why does the vertical axis begin at 41?” — This is addressed in Appendix C.
- “Appendix C does not address any of these.” — This is addressed in Appendix C.
ERRATA / FIFTH PRINTING
- p. 12. The 3.4× lift in weekly engaged surfaces was 1.1×. An events pipeline recorded each session twice between March and July. The finding stands.
- p. 47. The figure given as “n = 400” refers to sessions, not people. The people number eleven. The eleven were unusually engaged.
- p. 63. The percentages on pp. 58–71 share a denominator that was revised in April to exclude trial accounts. The numerators were not revised. The direction of the effect is unchanged.
- p. 96. The experiment described as having “concluded” was halted on day four, while ahead. Days five through fourteen were also run. They are not reproduced here.
- p. 141. The phrase “statistically significant” appears eleven times in this edition. It was calculated once, for a different metric, in the second printing. The phrase has been retained for continuity.
Measurement is genuinely hard, and honest measurement is harder — which is why the people asking about sample sizes, denominators, and stopping rules are doing the most valuable work in the room. Treating their questions as friction is how an organization ends up confidently wrong at scale.
HUMAN TRANSLATION
Before you show anyone the chart, write down what result would have changed your decision. If there isn’t one, you are not using data — and it is more honest, and usually more persuasive, to say “this is a judgment call, here’s my reasoning” and own it. When you do present a number, say what it measures, what it doesn’t, when the definition last changed, and how many actual people it rests on. And when a colleague asks a hard question about the method, they are not slowing you down; they are the only reason the number is worth anything.