A system that answers everything asked of it is answering some things badly. Defending the boundary is what keeps it trusted on the rest.
Whether someone is working hard
Why not: rate reflects the work available, the conditions, the equipment and the layout far more than effort. A slow figure in a badly slotted aisle is a slotting measurement.
Use instead: supervision, which involves being present and knowing the work.
Whether a target is achievable
Why not: the data shows what happened, not whether it was sustainable. Confusing the two is the mechanism behind rate-driven injury.
Use instead: observation of the work at the target rate, and asking whether the safe method survives.
Why something happened
Why not: the data shows sequence, not reason.
Use instead: asking the people who were there, which is also how you find out what the data got wrong.
Whether a change caused an improvement
Why not: ordinary variation, regression to the mean and the novelty effect all produce apparent improvement.
Use instead: a control area or the historical series, a settling period, and the normal range.
What happens between scans
Why not: the gap is unexplained by definition — travel, waiting, searching or the work itself.
Use instead: positioning where it is justified, or observation, which is cheaper for a one-off question.
Whether the process is being followed
Why not: the system records what was entered, which in a facility with workarounds is not what happened.
Use instead: watching the work, which reliably finds practices no system records.
Near-misses that nobody reports
Why not: unreported events leave no trace unless instrumented.
Use instead: a reporting culture, and the ratio of detected to reported events as the diagnostic.
Whether people are safe
Why not: injury data is too rare to move usefully, and its absence is not evidence of safety.
Use instead: leading indicators — near-miss reporting, detected events, discomfort reports, the heavy-frequent item list — and a proper risk assessment.
How to decline
Say what the data does support, which is usually adjacent and substantial.
Say what would answer the question and roughly what it would cost.
Give a range where a range is honest.
Write the limitation into the report, not the covering email, because the report is what circulates.
Do not produce a precise number you cannot support, since it will be quoted without the caveat and the whole programme's reporting is discounted when it turns out to be wrong.
What it does support
Where work waits, and for how long.
How far things travel.
Where the constraint is.
Where errors originate.
Whether equipment is available when it is needed.
Where and when near-misses cluster.
What conditions were present at a given moment.
Seven things answered reliably is a programme worth running.
Declining a request properly
The specific skill that keeps the programme trusted.
Name what the data does support, which is usually adjacent and useful.
Name what would answer the question and roughly what it would cost.
Offer a range if a range is honest.
Offer the alternative method — observation, asking, a one-off study.
Do not produce a weak number with a caveat, because the caveat is dropped in the second retelling and the number is quoted in the third.
Watching the work
The method that answers most of what the data cannot, and it costs an afternoon.
Stand and watch a task, for an hour, without intervening.
Note every deviation from the recorded process.
Note every wait, search and workaround.
Ask afterwards why, without consequence.
This reliably finds practices no system records, and it is the check that keeps the analysis connected to reality.
Do it before every major conclusion, and after any finding that surprises the people who work there.
Also in this section