The same errors recur across facilities. Knowing them is worth more than any technique.
Averaging dissimilar things
Units per hour across a mixed product range.
Cycle time across order types with different work content.
Shifts handling different task mixes.
The check: segment before averaging, and if segmentation changes the conclusion, the average was hiding it.
Comparing without normalising
Two shifts, two sites, two periods, with different volume, mix and staffing.
The check: list what differs before comparing, and normalise or abandon the comparison.
Reacting to ordinary variation
Two numbers compared without the series.
The check: plot the whole series with its normal range, and ask whether the movement exceeds it.
Attributing system performance to people
A slow picker in a badly slotted aisle.
The check: does the difference persist when the same person works elsewhere, and does it persist when someone else works there.
Measuring at detection rather than origin
Error rates attributed to the checking stage.
The check: trace the event history for a sample of errors and find where they were created.
Confusing correlation with cause
Errors rise on Thursdays, and Thursday is also the day the mix changes and the agency staff work.
The check: list the alternatives before concluding, and test the ones you can.
Survivorship in the data
Analysing completed tasks only, which excludes the ones that failed, which are where the problems are.
The check: include cancelled, failed and abandoned records explicitly.
The clock problem
Cross-system timings computed on unsynchronised clocks.
The check: verify clock agreement directly before any cross-system analysis, and re-verify periodically.
Regression to the mean
Intervening after a bad period, then observing an improvement that would have happened anyway.
The check: use a control area or the historical series to establish what would have been expected without the change.
The novelty effect
Improvement immediately after deployment, which decays.
The check: re-measure after a settling period and report that figure as the result.
Optimising a non-constraint
Speeding up a stage that was never the limit.
The check: identify the constraint before choosing what to improve, and expect no throughput gain from anything else.
Confirming a decision already made
The most common trap and the hardest to see from inside.
The check: write down what result would change the decision, before running the analysis. If nothing would, the analysis is decoration.
The general defence
Show the working, so someone else can check it.
Show the spread, always.
Sanity-check against physical reality, and against the people who do the work.
State what would falsify the conclusion.
Report the null results, which is what makes the positive ones believable.
Writing down what would falsify it
The single discipline that prevents the most common trap.
Before running the analysis, write what result would change the decision.
If no result would, the analysis is decoration and should not be run.
Share it with whoever requested the work, which frequently reveals that the decision is already made.
Check it afterwards, honestly.
This is uncomfortable and it is the difference between analysis and justification, which is the trap hardest to see from inside.
Showing the working
What makes a finding survive challenge, and what makes it correctable when wrong.
The query or the method, reproducible.
The filters and exclusions.
The period and the conditions.
The spread, not just the central figure.
The known weaknesses, stated by you rather than discovered by a critic.
A finding that cannot be reproduced by someone else is an opinion, and it will be treated as one the first time it is inconvenient.