Quality Assurance Sampling: How Much to Check, and How

A flat rule like 'review five percent of everything' is easy to explain but arbitrary, and it often reviews too little of a low-volume, high-risk category while reviewing far more than necessary of a high-volume, low-risk one. Instead, size the sample to what you're actually trying to catch: if a defect happens roughly one in twenty times and you want a reasonable chance of catching at least one instance per week, the sample needs to be large enough to make that statistically plausible, not just a tidy percentage.

For most small operations teams, this means sampling more heavily on newer processes or newer team members until quality stabilizes, then tapering off as confidence builds, rather than a flat rate applied uniformly forever.

Random sampling tells you the baseline quality level across everything, which is what you need for a general health check. Targeted sampling, reviewing every escalation, every low customer rating, or every interaction flagged by a specific rule, tells you about your worst cases specifically, which is what you need to find and fix root causes. Run both, and don't let one substitute for the other; a program that only reviews escalations will systematically miss quiet, undetected problems in the interactions that never got flagged.

Score against a written rubric with specific, observable criteria, not a general impression of whether an interaction 'felt right.' A rubric with three or four concrete dimensions, accuracy, completeness, tone, and whether the resolution actually addressed the underlying issue, produces scores that different reviewers apply consistently. Without a written rubric, two reviewers scoring the same interaction routinely disagree, which makes trend data across a team meaningless.

Score each interaction on these dimensions:

• Accuracy: whether the information given or action taken in the interaction was actually correct.
• Completeness: whether every part of the customer's request was addressed before the interaction ended.
• Tone: whether the language and manner fit the situation and the customer.
• Resolution: whether the outcome addressed the underlying issue, not just the question as first asked.

Even with a good rubric, reviewers drift apart over time in how strictly they apply it. Have two or three reviewers independently score the same handful of interactions every month and compare results, discussing any meaningful gap until they converge on a shared interpretation. Skipping calibration is how a QA program slowly loses credibility: a team member scored harshly by one reviewer and leniently by another has a legitimate reason to distrust the whole process.

A QA score that lives in a spreadsheet and never reaches the person whose work it describes doesn't improve anything. Route flagged interactions into a workflow tool like Process Street or ClickUp so the reviewer's notes, the specific example, and a required follow-up conversation are all attached to one record, not scattered across a scorecard and a separate coaching conversation nobody documented.

The easiest interactions to review are often the shortest, best-documented, or most recently completed ones, and a sample built from whatever's easiest to grab systematically underrepresents the messy, long, or older cases where problems are more likely to hide. Pull your sample from a genuinely randomized list, not the top of whatever queue happens to be open, and periodically check whether your sample's average length and complexity actually still match the full population it's supposed to represent.

A quick gut check: if every reviewed case this month finished in under ten minutes, ask where the longer, messier cases went instead.