How to Build a Structured Interview Scorecard

Work through these in order, and keep the whole thing to one page:

1. Write the three to five outcomes the person must deliver in their first year. These come from the role brief, not from a generic list.
2. Turn each outcome into a competency, a skill or trait that predicts it. Aim for four to six in total.
3. Write one behavioral question per competency, asking for a specific past situation.
4. Define a five-point rating scale with anchors for 1, 3 and 5, so "strong" means the same to every interviewer.
5. Assign each competency to one or two interviewers so no topic is asked five times and others never.

The scorecard is a decision tool, not a form. If interviewers treat it as paperwork to complete afterward, it's failing, so hand it out before the interview and ask for scores within an hour of it ending.

Take an operations manager hire. One outcome is "reduce order-to-delivery delays". That becomes the competency of process diagnosis, with the question: "Tell me about a recurring delay you traced to its cause. How did you find it, and what changed afterward?"

The anchors might read like this:

• 1: Describes symptoms only, blames individuals, and can't say what changed.
• 3: Found a cause with data or observation and made one fix, but didn't check whether it held.
• 5: Mapped the steps, found the constraint, changed the process, measured the result and kept the fix in place.

Notice that anchors describe behavior, not adjectives. "Impressive" isn't an anchor. "Measured the result after the change" is, because two interviewers can agree on whether it happened.

The order of the debrief matters more than the length. Follow these rules:

• Score first, talk second. Each interviewer submits ratings and notes before they see anyone else's. This stops the first or most senior voice from anchoring the room.
• Discuss the gaps. Spend time only on competencies where ratings differ by two points or more, and ask each person what evidence they heard.
• Decide against the requirements. The role brief lists what you'd reject a candidate for lacking. Check those first, then weigh the rest.
• Record the reason. Write one or two sentences on why you're hiring or passing, tied to the competencies. That record protects you if a decision is challenged.

Avoid averaging blindly. A candidate who scores 5 on four competencies and 1 on a core requirement shouldn't win because the mean looks good.

Watch for these when you review your own version:

• Too many competencies. Past six, interviewers can't cover them and start guessing.
• Personality traits with no evidence test. "Culture fit" is vague and invites bias. Replace it with observable behaviors tied to how your team works.
• Hypothetical questions. "What would you do if..." rewards good talkers. Ask what they did.
• No follow-ups planned. Give interviewers two probing prompts, such as "What was your specific role?" and "What would you do differently?"
• Changing the scorecard midway. If you change it after seeing candidates, you can't compare them fairly. Revise between searches instead.

Because rules on interview practices and protected characteristics differ by location, run your question bank past an employment attorney.

Store one master scorecard per role family where interviewers already work. A shared workspace like Notion suits a question bank, anchors and interviewer guidance. Your task tool can hold each candidate's scorecards as attachments or fields, for example in Asana with a task per candidate and a required scoring subtask for every interviewer.

The same structure works beyond hiring. A vendor scorecard uses weighted criteria in the same way, and a leadership team scorecard applies the idea to ongoing performance. If you're setting up the wider process around interviews, start with the hiring process outline, and see the project tool comparison.