Candidate Assessment
How to Evaluate Candidates More Consistently
To evaluate candidates consistently, decide what good looks like before you meet anyone, score every candidate against the same anchored criteria, record evidence rather than impressions, and have each interviewer score independently before the debrief. This playbook walks through each step, with examples you can adapt for your next role.
By VritiPublished 4 min read
Why evaluation drifts
Two interviewers can meet the same candidate and walk out with opposite views. That isn't usually a sign that one of them is bad at interviewing. More often, the process gives them nothing shared to evaluate against. The usual causes are:
- No agreed definition of "good". Each interviewer brings their own picture of the ideal hire.
- Different questions. Candidates are compared on evidence that was never gathered from all of them.
- Contrast effects. A solid candidate looks weak after an exceptional one, and strong after a poor one.
- Fatigue and timing. The fifth interview of the day rarely gets the attention the first one did.
- Group dynamics. In an open debrief, the first or most senior opinion tends to become the room's opinion.
Each of these has a process fix.
1. Build the scorecard before you open the role
Start with outcomes, not traits. Ask the hiring manager what this person must have achieved after six and twelve months. Work backwards from those outcomes to the skills and behaviours that produce them, then keep the four to six that matter most.
Weight them. If stakeholder management will make or break the role, it should count for more than a nice-to-have technical skill. Agree the weights with the hiring manager before the first interview. Changing them afterwards to fit a favourite candidate defeats the purpose.
2. Anchor every rating
"Rate problem solving from 1 to 4" invites four different interpretations. An anchored scale describes each level in terms of what the candidate says or does:
| Score | Evidence you would expect |
|---|---|
| 1 | Jumps to a solution without understanding the problem; can't explain trade-offs |
| 2 | Reaches a workable solution with prompting; considers one approach |
| 3 | Clarifies the problem, compares approaches, and justifies the choice |
| 4 | Does all of the above, spots risks unprompted, and explains how they would validate the result |
Anchors also make feedback to candidates and hiring managers easier, because the reason for a score is written down.
3. Ask the same core questions
Every candidate for a role should be asked about every criterion on the scorecard, using the same core questions. Interviewers can and should follow up, but the starting point is shared. If you haven't moved to this format yet, our comparison of structured and traditional interviews explains why it matters and how to switch.
4. Write down evidence, not impressions
"Great communicator" is a conclusion. "Explained the caching trade-off to a non-technical stakeholder using a concrete example" is evidence. Ask interviewers to note what was said and done, mapped to the criterion it supports. When scores are challenged later, evidence holds up and impressions don't.
5. Score independently, before the debrief
Each interviewer submits their scores and evidence before seeing anyone else's. This single habit removes most of the anchoring and groupthink in hiring decisions, and it costs nothing.
6. Run a structured debrief
A good debrief is short and follows a fixed order:
- Go criterion by criterion, not interviewer by interviewer.
- For each criterion, share evidence first, then scores.
- Where scores differ by more than a point, discuss the evidence behind each score rather than averaging them.
- Decide against the bar you set for the role, not by ranking the candidates in the room.
- Record the decision and the main reasons.
7. Calibrate regularly
Consistency decays without maintenance. A few habits keep a team calibrated:
- Have new interviewers shadow experienced ones, then lead while being shadowed.
- Every quarter, look at each interviewer's score distribution. Someone who gives almost everyone a 4, or almost no one, needs a conversation.
- Review a few past hires: did the criteria you scored predict how they performed?
- Update anchors when the role changes, not halfway through a hiring round.
Use your tools to enforce the process
Process holds when the tools make the right thing the easy thing. Keep scorecards and interview feedback attached to the candidate's record in your applicant tracking system, so every reviewer sees the same evidence. For first-round screening, an AI interview scores every candidate against the same weighted criteria and shows the reasoning and transcript behind each score, which gives your team a consistent baseline before human rounds begin. For the bigger picture, read our guide to AI interviewing.
