AI in recruitment: a controlled evaluation guide

Evaluate recruitment assistance with job-relevant tests, candidate safeguards and accountable human decisions. This practical guide to AI in recruitment brings together a step-by-step approach, illustrative examples, a reusable worksheet and answers to common questions. Start with the section closest to your current challenge, then use the working session to turn the guidance into a clear next action.

Separate administration from candidate assessment

Recruitment uses of AI range from drafting an advertisement to ranking candidates. The consequences differ substantially. Define the exact task and what decision someone may take from its output before evaluating a tool.

A drafting assistant working from approved role requirements is different from a system that recommends who advances. Candidate scoring, screening and other consequential uses require specialist legal, privacy and fairness review. Do not assume a supplier’s marketing statement establishes suitability for your hiring context.

Make role requirements explicit

Use approved, job-related criteria and remove unnecessary inherited requirements. If AI drafts an advertisement or interview questions, check that it does not introduce new criteria, biased language or unsupported benefits.

Tell candidates what to expect through the organisation’s approved communications and applicable transparency process. Provide accessible routes for adjustments and questions. Do not infer suitability from facial expressions, voice, personality guesses or unrelated personal characteristics.

Test the whole workflow

Evaluate outputs against realistic cases, including unusual career paths, missing information and alternative formats. Use approved test information and document who reviews the result. Check that an error can be corrected before it affects a candidate.

For any assessment use, involve qualified reviewers to examine relevance, consistency and potential unequal effects. The EEOC employer resources provide US equality context. Applicable rules vary by location and use; obtain current local advice before deployment.

Worked example: interview-question drafting

Illustrative scenario: a hiring team asks an approved assistant to draft questions from a role’s documented capabilities. The assistant adds a question about working late, even though the role does not require that pattern.

The panel removes the question and checks all others against the role criteria. It pilots the revised set on fictional examples and agrees scoring anchors. Human interviewers collect evidence and make decisions through the established process. The assistant does not select candidates.

Evaluation worksheet

• Use case: Is the tool drafting, organising information or influencing selection?
• Criteria: Which approved role requirements govern the task?
• Data: What candidate information enters the system and under which reviewed arrangement?
• Failure cases: Which omissions or misinterpretations could disadvantage someone?
• Human review: Who checks the output before a consequential action?
• Candidate route: How are questions, corrections and accessibility needs handled?
• Monitoring: What evidence would trigger a pause or re-evaluation?

Require evidence from the supplier for the actual feature and deployment context. A test on another population or a different task does not settle your evaluation.

Keep final accountability with the organisation

Record the reasons for hiring decisions using the approved criteria and evidence. Do not copy an AI explanation into a decision record as if it were an independent observation.

Monitor workflow changes and supplier updates. Reassess when the model, input data, criteria or candidate population changes materially. Maintain a workable manual route. Use the structured recruitment guide as the underlying process and the AI procurement guide for supplier questions.

Put the guide into practice

Set aside a working session with the people who own this process and one or two people who experience it. Use a fictional or appropriately authorised case, so the discussion can be specific without sharing unnecessary personal information. The purpose is to leave with a usable decision or document, not just agreement that the topic matters.

Prepare the case

Choose one recruitment activity and identify whether AI drafts material, organises information or influences a selection decision. Map the candidate information involved and the people affected. Start with approved role criteria. A broad label such as recruitment automation can hide several different uses with very different consequences.

Write the starting assumptions down before discussing solutions. If the group disagrees on what happened, identify the information needed to resolve that difference rather than building a plan on an untested story.

Work through the decision

Prepare test cases that reflect the role and include varied career histories and formats. Define the expected behaviour and reviewer responsibilities before testing. For drafting assistance, check that no new criteria appear. For consequential assessment, identify the specialist evidence and review required before the use could be considered.

Ask each participant to explain the proposed decision in their own words. Differences in interpretation often reveal an unclear criterion, a missing responsibility or an instruction that will be difficult to follow.

Test an exception

Introduce a candidate with missing information or a request for an accessible alternative. Check how the tool and process respond without treating missing data as negative evidence. Test whether an error can be corrected before the candidate is affected. Maintain a practical route that does not depend on the AI feature.

Record what changes in this situation and what remains the same. An exception should lead to a clear next step, with an owner, rather than an informal workaround that nobody can explain later.

Agree the handoff

Give the hiring team approved instructions, candidate communication requirements and a clear decision record. Confirm who reviews supplier changes and who can stop the use. The panel should be able to explain its decisions from relevant evidence, rather than referring to an unexplained model score.

Finish by confirming the owner, the next action and the date when the result will be reviewed. Give the person receiving the work enough context to continue without repeating the whole discussion.

Frequently asked questions

Can AI remove bias from recruitment?

No tool should be assumed to remove bias simply because it automates part of a process. Criteria, training data, inputs, interpretation and human use can all affect outcomes. Evaluate the specific task and deployment with qualified reviewers, and examine potential unequal effects. Keep job relevance, candidate access and a correction route central to the process rather than relying on a supplier’s general claim.

Is AI-generated interview content ready to use?

Treat it as a draft. Check each question against the approved role criteria, remove irrelevant or inappropriate assumptions and review the assessment method. Prepare clear scoring anchors where used and test the questions before the hiring cycle. The panel remains responsible for using a fair, job-relevant process and for complying with applicable local requirements.

What should candidates be told about AI use?

Use the organisation’s reviewed communication process and applicable transparency requirements. Explain the relevant use clearly enough that candidates understand the process and know where to ask questions or request support. The required detail depends on the tool, decision and jurisdiction. Do not rely on a generic privacy statement to settle every question about a consequential recruitment system.

Review the first cycle

Review whether the assistance improved the defined task and whether candidates could access the process fairly. Examine errors, overrides and complaints through appropriate controls. Reassess when criteria, model behaviour or the candidate population changes materially, and keep the manual route workable rather than theoretical.

Keep a brief record of what was tried, what participants found useful and what needs to change. Compare the result with the original problem rather than judging success only by completion. If the process created extra work without improving clarity, quality or support, simplify it and test again. Share the agreed change with the people who will use it, and name the person responsible for keeping the guidance current.

About New Dynamics

New Dynamics connects goals, feedback, recognition and reviews around the way organisations work. This guide is published by the New Dynamics Editorial Team as part of our practical library for HR leaders, managers and People teams.

Use the examples and worksheets to structure your own discussions and adapt them to your organisation. Illustrative scenarios are not customer case studies. Policy and employment guidance needs appropriate local review before adoption.

For questions about this guide, corrections or a conversation about your performance management process, email contact@new-dynamics.com. Explore the complete guide library for related resources.