Are Your Skills Assessments Actually Predicting Job Performance?

A candidate completes an online assessment, scores well, and moves to the next stage. But what does that score mean if the candidate used ChatGPT to help write the answers?

Employers have good reason to use skills tests. CVs tell them where someone has worked and what they claim to know. Interviews reveal how candidates explain their experience. A practical assessment can show what they can do.

The difficulty is that an assessment completed with outside help may measure something different from the skill the employer intended to test. As generative AI becomes part of everyday work, hiring teams need to decide when its use is acceptable, when it undermines the result, and whether their tests still serve their original purpose.

A study published in the International Journal of Selection and Assessment in March 2026 examined self-reported generative AI use during pre-employment assessments.

The researchers analysed two groups of applicants: 5,675 people who completed assessments in the third quarter of 2024 and 3,356 who took them in the third quarter of 2025. Reported use of generative AI increased between the two periods.

The study also tested whether warning messages could discourage candidates from using AI. The warnings reduced reported use, suggesting that employers do not necessarily need sophisticated monitoring systems to address every case. Clear instructions about permitted assistance and the consequences of breaking the rules can make a difference.

There is an important qualification. The researchers found that AI use alone was not systematically associated with applicants’ potential job-fit scores. The findings therefore do not support the claim that everyone who uses AI performs poorly, or that every AI-assisted assessment produces an invalid result.

They do show why employers need to be clear about what a test is supposed to measure.

The right approach depends on the job.

If an employee will regularly use AI tools to research information, draft content or analyse data, banning those tools from every assessment may give the employer a misleading picture of the work. The candidate’s ability to use AI responsibly could be part of the job itself.

A different standard is needed when the purpose is to establish whether someone can perform a task independently. An assessment designed to measure unaided reasoning, for example, cannot provide the same evidence if candidates can consult an AI assistant throughout the exercise.

Employers should settle this question before inviting applications. Candidates need to know whether AI tools are allowed, which tools they may use, and whether they must explain how they used them.

Leaving those rules unstated creates a problem for both sides. Applicants may make different assumptions, while recruiters end up comparing results produced under different conditions.

The assessment itself deserves just as much attention as the rules around it.

A marketing candidate might be asked to identify weaknesses in a campaign report and recommend changes. A customer support applicant could respond to a complaint that requires judgment rather than a standard answer. A data analyst might be given a small dataset and asked to explain what the figures show, including the limits of their conclusions.

These tasks give employers something specific to evaluate. They can examine the reasoning, the accuracy of the work and whether the candidate recognises information that is missing.

A generic online quiz may be easier to administer, but a high score does not automatically mean that someone can handle the responsibilities of the position. The employer needs evidence that the assessment measures a skill the employee will actually use.

That principle also applies to AI-assisted work. If the job involves using AI, ask candidates to demonstrate how they check its output, spot errors, and decide when the tool is unsuitable. Producing a fluent answer is not the same as knowing whether it is correct.

A completed assignment does not always reveal how the candidate reached the result.

One way to get a clearer picture is to ask a few follow-up questions about the work. Why did the candidate choose a particular approach? What information would they need before making a final decision? Which part of the answer would they check first?

For an AI-assisted task, candidates can be asked to identify what the tool contributed and what they changed or verified themselves. Their explanation gives the reviewer another source of evidence alongside the submitted work.

This does not require a lengthy oral examination. A short discussion can expose gaps that are difficult to spot in a polished document.

The aim is not to catch candidates out. It is to understand the work they can do and the judgment they bring to it.

Some employers use AI systems to evaluate candidates directly, rather than allowing applicants to use AI during a test. These are separate issues, but both raise questions about the reliability of the result.

The Society for Industrial and Organizational Psychology (SIOP) published recommendations on AI-based employment assessments that address predictive accuracy, consistency, fairness, appropriate use and documentation. Its guidance says that AI assessments should meet rigorous standards, even when the methods used to evaluate them differ from those applied to traditional tests.

An employer buying an assessment tool should therefore ask the provider what the scores measure, how the system has been evaluated, and whether the results are relevant to the jobs for which it will be used. A convincing demonstration is not the same as evidence that an assessment predicts performance.

Companies developing their own tests should keep records of how scores are calculated and review whether the results are useful after people are hired.

An assessment should not remain in the recruitment process simply because the company has always used it.

Employers can compare test results with later performance, using measures suited to the role. For a support position, this might include the quality of case handling. For a writing role, it could include factual accuracy and editorial feedback. For a technical position, it may involve the quality and reliability of completed work.

The comparison will not establish that the test caused a particular outcome. Training, management, and working conditions also affect performance. But it can help identify whether the assessment is providing useful information or filtering out people who could succeed in the job.

If a test consistently rewards a skill that employees rarely use, it needs another look. The same applies if candidates perform well during assessment but struggle with the work the test was meant to represent.

If your hiring process isn't consistently identifying the right candidates, it may be worth looking at the wider strategy, too. Our guide, Your Job Has Been Open for 60 Days. Is It Time to Change Your Hiring Strategy?, explains how to identify where a recruitment process is falling short.

Employers do not need to abandon skills assessments because AI exists. They need to know what evidence they want from candidates and set up the process accordingly.

That means choosing a relevant task, explaining the rules on AI use, asking candidates to account for their decisions where appropriate, and checking whether the scores correspond with job performance.

A test is worth keeping when it helps a hiring team make a better-informed decision. A high score, by itself, is not enough.