Validating assessment tools: a walkthrough for workplace trainers

Validation is the meeting workplace trainers try to postpone until the folder looks tidy. That is the wrong order. You validate so the folder can be tidy. You also validate so a competent boilermaker in Ballarat and a competent boilermaker in Karratha are being judged against the same unit, not against whichever checklist a trainer happened to like.

In nationally recognised training, an assessment tool is the set of instruments and instructions you use to gather and judge evidence. A single quiz is not a tool. A mapped set of tasks, candidate instructions, assessor instructions, recording forms and decision rules can be. Validation asks whether that set still measures the unit of competency on training.gov.au, and whether the judgements coming out of it are consistent.

This walkthrough is for people who assess on the floor, not only for compliance officers. It sits with planning competency-based assessment in the training and assessment hub. It is a method. It is not an offer to mark your tools for a fee.

Two different jobs people collapse into one word

Before use and after use

Pre-use validation — sometimes called quality review of the tool — happens before a candidate is assessed with a new or revised instrument. You are looking at design. Does the task address the performance evidence? Do the questions sample the knowledge evidence? Do the assessment conditions match what the unit names? Could two assessors use the same form and mean the same thing by “satisfactory”?

Post-use validation looks at completed assessments. You are looking at judgement. Did assessors apply the tool as written? Are comments specific? Did everyone pass with identical phrases? Did a cluster bury a unit that nobody actually evidenced? Are reasonable adjustments recorded, or is LLN being silently forgiven?

You need both. A beautiful unused tool can still be invalid. A messy tool that happens to work in the hands of one expert assessor can still be unreliable the week that person is on leave.

Industry engagement sits beside both. If the task no longer looks like the job — because the software changed, the isolation procedure changed, or the workplace never does the thing the 2018 tool assumed — the tool is already out of date. Currency is a rule of evidence for candidates. It is also a property of the instrument.

Who should be in the room

At least one person who did not write the tool. At least one person who is vocationally current in the work. Someone who understands units of competency well enough to notice a mapping fantasy. If the unit has serious risk attached — high-risk work, medication, isolation, child-related work — do not staff the meeting only with people who will be polite.

A candidate representative is not required. A loud manager who wants “a simpler course” is not a validator. They can be consulted about workplace constraints. They do not get a vote on whether a performance criterion exists.

Certificate IV in Training and Assessment is supposed to prepare you to participate in this conversation. What the Cert IV actually prepares you for includes validation as a core duty, not an elective personality trait. If your own TAE program never made you tear a tool apart, you are learning it now. That is normal. Pretending you already know is how bad checklists live for a decade.

Where ASQA regulates the RTO, the Standards expect a planned, systematic validation of assessment practices and judgements, covering each training product over a cycle, with people who are independent enough to be useful. The Standards do not require a particular brand of meeting cake. They do require a record of what was reviewed and what changed.

A walkthrough you can run in ninety minutes

Read the unit before you sit down

Send the current unit release, the mapping document, the candidate instructions, the assessor instructions, and two or three completed samples if you have them. Ask people to read before they sit down. A validation meeting that starts with everyone seeing the unit for the first time is a reading group.

Open the unit. Read the assessment conditions aloud. It feels theatrical. It stops the “we’ve always done it in the classroom” argument ten minutes later. Then take the performance evidence item by item. For each item, find the task that gathers it. If the answer is “it’s sort of in the observation”, write that down as a gap, not as a feeling.

Do the same with knowledge evidence. A long written paper that never touches the knowledge listed, but asks twelve questions about the trainer’s favourite war story, is invalid. A professional conversation with a brief and a benchmark can be valid. The method is not the issue. The match is the issue.

Check foundation skills and language, literacy and numeracy. Does the tool demand a 2,000-word essay the unit never asked for? Does it assume reading skills far above the job and then punish people for them? Or does it skip documentation the unit clearly requires? Both errors show up in workplace tools written by busy people.

Then apply the principles of assessment and the rules of evidence as questions, not as a poster on the wall.

Write decisions as actions. “Review mapping for element 3” is an action. “Tools are fine” is a wish. Name the owner and a date. If you change a question, change the mapping. If you change the observation, brief every assessor who still has the old PDF in their bag.

After real candidates have been through it

Sample completed files, not only distinctions and not only fails. Look for identical assessor comments. Look for every candidate competent on the first attempt with no notes. Look for not-yet-competent results that never say which criterion failed. Look for RPL files that are a resume and a smile — the problem whether RPL is the right path exists to prevent.

Compare two assessors on the same unit. If one always passes and one always finds gaps, you do not automatically know who is right. You know the tool is not reliable yet, or the moderation never happened. Sit those two people together with a de-identified file and make them talk to the criteria.

Watch clustering. Workplace delivery loves a cluster because the job is already a cluster. Validation has to unpick it. Each unit still needs its performance evidence. If the observation day never touched unit B, unit B was not assessed. Mixed-experience groups make this worse, because the experienced person flies through the familiar parts and the gap hides in the unit nobody likes teaching.

What to do with an inherited tool

Most workplace trainers do not write on a blank page. They inherit a zip folder, a SharePoint maze, or a well-meaning booklet from a consultant who has not seen the site. Treat inheritance as a reason to validate sooner, not later.

Confirm the unit release. Superseded units are a credit-transfer and transition problem; they are also a tool problem. Confirm the logo and the context. A tool written for a college simulation may fail assessment conditions on a live plant. Confirm the decision rules. If the marking guide says “must get 80 per cent” on a knowledge paper, ask whether that percentage maps to the knowledge evidence or was copied from a school exam.

You can keep a good inherited structure. You cannot keep a mapping that points at a deleted element. Training packages move. Your folder has to move with them.

Records that are useful on a bad day

Keep the unit release, who was in the room and why they belonged there, the sample, the gaps, and the version that came out. A record in one inbox is a rumour. You do not need fifty pages. You need enough that a new assessor can see someone compared the tool with the unit and changed what was wrong.

Validation will not make a weak trainer strong. It will stop a weak tool from laundering weak judgements. Validate the unit you assess most often, with someone who will call the checklist lazy, then use that version on the floor. That is how assessment starts to mean the same thing in two rooms.