Introduction to assessment validation

Validation is defined as the quality review of the assessment process. It involves checking that the assessment tool produces valid, reliable, sufficient, current and authentic evidence to enable reasonable judgements to be made as to whether the requirements of the training package or VET accredited courses are met. It includes reviewing a statistically valid sample of the assessments and making recommendations for future improvements to the assessment tool, process and/or outcomes and acting upon such recommendations. [1]

Assessment validation has two distinct parts:

• Part 1. Check the assessment tool for compliance
• Part 2. Review a sample of the assessments.

This article covers the first part only.

If you want to know more about the second part, then I recommend reading the information published by ASQA about how to conduct assessment validation. This information covers: [2]

• Who conducts validation?
• Scheduling validation
• Statistically valid sampling and randomly selecting samples to be validated
• Effective validation
  • Reviewing assessment practice
  • Reviewing assessment judgements

  Part 1. Check the assessment tool for compliance

  The assessment tool must be checked to ensure it complies with the requirements specified by the Standards for RTOs, in particular: [3]

  • Compliance with the principles of assessment and the rules of evidence
  • Compliance with the requirements specified by the training package or VET accredited course.

  The following 6-step process can be used to check the assessment tool for compliance:

  • Step 1. Read the assessment requirements
  • Step 2. Review the assessment plan
  • Step 3. Review the assessment matrix (mapping)
  • Step 4. Check the details about how the knowledge evidence is planned to be being gathered
  • Step 5. Check the details about how the performance evidence is planned to be being gathered
  • Step 6. Check the overall quality of the assessment tool.

  Step 1. Read the assessment requirements

  This is a quick step to perform. You will read and re-read the unit of competency and its assessment requirements many times during the assessment validation process. During this first step, have a quick read of the assessment requirements and answer the following questions:

  1. What is the volume or frequency of performance evidence?
  2. Is the location, facilities, equipment, or other assessment conditions specified?

  Step 2. Review the assessment plan

  This step should also be quick. The purpose of this step is to get an overview of what is the planned assessment approach During this second step, answer the following questions:

  1. Has the correct unit code and title been used?
  2. How many assessment tasks are planned?
  3. Is there a plan to gather the knowledge evidence?
  4. Does there appear to be sufficient assessment tasks for gathering the volume or frequency of performance evidence?
  5. Does the planned assessment approach seem to be simple or complex?

  Note: This planned assessment approach may be found in the Training and Assessment Strategy (TAS) or other documents covering how the RTO plans to implement the delivery of the training and assessment for a unit or cluster of units.

  Step 3. Review the assessment matrix (mapping)

  This step should be a relatively quick step. The assessment matrix is an important document used to display how the RTO plans to gather evidence that comply with the requirements specified by the training package or VET accredited course. The assessment matrix will be used during Step 4 and Step 5 to cross-check the RTO’s planned assessment approach and the assessment instruments being used to gather evidence.

  During this third step, answer the following questions:

  1. Has the correct unit code and title been used?
  2. Has the entire unit of competency and its assessment requirements been copied into the matrix? Are the number of items the same? For example, if the unit has five elements does the matrix have five elements? And scan the wording to ensure the matrix has the exact words as the unit of competency and its assessment requirements.
  3. Is there one column for each planned assessment task?
  4. Are the titles or descriptions of the assessment tasks the same in the assessment plan and assessment matrix?
  5. Is every item from the unit of competency and its assessment requirements planned to be assessed? For example, is there at least one ‘tick’ in every row?

  Note: Some assessment matrices will provide information or numerical indicator about the assessment item instead of using a ‘tick’. For example, the matrix may indicate that a piece of knowledge evidence will be gather by Question 1.

  Step 4. Check the details about how the knowledge evidence is planned to be being gathered

  This step requires an attention to details. The purpose is to ensure that the assessment tool will gather the required knowledge evidence. During this fourth step, answer the following questions:

  1. Is there an assessment instrument for gathering the knowledge evidence?
  2. Are the instructions to the assessor clear and concise?
  3. Are the instructions to the candidate clear and concise?
  4. Is the structure, format, and layout of the assessment instrument easy to follow? This includes headings, sub-headings, page numbers, and numbering of questions.
  5. Is there consistency between the assessment plan, assessment matrix and assessment instrument? For example, if the assessment plan states that there are 17 questions, does the assessment instrument have 17 questions?
  6. Is every item of knowledge evidence being adequately gathered? A judgement about ‘adequately’ will need to be made.

  Step 5. Check the details about how the performance evidence is planned to be being gathered

  This step requires an attention to details and it can take time to examine the assessment documents for compliance. The purpose is to ensure that the assessment tool will gather the required performance evidence. During this fifth step, answer the following questions:

  1. Is there one or more assessment instruments for gathering the performance evidence?
  2. Are the assessment conditions compliant with those stated in the Assessment Requirements for the unit of competency? This may include assessment location, facilities, equipment, and access to specified documents. For example, if the assessment conditions state that the assessment occurs in the workplace, then the assessment tasks must state that the evidence must be gathered from a workplace (not from a simulated workplace).
  3. Are the instructions to the assessor clear and concise?
  4. Are the instructions to the candidate clear and concise?
  5. Are the items of performance evidence clearly listed or identified?
  6. Is the structure, format, and layout of the assessment instrument or instruments easy to follow? This includes headings, sub-headings, and page numbers.
  7. Is there consistency between the assessment plan, assessment matrix and assessment instrument? For example, if the assessment matrix states that evidence for Performance Criteria 1.1 will be gather during Assessment Task 2, then Assessment Task 2 must cover the gathering of evidence for Performance Criteria 1.1.
  8. Is every item of performance evidence being adequately gathered? A judgement about ‘adequately’ will need to be made. This includes a check that the amount of evidence being gathered is compliant with the specified volume or frequency of performance evidence.

  Note: Verbs are important. For example, if performance criteria says, ‘negotiate and agree with a supervisor’, then there needs to be evidence that the candidate has negotiated and agreed with a supervisor’. Also, the letter ‘s’ is important. A item of performance evidence may specify plural rather than singular. For example, if it states ‘write reports’, then more than one written report is required for evidence.

  Step 6. Check the overall quality of the assessment tool

  This step can take time to examine the assessment tool for compliance, readability, and usability.

  1. Are there sample answers and assessment decision criteria for assessors?
  2. Is the structure, format, and layout of all assessment documents easy to follow?
  3. Are all instructions written clearly and concisely?
  4. Are there any grammar, spelling and typo errors?
  5. Is there a list of all the assessment documents required for the assessor?
  6. Does the assessment tool have all the documents required for the assessor?
  7. Is there a list of all the assessment documents required for the candidate?
  8. Does the assessment tool have all the documents required for the candidate?
  9. Has the correct unit code and title been used throughout all the assessment documents? This may include release number.
  10. Do all the assessment documents have version control information?

  In conclusion

  Assessment validation has two distinct parts:

  • Part 1. Check the assessment tool for compliance
  • Part 2. Review a sample of the assessments

  Assessment validation can be time-consuming and mind-bending.

  Preparation before an assessment validation meeting can reduce the time at the assessment validation meeting. However, you can expect a typical assessment validation meeting to require anywhere between a few hours and an entire day. The duration of the assessment validation meeting can depend on the quality of the assessment tool and number of assessment samples to be reviewed. I regularly see poor quality assessment tools, and it takes time to properly check large numbers of assessment samples.

  Clear and critical thinking is required by people participating in an assessment validation meeting. There are usually many documents to be reviewed and checked. Printing paper copies of documents (or some documents) and using ‘split screens’ on computers will help when comparing information from two or more documents, such as:

  • unit of competency
  • assessment requirements
  • assessment plan
  • assessment matrix
  • assessment instructions
  • assessment instruments.

  Frustration and fatigue can be experienced during long assessment validation meetings. Breaks will be needed (and sometimes chocolate helps). It is a good idea to assign an experienced VET practitioner to lead the assessment validation meeting.

  References

  Do you need help with your TAE studies?

  Are you doing the TAE40116 or TAE40122 Certificate IV in Training and Assessment, and are you struggling with your studies? Do you want help with your TAE40116 or TAE40122 studies?

  

  Ring Alan Maguire on 0493 065 396 to discuss.

  Training trainers since 1986