Search results

The research literature reveals that there are many factors that influence the consistency of assessors' or examiners' judgements. One issue that has not been considered is whether National Vocational Qualifications [NVQs] assessors' consistency of judgement is affected by different types of evidence. In this article, 15 Customer Service and 12 Assessor and Verifier Assessors made assessment judgements about fictitious candidates' evidence. For the Customer Service and Assessor and Verifier units the assessors made judgements about different types of evidence (assessor observations, witness testimonies, personal statements, written underpinning knowledge questions and answers, and record of professional discussions). The assessors’ judgements were statistically analysed. The internal consistency reliability was high, indicating that assessors tend to respond to each assessment criterion within a set in a similar way or that each set of criteria are measuring a common trait. The results are also used to discuss the consistency of assessors' judgements and whether they are affected by different types of evidence. When absolute differences were used to measure inter-rater reliability there was a statistically significant difference for the Assessor and Verifier Award - the assessors' judgements were in less agreement when using the witness testimony than when they assessed other types of evidence. Therefore, the effect of different types of evidence on the agreement between assessors' judgements about competence cannot be dismissed as an issue. Further research is required before we can say precisely which types of evidence affect assessor judgements and in which circumstances.