Thinking on the edge: the influence of discussion and statistical data on awarders' perceptions of borderline candidates in an Angoff awarding meeting

Permanent URL for this page: http://hdl.voced.edu.au/10707/94669.

Abstract:

The Angoff method is a widely used procedure for setting pass scores in vocational examinations, in which the awarders estimate the performance of minimally competent candidates (MCCs) on each test item. Within the context of some UK vocational examinations, the procedure consists of two stages: after making the first round of estimates, awarders make final decisions after discussion and after receiving statistical data about candidate performance. This study investigated the relative effects of discussion and performance data on awarders' estimates relating to a UK vocational qualification. The results of the study showed that performance data had more influence on the awarders' estimates than discussion alone. However, neither discussion nor performance data had the power to drastically sway the awarders from their original decisions, nor did they significantly reduce the variability of individual judgments. These data were compared to what the awarders reported in questionnaires and interviews. The comparison revealed that there was often a discrepancy between what the awarders said about the effect of discussion and statistics on their estimates and what one can conclude by looking at quantitative data alone.

Published abstract reprinted by permission of the copyright owner.

[-] Show less

Subjects: Vocational education and training; Performance; Qualifications; Assessment; Research; Outcomes

Keywords: Research project

Geographic subjects: Europe; Great Britain

Published: Wagga Wagga, New South Wales: Charles Sturt University, 2008

Access item:
http://www.avetra.org.au/downloads/Novakovic%20final.pdf
Request Item from NCVER
Publisher or alternative source