Search results

The Angoff method is a widely used procedure for setting pass scores in vocational examinations, in which the awarders estimate the performance of minimally competent candidates (MCCs) on each test item. Within the context of some UK vocational examinations, the procedure consists of two stages: after making the first round of estimates, awarders make final decisions after discussion and after receiving statistical data about candidate performance. This study investigated the relative effects of discussion and performance data on awarders' estimates relating to a UK vocational qualification. The results of the study showed that performance data had more influence on the awarders' estimates than discussion alone. However, neither discussion nor performance data had the power to drastically sway the awarders from their original decisions, nor did they significantly reduce the variability of individual judgments. These data were compared to what the awarders reported in questionnaires and interviews. The comparison revealed that there was often a discrepancy between what the awarders said about the effect of discussion and statistics on their estimates and what one can conclude by looking at quantitative data alone.