Title: Estimating the Consistency and Accuracy of Classifications Based on Test Scores
Abstract: This article presents a method for estimating the accuracy and consistency of classifications based on test scores. The scores can be produced by any scoring method, including a weighted composite. The estimates use data from a single form. The reliability of the score is used to estimate effective test length in terms of discrete items. The true‐score distribution is estimated by fitting a 4‐parameter beta model. The conditional distribution of scores on an alternate form, given the true score, is estimated from a binomial distribution based on the estimated effective test length. Agreement between classifications on alternate forms is estimated by assuming conditional independence, given the true score. Evaluation of the method showed estimates to be within 1 percentage point of the actual values in most cases. Estimates of decision accuracy and decision consistency statistics were only slightly affected by changes in specified minimum and maximum possible scores.
Publication Year: 1995
Publication Date: 1995-06-01
Language: en
Type: article
Indexed In: ['crossref']
Access and Citation
Cited By Count: 148
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot