Title: An Analysis of Sources of Error and an Estimation of Reliability in Science Performance Assessment using Generalizability Theory
Abstract: In this study, we analyzed the sources of error underlying in science performance assessment by employing the generalizability theories, and its reliabilities estimated from this analyzed results, are presented and discussed. Science performance assessment data was obtained from two tenth-grade classes located in Seoul city, and two generalizability studies(G study) - p× ( i : r ) design and p× ( t : r ) design - is applied for both 1st and 2nd semesters. Where, facet i, t, and r indicate essay-type item, performance task and rater, respectively. The results, in the p× ( i : r ) G studies, show that the variance components associated with rater( r) are found to be larger than those resulting from item( i). In addition, the estimated variance components showed a variety of differences from class to class. Variance components related to rater( r) were decreased apparently in the 2nd semester compared with the 1st semester. This result appears to be correlated with the increased training effect, mainly due to the enhancement of raters