Abstract

This study examined the dimensionality of 10 different calibration measures using confirmatory factor analysis (CFA). The 10 measures were representative of five interpretative families of measures used to assess monitoring accuracy based on a 2 (performance) × 2 (monitoring judgment) contingency table. We computed scores for each of the measures using a common data set and compared one-, two-, and five-factor CFA solutions. We predicted that the two-factor solution corresponding to measures of specificity and sensitivity used to assess diagnostic efficiency would provide the best solution. This hypothesis was confirmed, yielding two orthogonal factors that explained close to 100% of sample variance. The remaining eight measures were intercorrelated significantly with the sensitivity and specificity factors, which explained between 91 and 99 percent of variance in each measure. The two-factor solution was consistent with two different explanations, including the possibility that metacognitive monitoring may utilize two different types of processes that rely on separate judgments of correct and incorrect performance, or may be sufficiently complex that a single measurement statistic fails to capture all of the variance in the monitoring process. Our findings indicated that no single measure explains all the variance in monitoring judgments. Implications for future research are discussed.

Citation

Schraw, G., Kuch, F. & Gutierrez, A.P. (2013). Measure for measure: Calibrating ten commonly used calibration scores. Learning and Instruction, 24(1), 48-57. Elsevier Ltd. Retrieved August 8, 2024 from https://www.learntechlib.org/p/199824/.

Keywords

• calibration
• metacognition
• Monitoring
• Two-factor model