This article described a revised index of relative metamnemonic accuracy (also known as resolution) for research on metacognitive monitoring. A number of different indices have served in the measure of metamnemonic accuracy, such as Goodman-Kruskal gamma coefficient (?) and indices of the signal detection theory (SDT). SDT indices (e.g. d’ and Ag) suffer from their own limitations when applied to relative metamnemonic accuracy. In contrast, gamma coefficient has dominated in metamnemonic research since Nelson (1984) introduced it. Even though researchers found some of its crucial shortcomings, no index could evaluate the resolution of metamnemonic judgments better than?; however, a revised SDT index da may serve as a desirable replacement of?. Da has a larger scale of [0, ∞) than [-1, 1] of? , and it can overcome the shortcomings of ? in the respect of interval-level analyses and conclusions. A couple of studies focused on the use of this new index to evaluate metamnemonic resolution (Benjamin & Diaz, 2008; Masson & Rotello, 2009). We explained the advantages of da in respect of its application in research, instead of providing mathematical and statistical accounts, and continued with how to compute da, using the example from Benjamin and Diaz (2008). Based on those empirical comparisons of da and?, we made a recommendation that da would be the most desirable index of relative metamnemonic accuracy in a 2×N (N≥4) metacognitive task. Da has been adopted in several studies on recognition as an index of discrimination (Benjamin, Diaz, & Wee, 2009; Macmillan & Creelman, 2004; Matzen & Benjamin, 2009; Tullis & Benjamin, 2011), and recently, Benjamin and his colleague investigated metamnemonic accuracy for faces with da for the first time (Hourihan, Benjamin, & Liu, 2012). Therefore, we concluded that da has a hopeful prospect in its application in research on metacognitive monitoring, and we suggested that researchers should develop a standardized procedure as well as an advanced program for data processing to make da widely accepted by metacognitive researchers.
Key words
Mnemonic monitoring /
Signal Detection Theory (SDT) /
Relative accuracy /
Discrimination
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
References
[1]Benjamin, A. S. & Diaz, M. (2008). Measurement of relative metamnemonic accuracy. In J. Dunlosky & R. A. Bjork (Eds.), Handbook of Memory and Metamemory (pp. 73-94). New York, New York: Psychology Press
[2]Benjamin, A. S., Diaz, M. L., & Wee, S..Signal detection with criterion noise: Applications to recognition memory[J].Psychological Review,2009,116(1):84-115
[3]Butterfield.[J].B. & Metcalfe, J. (2006). The correction of errors committed with high confidence. Metacognition and Learning,1556,1:-
[4]Connor, L. T., Dunlosky, J., & Hertzog, C. (1997). Age-related differences in absolute but not relative metamemory accuracy. Psychology and Aging, 12, 50-71
[5]Dunlosky, J., & Lipko, A. (2007). Metacomprehension: A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16, 228-232
[6]Finn, B., & Metcalfe, J. (2008). Judgments of learning are influenced by memory for past test. Journal of Memory and Language, 58, 19-34
[7]Hourihan, K., Benjamin, A. S., & Liu, X. (2011). An own-race bias in metamnemonic accuracy for faces. Memory and Cognition. Paper under review
[8]Kimball, D. R, & Metcalfe, J. (2003). Delaying judgments of learning affects memory, not metamemory. Memory and Cognition, 31, 918-929
[9]Konopka, A. E., & Benjamin, A. S. (2009). Schematic knowledge changes what judgments of learning predict in a source memory task. Memory and Cognition, 37, 42-51
[10]Koriat, A. (1997). Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General, 126, 349-370
[11]Macmillan, N. A., & Creelman, C. D. (2004). Detection theory: A user’s guide. (Second Edition). USA: Psychology Pres
[12]Masson, M. E. J., & Rotello, C. M..Sources of bias in the Goodman–Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes[J].Journal of Experimental Psychology: Learning, Memory, and Cognition,2009,35:509-527
[13]Matzen, L. E. & Benjamin, A. S..Remembering words not presented in sentences: How study context changes patterns of false memories[J].Memory and Cognition,2009,37(1):52-64
[14]Mazzoni.[J].G., & Nelson, T. O. (1995). Judgments of learning are affected by the kind of encoding in ways that cannot be attributed to the level of recall. Journal of Experimental Psychology: Learning, Memory, and Cognition,1263,21:-
[15]Metcalfe, J., & Greene, M.J. (2007). Metacognition of agency. Journal of Experimental Psychology: General, 136, 184-199
[16]Nelson, T. O. (1984). A comparison of current measures of feeling-of-knowing accuracy. Psychological Bulletin, 95, 109-133
[17]Nelson, T. O..ROC curves and measures of discrimination accuracy: A reply to Swets[J].Psychological Bulletin,1986,100:128-132
[18]Nelson, T. O. (1996). Gamma is a measure of the accuracy of predicting performance on one item relative to another item, not of the absolute performance on an individual item: Comments on Schraw (1995). Applied Cognitive Psychology, 10, 257-260
[19]Nelson, T. O., & Dunlosky, J. (1991). The delayed-JoL effect: When delaying your judgments of learning can improve the accuracy of your metacognitive monitoring. Psychological Science, 2, 267-270
[20]Nelson, T. O., Narens, L., & Dunlosky, J. (2004). A revised methodology for research on metamemory: Pre-judgment recall and monitoring (PRAM). Psychological Methods, 9, 53-69
[21]Nietfeld, J. L., Enders, C. K., & Schraw, G..A Monte Carlo comparison of measures of relative and absolute monitoring accuracy[J].Educational and Psychological Measurement,2006,66(2):258-271
[22]Pieschl, S. (2009). Metacognitive calibration— an extended conceptualization and potential applications. Metacognition and Learning, 4, 3-3
[23]Schraw, G..A conceptual analysis of five measures of metacognitive monitoring[J].Metacognition and Learning,2009,4:33-45
[24]Swets, J. A. .Indices of discrimination or diagnostic accuracy: Their ROCs and implied models. [J].Psychological Bulletin,1986,99:100-117
[25]Tullis, J.G. & Benjamin, A. S..On the effectiveness of self-paced learning. [J].Journal of Memory and Language,2011,64:109-118
[26]Weaver, C. A., III, & Kelemen, W. L..Processing similarity does not improve metamemory: Evidence against transfer-appropriate monitoring[J].Journal of Experimental Psychology: Learning, Memory and Cognition,2003,29:1058-1065
PDF(256 KB)
