Psychometric Reliability, Validity, and Generalizability of MoCA in American Indian Adults: The Strong Heart Study.
Authors:
Journal: Assessment
Publication Type: Journal Article
Date: 2025
DOI: NIHMS2073391
ID: 39046194
Abstract
Standardized neuropsychological instruments are used to evaluate cognitive impairment, but few have been psychometrically evaluated in American Indians. We collected Montreal Cognitive Assessment (MoCA) in 403 American Indians 70 to 95 years, as well as age, sex, education, bilingual status, depression symptoms, and other neuropsychological instruments. We evaluated inferences of psychometric validity, including scoring inference using confirmatory factor analysis and structural equation modeling, generalizability inference using reliability coefficient, and extrapolation inference by examining performance across different contexts and substrata. The unidimensional (total score) model had good fit criteria. Internal consistency reliability was high. MoCA scores were positively associated with crystallized cognition (ρ = 0.48, < .001) and inversely with depression symptoms (ρ = -0.27, < .001). Significant differences were found by education ( = 0.79, < .05) depression ( = 0.484, < .05), and adjudicated cognitive status ( = .0001) strata; however, MoCA was not sensitive or specific in discriminating cognitive impairment from normal cognition (area under the curve <0.5). MoCA scores had psychometric validity in older American Indians, but education and depression are important contextual features for score interpretability. Future research should evaluate cultural or community-specific adaptations, to improve test discriminability in this underserved population.
Reference List
- Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, Gamst A, Holtzman DM, Jagust WJ, Petersen RC, Snyder PJ, Carrillo MC, Thies B, & Phelps CH (2011). The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement, 7(3), 270–279. 10.1016/j.jalz.2011.03.008|||Apolinario D, Dos Santos MF, Sassaki E, Pegoraro F, Pedrini AVA, Cestari B, Amaral AH, Mitt M, Muller MB, Suemoto CK, & Aprahamian I (2018). Normative data for the Montreal Cognitive Assessment (MoCA) and the Memory Index Score (MoCA-MIS) in Brazil: Adjusting the nonlinear effects of education with fractional polynomials. Int J Geriatr Psychiatry, 33(7), 893–899. 10.1002/gps.4866|||Avila JF, Renteria MA, Witkiewitz K, Verney SP, Vonk JMJ, & Manly JJ (2020). Measurement invariance of neuropsychological measures of cognitive aging across race/ethnicity by sex/gender groups. Neuropsychology, 34(1), 3–14. 10.1037/neu0000584|||Bernier PJ, Gourdeau C, Carmichael PH, Beauchemin JP, Voyer P, Hudon C, & Laforce R Jr. (2023). It’s all about cognitive trajectory: Accuracy of the cognitive charts-MoCA in normal aging, MCI, and dementia. J Am Geriatr Soc, 71(1), 214–220. 10.1111/jgs.18029|||Block CK, Johnson-Greene D, Pliskin N, & Boake C (2017). Discriminating cognitive screening and cognitive testing from neuropsychological assessment: implications for professional practice. Clin Neuropsychol, 31(3), 487–500. 10.1080/13854046.2016.1267803|||Brown TA (2015). Confirmatory factor analysis for applied research, 2nd ed. The Guilford Press.|||Browne MW, & Cudeck R (1992). Alternative Ways of Assessing Model Fit. Sociological methods & research, 21(2), 230–258. 10.1177/0049124192021002005|||Carson N, Leach L, & Murphy KJ (2018). A re-examination of Montreal Cognitive Assessment (MoCA) cutoff scores. Int J Geriatr Psychiatry, 33(2), 379–388. 10.1002/gps.4756|||Cecato JF, Martinelli JE, Izbicki R, Yassuda MS, & Aprahamian I (2016). A subtest analysis of the Montreal cognitive assessment (MoCA): which subtests can best discriminate between healthy controls, mild cognitive impairment and Alzheimer’s disease? Int Psychogeriatr, 28(5), 825–832. 10.1017/S1041610215001982|||Duro D, Simoes MR, Ponciano E, & Santana I (2010). Validation studies of the Portuguese experimental version of the Montreal Cognitive Assessment (MoCA): confirmatory factor analysis. J Neurol, 257(5), 728–734. 10.1007/s00415-009-5399-5|||Finney SJ, & DiStefano C (2013). Nonnormal and categorical data in structural equation modeling. In Structural equation modeling: A second course, 2nd ed. (pp. 439–492). IAP Information Age Publishing.|||Freitas S, Simoes MR, Alves L, Duro D, & Santana I (2012). Montreal Cognitive Assessment (MoCA): validation study for frontotemporal dementia. J Geriatr Psychiatry Neurol, 25(3), 146–154. 10.1177/0891988712455235|||Freitas S, Simoes MR, Alves L, Vicente M, & Santana I (2012). Montreal Cognitive Assessment (MoCA): validation study for vascular dementia. J Int Neuropsychol Soc, 18(6), 1031–1040. 10.1017/S135561771200077X|||Gagnon G, Hansen KT, Woolmore-Goodwin S, Gutmanis I, Wells J, Borrie M, & Fogarty J (2013). Correcting the MoCA for education: effect on sensitivity. Can J Neurol Sci, 40(5), 678–683. 10.1017/s0317167100014918|||Galvin JE, Chrisphonte S, & Chang LC (2021). Medical and Social Determinants of Brain Health and Dementia in a Multicultural Community Cohort of Older Adults. J Alzheimers Dis, 84(4), 1563–1576. 10.3233/JAD-215020|||Gil L, Ruiz de Sánchez C, Gil F, Romero SJ, & Pretelt Burgos F (2015). Validation of the Montreal Cognitive Assessment (MoCA) in Spanish as a screening tool for mild cognitive impairment and mild dementia in patients over 65 years old in Bogotá, Colombia. International journal of geriatric psychiatry, 30(6), 655–662. 10.1002/gps.4199|||Gorelick PB, Scuteri A, Black SE, Decarli C, Greenberg SM, Iadecola C, Launer LJ, Laurent S, Lopez OL, Nyenhuis D, Petersen RC, Schneider JA, Tzourio C, Arnett DK, Bennett DA, Chui HC, Higashida RT, Lindquist R, Nilsson PM, … Anesthesia. (2011). Vascular contributions to cognitive impairment and dementia: a statement for healthcare professionals from the american heart association/american stroke association. Stroke, 42(9), 2672–2713. 10.1161/STR.0b013e3182299496|||Hagell P, Hedin PJ, Meads DM, Nyberg L, & McKenna SP (2010). Effects of method of translation of patient-reported health outcome questionnaires: a randomized study of the translation of the Rheumatoid Arthritis Quality of Life (RAQoL) Instrument for Sweden. Value Health, 13(4), 424–430. 10.1111/j.1524-4733.2009.00677.x|||Hobson J (2015). The Montreal Cognitive Assessment (MoCA). Occup Med (Lond), 65(9), 764–765. 10.1093/occmed/kqv078|||Hoops S, Nazem S, Siderowf AD, Duda JE, Xie SX, Stern MB, & Weintraub D (2009). Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease. Neurology, 73(21), 1738–1745. 10.1212/WNL.0b013e3181c34b47|||Horton DK, Hynan LS, Lacritz LH, Rossetti HC, Weiner MF, & Cullum CM (2015). An Abbreviated Montreal Cognitive Assessment (MoCA) for Dementia Screening. Clin Neuropsychol, 29(4), 413–425. 10.1080/13854046.2015.1043349|||Janes H, & Pepe MS (2009). Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve. Biometrika, 96(2), 371–382. 10.1093/biomet/asp002|||Julayanont P, Brousseau M, Chertkow H, Phillips N, & Nasreddine ZS (2014). Montreal Cognitive Assessment Memory Index Score (MoCA-MIS) as a predictor of conversion from mild cognitive impairment to Alzheimer’s disease. J Am Geriatr Soc, 62(4), 679–684. 10.1111/jgs.12742|||Kane MT (2013). Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement, 50(1), 1–73. 10.1111/jedm.12000|||Kaplan D, & Sage Publications. (2004). The Sage handbook of quantitative methodology for the social sciences. Sage. Table of contents only http://www.loc.gov/catdir/toc/ecip0412/2003028071.html|||Publisher description. http://www.loc.gov/catdir/enhancements/fy0657/2003028071-d.html|||Contributor biographical information. http://www.loc.gov/catdir/enhancements/fy0734/2003028071-b.html|||Lee ET, Welty TK, Fabsitz R, Cowan LD, Le NA, Oopik AJ, Cucchiara AJ, Savage PJ, & Howard BV (1990). The Strong Heart Study. A study of cardiovascular disease in American Indians: design and methods. Am J Epidemiol, 132(6), 1141–1155. 10.1093/oxfordjournals.aje.a115757|||Lenhard W, & Lenhard A (2016). Computation of Effect Sizes. 10.13140/RG.2.2.17823.92329|||Malek-Ahmadi M, Powell JJ, Belden CM, O’Connor K, Evans L, Coon DW, & Nieri W (2015). Age- and education-adjusted normative data for the Montreal Cognitive Assessment (MoCA) in older adults age 70–99. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn, 22(6), 755–761. 10.1080/13825585.2015.1041449|||Manly JJ, Jones RN, Langa KM, Ryan LH, Levine DA, McCammon R, Heeringa SG, & Weir D (2022). Estimating the Prevalence of Dementia and Mild Cognitive Impairment in the US: The 2016 Health and Retirement Study Harmonized Cognitive Assessment Protocol Project. JAMA Neurol, 79(12), 1242–1249. 10.1001/jamaneurol.2022.3543|||McCann A, McNulty H, Rigby J, Hughes CF, Hoey L, Molloy AM, Cunningham CJ, Casey MC, Tracey F, O’Kane MJ, McCarroll K, Ward M, Moore K, Strain JJ, & Moore A (2018). Effect of Area-Level Socioeconomic Deprivation on Risk of Cognitive Dysfunction in Older Adults. J Am Geriatr Soc, 66(7), 1269–1275. 10.1111/jgs.15258|||McDonald RP (1999). Test theory: A unified treatment. Lawrence Erlbaum Associates Publishers.|||McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR Jr., Kawas CH, Klunk WE, Koroshetz WJ, Manly JJ, Mayeux R, Mohs RC, Morris JC, Rossor MN, Scheltens P, Carrillo MC, Thies B, Weintraub S, & Phelps CH (2011). The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement, 7(3), 263–269. 10.1016/j.jalz.2011.03.005|||McNeish D, & Wolf MG (2021). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychol Methods, 28(1), 61–88. 10.1037/met0000425|||Mehta KM, & Yeo GW (2017). Systematic review of dementia prevalence and incidence in United States race/ethnic populations. Alzheimers Dement, 13(1), 72–83. 10.1016/j.jalz.2016.06.2360|||Milani SA, Marsiske M, Cottler LB, Chen X, & Striley CW (2018). Optimal cutoffs for the Montreal Cognitive Assessment vary by race and ethnicity. Alzheimers Dement (Amst), 10, 773–781. 10.1016/j.dadm.2018.09.003|||Milani SA, Marsiske M, & Striley CW (2019). Discriminative Ability of Montreal Cognitive Assessment Subtests and Items in Racial and Ethnic Minority Groups. Alzheimer Dis Assoc Disord, 33(3), 226–232. 10.1097/WAD.0000000000000310|||Morris JC, Weintraub S, Chui HC, Cummings J, Decarli C, Ferris S, Foster NL, Galasko D, Graff-Radford N, Peskind ER, Beekly D, Ramos EM, & Kukull WA (2006). The Uniform Data Set (UDS): clinical and cognitive variables and descriptive data from Alzheimer Disease Centers. Alzheimer Dis Assoc Disord, 20(4), 210–216. 10.1097/01.wad.0000213865.09806.92|||Muthén B (1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54, 557–585.|||Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, Cummings JL, & Chertkow H (2005). The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc, 53(4), 695–699. 10.1111/j.1532-5415.2005.53221.x|||Nunnally JCBIH (1994). Psychometric theory. McGraw-Hill.|||Pendlebury ST, Cuthbertson FC, Welch SJ, Mehta Z, & Rothwell PM (2010). Underestimation of cognitive impairment by Mini-Mental State Examination versus the Montreal Cognitive Assessment in patients with transient ischemic attack and stroke: a population-based study. Stroke, 41(6), 1290–1293. 10.1161/STROKEAHA.110.579888|||Pett MA, Lackey NR, & Sullivan JJ (2003). Making sense of factor analysis : the use of factor analysis for instrument development in health care research. Sage Pub. Publisher description http://www.loc.gov/catdir/enhancements/fy0657/2002156318-d.html|||Table of contents only. http://www.loc.gov/catdir/enhancements/fy0657/2002156318-t.html|||Roebuck-Spencer TM, Glen T, Puente AE, Denney RL, Ruff RM, Hostetter G, & Bianchini KJ (2017). Cognitive Screening Tests Versus Comprehensive Neuropsychological Test Batteries: A National Academy of Neuropsychology Education Paperdagger. Arch Clin Neuropsychol, 32(4), 491–498. 10.1093/arclin/acx021|||Rossetti HC, Lacritz LH, Cullum CM, & Weiner MF (2011). Normative data for the Montreal Cognitive Assessment (MoCA) in a population-based sample. Neurology, 77(13), 1272–1275. 10.1212/WNL.0b013e318230208a|||Rossetti HC, Lacritz LH, Hynan LS, Cullum CM, Van Wright A, & Weiner MF (2017). Montreal Cognitive Assessment Performance among Community-Dwelling African Americans. Arch Clin Neuropsychol, 32(2), 238–244. 10.1093/arclin/acw095|||Rossetti HC, Smith EE, Hynan LS, Lacritz LH, Cullum CM, Van Wright A, & Weiner MF (2019). Detection of Mild Cognitive Impairment Among Community-Dwelling African Americans Using the Montreal Cognitive Assessment. Arch Clin Neuropsychol, 34(6), 809–813. 10.1093/arclin/acy091|||Sachs BC, Steenland K, Zhao L, Hughes TM, Weintraub S, Dodge HH, Barnes LL, Craft S, Parker ML, & Goldstein FC (2020). Expanded Demographic Norms for Version 3 of the Alzheimer Disease Centers’ Neuropsychological Test Battery in the Uniform Data Set. Alzheimer Dis Assoc Disord, 34(3), 191–197. 10.1097/WAD.0000000000000388|||Suchy-Dicey A, Oziel K, Sawyer C, Olufadi Y, Ali T, Fretts AM, Umans JG, Shibata D, Longstreth WT, Rhoads K, Buchwald D, & Grabowski TJ (2022). Educational and clinical associations with longitudinal changes in cognitive function and brain imaging among American Indians: the Strong Heart Study. Neurology.|||Suchy-Dicey A, Verney SP, Nelson LA, Barbosa-Leiker C, Howard BA, Crane PK, & Buchwald DS (2020). Depression Symptoms and Cognitive Test Performance in Older American Indians: The Strong Heart Study. J Am Geriatr Soc, 68(8), 1739–1747. 10.1111/jgs.16434|||Suchy-Dicey A e. a. (2024). Prevalence of dementia in American Indians: the Strong Heart Study. Alzheimer’s & Dementia|||Suchy-Dicey AM, Oziel K, Sawyer C, Olufadi Y, Ali T, Fretts AM, Umans JG, Shibata D, Longstreth WT, Rhoades K, Buchwald D, & Grabowski TJ (2022). Educational and clinical associations with longitudinal changes in cognitive function and brain imaging among American Indians: the Strong Heart Study. Neurology.|||Suchy-Dicey AM, Shibata DK, Madhyastha TM, Grabowski TJ, Longstreth WT Jr., & Buchwald DS (2017). Findings of Vascular Brain Injury and Structural Loss from Cranial Magnetic Resonance Imaging in Elderly American Indians: The Strong Heart Study. Neuroepidemiology, 48(1–2), 39–47. 10.1159/000459624|||Teng EL, & Chang Chui H (1987). The Modified Mini-Mental (3MS) Examination. Journal Clinical Psychiatry, 48(8), 314–318.|||Teresi JA, Ramirez M, Lai JS, & Silver S (2008). Occurrences and sources of Differential Item Functioning (DIF) in patient-reported outcome measures: Description of DIF methods, and review of measures of depression, quality of life and general health. Psychol Sci Q, 50(4), 538. https://www.ncbi.nlm.nih.gov/pubmed/20165561
|||Veizel K, & Zibulsky J (2013). Wide Range Achievement Test–Fourth Edition. Encyclopedia of Special Education: A Reference for the Education of Children, Adolescents, and Adults with Disabilities and Other Exceptional Individuals.|||Verney SP, Suchy-Dicey AM, Cholerton B, Calhoun D, Nelson L, Montine TJ, Ali T, Longstreth WT, & Buchwald D (2019). The associations among sociocultural factors and neuropsychological functioning in older American Indians: The Strong Heart Study. Neuropsychology, 33(8), 1078–1088. 10.1037/neu0000574|||Weintraub S, Besser L, Dodge HH, Teylan M, Ferris S, Goldstein FC, Giordani B, Kramer J, Loewenstein D, Marson D, Mungas D, Salmon D, Welsh-Bohmer K, Zhou XH, Shirk SD, Atri A, Kukull WA, Phelps C, & Morris JC (2018). Version 3 of the Alzheimer Disease Centers’ Neuropsychological Test Battery in the Uniform Data Set (UDS). Alzheimer Dis Assoc Disord, 32(1), 10–17. 10.1097/WAD.0000000000000223|||Weintraub S, Salmon D, Mercaldo N, Ferris S, Graff-Radford NR, Chui H, Cummings J, DeCarli C, Foster NL, Galasko D, Peskind E, Dietrich W, Beekly DL, Kukull WA, & Morris JC (2009). The Alzheimer’s Disease Centers’ Uniform Data Set (UDS): the neuropsychologic test battery. Alzheimer Dis Assoc Disord, 23(2), 91–101. 10.1097/WAD.0b013e318191c7dd|||Wood JL, Weintraub S, Coventry C, Xu J, Zhang H, Rogalski E, Mesulam MM, & Gefen T (2020). Montreal Cognitive Assessment (MoCA) Performance and Domain-Specific Index Scores in Amnestic Versus Aphasic Dementia. J Int Neuropsychol Soc, 26(9), 927–931. 10.1017/S135561772000048X|||Yang C, Wang L, Hu H, Dong X, Wang Y, & Yang F (2021). Montreal Cognitive Assessment: Seeking a Single Cutoff Score May Not Be Optimal. Evidence-based complementary and alternative medicine, 2021, 9984419–9984410. 10.1155/2021/9984419|||Zahodne LB, Sharifian N, Kraal AZ, Zaheed AB, Sol K, Morris EP, Schupf N, Manly JJ, & Brickman AM (2021). Socioeconomic and psychosocial mechanisms underlying racial/ethnic disparities in cognition among older adults. Neuropsychology, 35(3), 265–275. 10.1037/neu0000720|||Zahodne LB, Sol K, & Kraal Z (2019). Psychosocial Pathways to Racial/Ethnic Inequalities in Late-Life Memory Trajectories. J Gerontol B Psychol Sci Soc Sci, 74(3), 409–418. 10.1093/geronb/gbx113|||Zhang Y, Galloway JM, Welty TK, Wiebers DO, Whisnant JP, Devereux RB, Kizer JR, Howard BV, Cowan LD, Yeh J, Howard WJ, Wang W, Best L, & Lee ET (2008). Incidence and risk factors for stroke in American Indians: the Strong Heart Study. Circulation, 118(15), 1577–1584. 10.1161/CIRCULATIONAHA.108.772285|||Zhou Y, Ortiz F, Nunez C, Elashoff D, Woo E, Apostolova LG, Wolf S, Casado M, Caceres N, Panchal H, & Ringman JM (2015). Use of the MoCA in Detecting Early Alzheimer’s Disease in a Spanish-Speaking Population with Varied Levels of Education. Dement Geriatr Cogn Dis Extra, 5(1), 85–95. 10.1159/000365506