Evidence of validity and reliability of the Colombian version of Addenbroke’s Cognitive Examination Revised (ACE-R)

Abstract Objective The purpose of this study is to provide evidence that supports the validity and reliability of the Colombian version of the Addenbrooke’s Cognitive Examination Revised (ACE-R) in comparison to the MMSE at assessing and finding patients with Mild Cognitive Impairment (MCI). Additionally, the study aims to determine the optimal cut-off scores based on the age of a population with a low education level. Method This study included 314 individuals (235 participants diagnosed with MCI and 79 cognitively healthy) who live in two different rural departments (states) in Colombia. The participants were recruited for this study through community clubs for the older adults. Most of the individuals were female (236), the average age was 65.95 years of age (SD= 7.8), and the average education level was of 3.78 years (SD = 1.79). It is important to note that the sample only included people with a maximum of 6 years of schooling. Results A ROC analysis indicated that the ACE-R is more effective than the MMSE at evaluating and finding MCI individuals within the three groups. The cut-off points for the Under 60 years of age group was 83.50 (sensitivity 0.880% and specificity 0.632%); 61–69 years of age 80.50 (sensitivity 0.714% and specificity 0.677%); and Over 70 years of age was 79.50 (sensitivity 0.750% and specificity 0.659%). The internal consistency analysis with MacDonald’s Ω determined reliability indicators ≥70 in the ACE-R, except for the age range of 61 to 69 years. Conclusion The Colombian version of the ACE-R demonstrates to be a valid and reliable global cognitive screening tool. It is effective at discerning MCI individuals from healthy within a group of participants with a low education level.


Introduction
Mild cognitive impairment (MCI) is considered an intermediate state between the cognitive changes associated with normal aging and mild dementia (Petersen, 2004), with the prevalence between 10 and 20% in people older than 65 years (Petersen et al., 2010).
Dementia and its impact is predicted to increase from 50 million people to 82 million people in the next decade (World Health Organization [WHO], 2020) as a result of the global population aging process (Bonilla-Santos et al. 2021;Fang et al. 2014;Pan et al. 2022;Prince et al. 2013;Wang et al. 2017) and the increasing need for dependence of older adults who will be affected by this condition (Calderón et al. 2021;Hildreth & Church, 2015;Waheed et al. 2020).
Latin America and the Caribbean are among the regions that will be most affected, with an estimated increase of 7.6 million by 2030 (Organización Panamericana de la Salud, 2023).These regions exhibit a high prevalence of dementia compared to Europe and the United States, due to significant risk factors including genetics and social determinants of health (Parra et al. 2020;Santamaria-Garcia et al. 2023).In Colombia, the prevalence of dementia is 9.4% (>60 years), and it increases to 57.4% in people over 85 years of age, particularly among those with low income and low education levels (McEniry et al. 2019;O'Donovan et al. 2020;Paredes-Arturo et al. 2019;Pedraza et al. 2017;Vargas et al. 2014).
The availability of validated screening tools for assessing cognitive functioning in populations with low education levels and exposure to social adversities is indispensable as it can reduce healthcare costs.Therefore, highly sensitive neuropsychological instruments with diagnostic specificity are required (Broche-Pérez & López-Pujol, 2017;Carnero-Pardo, 2014;Lorentz et al. 2002;Matias-Guiu et al. 2015;Siciliano et al. 2016;Torralva et al. 2011;Wang et al. 2017) to identify dementia and its associated factors promptly (Porto et al. 2022).For multicentred dementia studies, the use of valid and reliable tools that ensure decision-making (Aiello et al. 2022;Tsoi et al. 2015) that can be applied in any globalized context becomtes a necessity (Santamaria-Garcia et al. 2023;Zugman et al.2023).
Addenbrooke's Cognitive Examination (ACE) and its subsequent versions (ACE-R, ACE-III, M-ACE, and Mini-Addenbrooke's Cognitive Examination) have demonstrated high sensitivity and specificity for the early detection of mild cognitive impairment and dementia.This instrument has been validated and translated into several languages (Mirza et al. 2017;Ospina, 2015;Waheed et al. 2020) and exhibits adequate psychometric properties (Calderón et al. 2021;Custodio et al. 2020;Junco & Prieto, 2014) (see Table 1).However, most studies have focused on patients in clinical or hospital care settings, with participants typically having a minimum of 9 years of schooling.Therefore, its interpretation for diagnosing mild cognitive impairment in populations with less than five years of schooling and deeper social problems is limited (Custodio et al. 2017;Ospina, 2015;Xiang et al. 2021) In the specific context of the low-income Colombian population, which encompasses a wide array of social, environmental, and economic challenges, this article aims to assess the validity and reliability of the Colombian version of the Addenbrooke's Cognitive Examination Revised (ACE-R) and determine the optimal in a population with less than 6 years of schooling.

Method
This study was carried out throughout the years 2019, 2020 and 2021, framed within the South Colombian study of dementia (see Figure 1).

Assessment procedure
The research protocol included a variety of neuropsychological tests for the early evaluation of Alzheimer's Disease.All study subjects joined the projects and followed the same stages until they were finally classified as part of the cognitively healthy control group or group with possible MCI.The stages consisted of 1. clinical interview and 2. application of cognitive screening tests (MMSE) and (ACE-R), depression screening (Yesavage Geriatric Scale -GDS), screening scale for the diagnosis of multi-infarct dementia-the Hachinski Ischemic Scale and the Consortium to Establish a Registry for Alzheimer's Disease (CERAD), Colombian version.
MCI diagnosis was determined through clinical consensus when the patient met the criteria stipulated by Petersen (2004) and Winblad et al. (2004): (a) changes in cognition recognized by the participant under evaluation and/or by a close informant; (b) MMSE score below 1.5 standard deviations according to the normative data for age and schooling; (c) Objective memory impairment (single domain amnestic MCI), memory impairment and another cognitive process (multiple-domain amnestic MCI), impairment of a cognitive process other than memory impairment (single domain non-amnestic MCI), impairment of two or more cognitive processes other than memory impairment (multi-domain non-amnestic MCI) assessed through CERAD domains (the participant having 1.5 standard deviations from what was expected for age and schooling based on the CERAD normative data for Colombia was taken into account); and (d) autonomy at performing functional activities of daily living assessed using the Lawton and Brody Scale (Scores of 8 for women and ≥ 5 for men).The Reiserbeg Global Scale of Impairment (GDS) was also used to assess the stage of cognitive deficit in (1) absence of cognitive impairment, (2) very mild cognitive impairment, (3) mild cognitive impairment, (4) moderate cognitive impairment and (5) absence of Alzheimer's dementia according to the nInCDS-ADRDA criteria, and absence of vascular dementia according to the nInDS-AIREn criteria.Participants with a history of (1) neurological disease, (2) traumatic brain injury and (3) neurodevelopmental and learning disabilities, were excluded from the study.The Colombian version of the ACE-R was adapted and validated in 2015 by Ospina, to create a reliable, low-cost, and easy-to-administer screening test that could assess multiple cognitive domains in the Colombian population.The adaptation of the original version was accomplished through translation and extensive revision by experts in neurology and neuropsychology with knowledge of the two languages involved (English and Spanish).The modifications made for the Colombian version were for the attention and orientation domain within the three-word register item; based on the word typification analysis, apple, ball, and key were the words included.In the memory domain, the address item was modified according to how it is used in a conventional and generalized manner in addresses in Colombia.The retrograde memory questions were also modified to be consistent with the schooling average of adults and older adults of the population in the study-that is, these questions would be known to the general population so that the act of recalling this information could be assessed, thus avoiding a bias due to lack of knowledge.no modifications were made for the fluency domain.In the language domain, the comprehension, repetition of words and writing items remained the same, as no significant linguistic or cultural differences could be found with respect to the original test.Changes were made to the sentence reading item, as sentences became meaningless when literally translated into Spanish.For this reason, they were changed to sentences that had been widely adopted in other tests validated in Colombia, such as the MMSE and CERAD: 'If I don't go down, then you go up' and 'The man is walking down the street' .In the naming item there are some drawings that differ from those in the original, such as the lion, the pig, the chicken, the guitar, and the shovel.The pencil, the watch, the camel, the crocodile, the anchor, the crown, and the accordion remained unchanged.For the comprehension and semantic associations item, modifications were made to add a different degree of difficulty to each question.In the reading item, after a consensus was reached by a group of experts that included neurologists, linguists and neuropsychologists, the words Hectare, Water, Access, Success, and That (Hectárea, Agüita, Acceso, Éxito, Aquello) were chosen.The author did not deem it necessary to make any modifications in the visuospatial skills domain.

Participants
A total of 314 subjects are included in the study, who were mostly female (75.2%), with a maximum level of 6 years of schooling and who belonged to two rural departments located in the Andean region: Huila (64.6%) and Caquetá (35.4%), in Colombia.The average age was 65.95 years of age (SD = 7.8), 74.8% of the participants had a diagnosis of MCI and 25.2% were cognitively healthy.

Data analysis
A descriptive analysis of the sample obtained and classified the variables 'presence/absence of mild cognitive impairment' , 'age' and 'educational level' .Subsequently, a Kolmogorov-Smirnov analysis was performed to determine whether the measurements were parametrically distributed.Age was stablished as a categorical variable, considering the ranges under 60 years, from 61 to 69 years and over 70 years.The comparison between cognitively healthy and MCI participants was made with the Mann-Whitney U test on independent samples, the Kruskal-Wallis test by age ranges in MCI and the Levene test was applied to determine the equality of variances.To establish the sensitivity, specificity and cut-off points of each test separate analyses were carried out using ROC curves by age range.MacDonald's Ω statistics were used to determine the internal consistency of the ACE-R test.For statistical decision-making, a significance of less than 0.05 was used.The analyses were performed with the SPSS v26 ® and Jasp ® software.It was impossible to match any of the measured variables to normality criteria.Therefore, we determined that the data showed non-parametric behavior.

Ethics
During the execution of this project, the fundamental base was the knowledge and compliance of the different ethical standards at international and national level regarding the study of humans.This included the Declaration of Helsinki, the

Results
Regarding the description of the sample, a mean age of 65.95 years of age was identified, and the corresponding mean level of education was 3.78 years (SD = 1.79).Of these, 11.8% reported a history of dementia in their family and less than 5% reported some major somatic comorbidity-except for diabetes-which was present in 12.7% of those evaluated (Table 2).
Moreover, there are no significant differences when comparing age and sex, but there are in terms of education (age ranges 61-69); however, it was identified that, according to the age ranges (under 60 years, 61-69 years and over 70 years), reported schooling decreases.Therefore (and in line with the available empirical evidence), this study defines the age ranges as criteria for establishing internal consistency indices (sensitivity and specificity) (Table 3).
The difference in medians in total MMSE and ACE-R between cognitively healthy control patients and patients with MCI, indicates that the assessment subscales presented significant differences, excluding the attention subscale (Table 4).
An ROC analysis was conducted to simultaneously compare the behavior in the ACE-R and MMSE in terms of area under the curve, cut-off point, sensitivity, and specificity for each of the age segments defined.When comparing the areas under the ROC curves covered by the two instruments, statistically significant differences were obtained (z = 3,288; diff.0.105; C.I. 95% = 0.041-0.168;p value = 0.001) (Table 5).
Both tests exhibited optimal indicators for their use in MCI discrimination, being higher for ACE-R in all three groups (Figure 2).
Regarding the differences in means by age group in patients with MCI, significant differences were identified in almost all factors measured, except for the ACE-R attention (Supplementary Table S1).
Finally, an analysis was performed using MacDonald's Ω, which determined the internal consistency of the ACE-R instrument with the entire sample and subsamples.The tests above yielded reliability indicators of ≥ 70 except for the age range of 61 to 69 years (Table 6).

Discussion
The objective of the study is to provide evidence of the validity and reliability of the Colombian version of the Addenbrooke Cognitive Test-Revised (ACE-R) in comparison with the MMSE at discriminating patients with Mild Cognitive Impairment (MCI) from healthy ones.In addition, the study aims to determine the optimal cut-off scores analyzed by age ranges for a population with a very low education level in Colombia According to the ROC analysis, the ACE-R outperformed the MMSE in detecting mild cognitive impairment across all three age groups.
This study determinates the ACE-R cut-off points by age as follows: for the group under 60 years of age, the score was 83.50 (with a sensitivity of 0.880%, and a specificity of 0.632%); for the group between 61 and 69 years of age, the score was 80.50 (with a sensitivity of 0.714% and a specificity of 0.677%); and for the group older than 70 years of age, the score was 79.50 (with a sensitivity of 0.750% and a specificity of 0.659%).The cut-off points that this study reports differ from Ospina (2015), who determined a cut-off point of 87 for healthy subjects from Colombia with a mean of 10 years of schooling and with higher specificity and sensitivity than ours.
Regarding the ACER-R validation studies in Latin America and the Caribbean, our study consists of a population in conditions close to illiteracy and that have been exposed to   (Torralva et al. 2011) 12.9(±4.6);Thus, the low-income Colombian population, with a wide range of social, environmental and economic difficulties, specifically makes the task of ensuring equivalency processes difficult.The evidence of validity and reliability of the ACE-R in the present study shows a lower internal consistency (≥70, except for the age range of 61 to 69 years) than the one obtained in the adaptation and validation carried out by Ospina (2015).This can be explained by the heterogeneity of the participants, particularly due to the differences in age and low educational level (nieto et al. 2016).Therefore, it is necessary to increase the use of the ACE-R in a population with such characteristics, given the implications of these difficulties for most of the Colombian adult population.This explains why the adaptation processes in this type of context cannot be extrapolated to those of other countries for example, Spain (García-Caballero et al. 2006;Matias-Guiu et al. 2015), Germany (Alexopoulos et al. 2010) or Japan (Yoshida et al. 2012), where the older adult population has on average a higher educational level and has not been exposed to similar vulnerable conditions.
In this study, the performance in terms of the cut-off point, sensitivity and specificity of the ACE-R exhibit optimal indicators, which allows discrimination by age segments and shows the best indicators in the age group under 60 years of age.This is  of special interest for the processes of cognitive evaluation through ACE-R in the Colombian population, given the variety of social, environmental, and economic problems that impact the access and the permanence in school systems, which undoubtedly have age implications.
There is no doubt that dementia and cognitive decline represent one of the major progressive public health problems worldwide.In this sense, it is of special interest to have instruments with adequate psychometric properties that respond to the specific cultural needs of the context and that offer pertinent and relevant data on cognitive functioning in the shortest possible time (Parra et al. 2020).
In this sense, this study provides a screening tool that has evidence of validity and reliability and allows the identification of possible cognitive impairments within groups of Colombian adults with a low education level who also live in adverse conditions.This is of special interest due to the use of the ACE-R in multicenter studies (Aiello et al. 2022;Tsoi et al. 2015), which have considered including populations with low education and socioeconomic precariousness (Santamaria-Garcia et al. 2023).
This study reports several limitations, including the fact that the participants were recruited from community centers.There are differences in the years of schooling mean between the range 61-69 years of age of MCI group and the healthy ones, and there are no biomarkers (Bonilla-Santos et al. 2021) that correlate with the results neuropsychological tests.Lastly, considering that the ACE-R is a sensitive screening test for the identification of MCI, the limitation in specificity should be reported.Besides, the validation analyzes are not carried out with the dementia clinical group.

Figure 2 .
Figure 2. Characteristics of the ROC curves of the ACe-R and the MMSe by educational level and total population.

Table 1 .
education level of participants worldwide assessed with the ACeR-R.
Figure 1.Selection of study population.
International Ethical Guidelines for Biomedical Experimentation in Human Beings, the Standards of Good Clinical Practices, Resolution no.008430 of 1993 of the Ministry of Health and Social Protection of Colombia as well as the Law that dictates the Code of Ethics and Bioethics of the Psychologist in Colombia, Law 1090 of 2006.This project included the Endorsement of the Ethics, Bioethics and Research Committee 002-006 University Hospital.

Table 2 .
Characteristics Of the sample assessed.

Table 3 .
Classification of study groups by sex, age, and level of education.
M.C.i.: Mild Cognitive impairment; C.H.: Cognitively Healthy.p value was made from the U Mann Whitney.

Table 4 .
Differences between cognitively healthy control patients and patients with MCi.

Table 5 .
ROC analysis to differentiate performance of ACe-R and MMSe.

Table 6 .
Reliability analysis of the ACe-R by educational level and total