Estimating the minimal clinically important difference of upper extremity outcome measures in chronic stroke patients with moderate to severe impairment: a cross-sectional study

ABSTRACT Background Approximately 70% of chronic stroke patients experience upper extremity (UE) functional impairments, and UE outcome measures are often used as quality-of-life indicators. Objective The purpose of this study was to estimate minimal clinically important difference (MCID) values for UE outcome measures in chronic stroke patients with moderate to severe UE hemiplegia. Methods This study was a cross-sectional study, conducted as a secondary analysis of data from the ReoGo-J study, a multicenter, prospective, randomized, parallel-group trial of robot-assisted self-training for UE hemiplegia in chronic stroke. The patients were randomized to 1 of 3 treatment groups. Treatment was provided 3 times a week for 10 weeks, and UE outcome measures were evaluated before and after treatment. The anchor-based method was used to estimate MCID values for UE outcome measures, with Stroke Impact Scale (SIS) subscales as anchors. MCID values were estimated by identifying cutoff values in a receiver operating characteristic (ROC) curve. Results Between-group comparisons of UE outcome measures, based on the clinically important difference (CID) values of SIS subscales, revealed significant differences in both the Amount of Use (AOU) and Quality of Movement (QOM) components of the Motor Activity Log (MAL)-14. The estimated MCID values were 0.89 for the AOU component and 0.77 for the QOM component. Conclusions The estimated MCID values for the MAL-14 not only add information regarding the clinical characteristics of the MAL-14 but also facilitate interpretations of changing scores in chronic stroke patients with moderate to severe UE hemiplegia undergoing rehabilitation therapy. Study registration https://www.umin.ac.jp/ctr/index.htm (UMIN000022509; 1 July 2016).


Introduction
Stroke impairs physical, cognitive, and emotional functions, negatively affecting quality of life (QOL). 1Approximately 70% of patients experience upper extremity (UE) functional impairment on the affected side following stroke, 2 making UE hemiplegia one of the most common strokeassociated disabilities.UE hemiplegia limits hand use, affecting activities of daily living (ADL) and reducing QOL. 3 Therefore, UE outcome measures are important indicators for assessing meaningful improvements in QOL among stroke patients. 4veral standardized UE outcome measures are commonly used to assess post-stroke rehabilitation progress. 5,6The validity and reliability of many UE outcome measures used in routine clinical practice and research are well established.However, determining whether observed statistically significant differences in UE outcomes translate into clinically significant or trivial effects is valuable for understanding how rehabilitation affects QOL.Therefore, defining minimal clinically important difference (MCID) 5 values for UE outcome measures will improve interpretations of rehabilitation studies in stroke patients.
MCID values can be derived through various methods, which are often classified as either distribution-based or anchor-based. 7Some studies using distribution-based methods have focused only on minimally detectable changes, without assessing their clinical impacts. 7When using anchor-based methods, MCID values are estimated based on changes considered important to either the clinician or the patient. 8Because anchor-based methods typically rely on patient-reported outcomes (PROs), the estimated MCID directly reflects the patients' point of view.By using a PRO that assesses QOL in anchorbased MCID estimations for UE outcome measures, the MCID for improved QOL can be determined.We were unable to locate any published studies using an anchor-based method with a QOL measure as an anchor to estimate MCID values for the Motor Activity Log (MAL)-14, 9 which measures spontaneous, real-world arm use.
The present study attempted to estimate MCID values for UE functional outcome measures, including the Fugl-Meyer Assessment (FMA), 10 Action Research Arm Test (ARAT), 11 and MAL-14, in chronic stroke patients with moderate to severe UE hemiplegia using an anchor-based method with a QOL measure [Stroke Impact Scale (SIS) 12 as the anchor.According to Kelly etal., 13 the MAL-14 is the only QOL-associated UE outcome measure used to assess chronic poststroke UE hemiparesis.Therefore, this study tested the hypothesis that MCID values for the MAL-14 could be estimated using the SIS as an anchor.

Study design
This study was a cross-sectional study, conducted as a secondary analysis of data from the ReoGo-J study, a multicenter, prospective, randomized, parallelgroup trial of robot-assisted self-training for UE hemiplegia in patients with chronic stroke.The trial was conducted at 25 centers from 29 November 2016, to 12 November 2018. 14Participants were randomly assigned to 1 of 3 groups, with each group receiving a different therapist-guided intervention delivered in 1-hour sessions 3 times a week for 10 weeks: robotic therapy (RT), multi-therapy (MT), or control therapy (CT).The CT group received standard occupational therapy and unassisted self-training.The RT group received standard occupational therapy and robotassisted self-training using the ReoGo-J unit, a UE rehabilitation device (Teijin Pharma Ltd, Tokyo, Japan.Certification number 226AHBZX00029000).The MT group received occupation therapy based on constraint-induced movement therapy (CIMT), a task-oriented practice designed to transfer the functional improvements acquired during practice to daily life, and robot-assisted self-training using the ReoGo-J unit.
The study protocol 15 was approved by the institutional review board of the Hyogo College of Medicine (No. 2248) and was conducted in accordance with the Declaration of Helsinki.All participants provided written informed consent.The study was registered at https://www.umin.ac.jp/ ctr/index.htm(UMIN000022509; 1 July 2016).The study conforms to the STROBE guidelines.

Participants
The characteristics of the study participants are shown in Table 1.Patients were eligible if they were between 20 and 80 years of age at study entry, diagnosed with UE hemiplegia due to a first clinically diagnosed supratentorial stroke that occurred at least 6 months prior to study entry, and received outpatient rehabilitation therapy for UE dysfunction. 14atients were required to have FMA-UE scores below 44, UE distal function of 1b or above on the Stroke Impairment Assessment Set, 16 and a score equal to or less than 2 on the Modified Ashworth Scale. 17Exclusion criteria were multiple cerebral infarctions, diagnosis of cerebellar or brainstem infarction, severe pain in the UE, and improvement of UE function without treatment.To avoid complications with other diseases, patients were excluded if they were diagnosed with neuromuscular diseases; malignant tumors; balance or gait disorders; or other serious, uncontrolled diseases, including heart, kidney, and liver diseases.Patients diagnosed with severe aphasia or cognitive impairment (Mini-Mental State Examination score ≤ 24) were excluded to ensure sufficient cognitive ability to participate in training and assessment and provide informed consent.To avoid potential confounding related to other treatments, patients were excluded if they had any history of robot-assisted UE training or CIMT for UE hemiplegia or received botulinum toxin injections within 16 weeks prior to enrollment.Patients could also be deemed ineligible at the investigator's discretion. 15ata on demographics, anthropometrics, stroke information, comorbidities, lifestyle factors, and baseline functional measures were collected from all patients at study entry and entered into an electronic data collection system (Viedoc 4, Pharma Consulting Group) and a blinded web allocation system (web allocation system: CliSSS Randoman, Meditrix Co, Ltd).

Outcome measures
Each group was evaluated for UE outcome measures at baseline, 5 weeks, and 10 weeks post-training.The study analyzed changes in the MAL-14; the FMA-UE; the ARAT; and the SIS.
The MAL-14 contains 14 items to determine (1) Amount of Use (AOU) and (2) Quality of Movement (QOM), with average scores for both measures ranging from 0 to 5. 9 The FMA-UE consists of 33 items in four subsections, with summary scores ranging from 0 to 66.The ARAT consists of 19 item in four subscales, with summary scores ranging from 0 to 57. 11 The SIS contains 59 items measuring eight subscales, with summary scores for each subscale ranging from 0 to 100, calculated as [(actual raw score -lowest possible raw score)/ possible raw score] * 100. 12he FMA-UE and ARAT were assessed remotely using video -video assessments processed for blinding. 18Other assessments required subjective evaluations and brief palpations and were performed at each medical facility.The clinicians and therapists who performed physical assessments were blinded to treatment groups to ensure objectivity and were not present during training sessions.

Data analysis
Baseline UE outcome measures (FMA-UE, ARAT, and MAL-14) and SIS physical domain subscales [Strength, Hand Function, ADL/IADL, and Mobility] were analyzed with descriptive statistics to characterize the sample.
To estimate MCID values for UE outcome measures, the anchor-based method was used with SIS subscale anchors.Participants were divided into 2 groups based on the following established clinically important difference (CID) values for each SIS subscale: 9.2 for Strength, 5.9 for ADL/IADL, 4.5 for Mobility, and 17.8 for Hand Function. 19Betweengroup differences in mean change from baseline in UE outcome measures were assessed using Student's t-test.As described by recent reports, 20 a receiver operating characteristic (ROC) curve was generated for each UE outcome measure, and the area under the curve (AUC) was estimated to describe the ability of each measure to distinguish between subjects who improved and those who did not.AUC values were interpreted according to previous studies: greater than 0.9 indicated high accuracy, 0.7 to 0.9 indicated moderate accuracy, 0.5 to 0.7 indicated low accuracy, and less than 0.50 indicated no ability to distinguish improving participants from stable participants. 21,22CID values were estimated for each UE outcome measure by identifying the point in the ROC curve (cutoff score) with the largest Youden Index, 23 defined as sensitivity + specificity − 1.The cutoff score was used to distinguish participants who improved from those who did not.All

Results
This study included 121 subjects with hemiplegia after stroke (Table 1), including 93 men (76.9%) and 28 women (23.1%), with an average age of 58.9 years.Most subjects were independent before stroke.The first evaluation (before intervention) was performed at a mean of 36.7 months poststroke, and the second evaluation occurred 10 weeks after intervention.For all UE outcome measures, the mean ±1 standard deviation at baseline did not exceed the upper or lower limits of each score range, meaning there was no apparent floor or sealing effect.Mean changes in UE outcome measures after intervention were shown in Table 2. Comparison of changes in UE outcome measures between groups divided by SIS subscale CID values and estimated MCID values (cutoff values) are shown in Table 3. ROC curve analysis revealed that changes in UE outcome measures during the intervention were able to distinguish participants who demonstrated CIDs in SIS subscales from those who did not (Figure 1).
The ability of the FMA-UE to distinguish patients who improved from those who did not depends on the SIS subscale anchor used, and no significant between-group differences were observed for any SIS subscale anchor.When the Strength subscale was used as the anchor, the mean FMA-UE value was higher in the above-CID group than in the below-CID group, whereas the mean FMA-UE value was lower in the above-CID group than in the below-CID group when using any other SIS subscale anchor.The FMA-UE cutoff values were estimated at 5 when anchored by the Strength subscale and at 1 when anchored by the other SIS subscales, but the mean AUC values did not exceed 0.6 for any SIS subscale anchor.
The ARAT showed similar trends to the FMA-UE.No significant between-group differences were observed for any SIS subscale anchor.The ARAT cutoff values were estimated at 5 when anchored by the Strength or Mobility subscales, at 8 when anchored by the Hand Function subscale, and at − 1 when anchored by the ADL/IADL subscale, but the mean AUC values did not exceed 0.6 for any SIS subscale anchor.
Both the AOU and QOM components of the MAL-14 showed similar results, with significant between-group differences when anchored by the Strength subscale (p < 0.001).In addition, significant between-group differences were observed for both the AOU and QOM components when anchored by the ADL/IADL (p = 0.043) and Hand Function subscales (p = 0.018), but no significant differences were observed for any other SIS subscale anchor.The cutoff values were 0.89 for the AOU component and 0.77 for the QOM component when using the Strength subscale anchor, with mean AUC values of 0.73 [95% confidence interval (CI), 0.64-0.83)]and 0.71 (95% CI, 0.61-0.81),respectively.

Discussion
This study estimated MCID values for UE outcome measures in chronic stroke patients with moderate to severe UE hemiplegia using an anchor-based method to determine the significance of changes in UE outcome measures.The SIS, a strokespecific, health-related QOL scale that has been widely used to assess QOL improvements after stroke rehabilitation [24][25][26][27][28][29][30][31][32] was selected as the anchor.MCID values could not be estimated for the FMA or ARAT, but MCID values were able to be estimated with moderate accuracy for the MAL-14 components.An MCID of 0.89 was estimated for the AOU component, and an MCID of 0.77 was estimated for the QOM component.The AUC values for both components were larger than 0.7, and the lower limits of both 95% CIs were larger than 0.5.MCID values can be obtained using the distribution-based method or the anchor-based method. 7he distribution-based method is limited because derived values do not necessarily indicate the importance of the scores. 8By contrast, the anchorbased method estimates MCID values by comparing changes against an "anchor," most often the patient's or clinician's assessment of the change. 7,8n this study, to determine which UE outcome measures are important indicators for assessing meaningful improvements in QOL, 4 we applied an anchor-based method using the SIS PRO as the anchor to determine MCID values.Although prior studies have reported MCID values for the MAL-14 using the distribution-based method in stroke patients, [33][34][35] only one MCID value has been reported for the QOM component of the MAL-14 using the anchor-based method in stroke patients. 8To the best of our knowledge, this is the first study to estimate an MCID value for the AOU component of the MAL-14 using the anchor-based method in stroke patients.
Lang et al. 8 used the anchor-based method to estimate MCID values for the QOM component of the MAL-14 in acute stroke patients, reporting MCID values of 1.0 and 1.1 for the affected dominant and non-dominant sides, respectively.These MCID values are larger than the MCID values estimated in the present study, possibly because apparent values may be larger when considering spontaneous UE functional recovery and rapid improvement after forced rest during the acute stroke period.Using the distribution-based method, Van der Lee et al. 33 reported minimal detectable change (MDC) with 95% certainty (MDC95) values of 0.6-0.75(12%-15%) for the AOU and QOM components, and Chen et al. 35 reported MDC with 90% certainty (MDC90) values of 0.84 (16.8%) for the AOU component and 0.77 (15.4%) for the QOM component.The MDC is defined as the smallest change in an outcome measure that exceeds random variation or measurement error 36 and represents the threshold for true change with some degree of certainty.Ideally, the MDC should be smaller than or equal to the CID so that no significant changes indicated by the CID can be attributed to chance or measurement error. 21In this study, the estimated MCID values for the AOU and QOM components of the MAL-14 were equal to or greater than the MDC95 and MDC90 values reported by previous studies.Therefore, an improvement of 0.89 (17.8%) in the AOU component or an improvement of 0.77 (15.4%) in the QOM component can be considered real and clinically meaningful changes in patients with chronic stroke.The MAL-14 was developed as a subjective, functional UE outcome measure to assess realworld performance and is administered through a semi-structured interview to determine how much (AOU) and how well (QOM) the individual uses the affected UE. 9 Similarly, the SIS was developed as a subjective outcome measure that assesses various problems that may have arisen due to stroke. 37Duncan et al. 12 assessed the item difficulty ranges for the SIS subscales using a Rasch analysis expressed by a logit, reporting ranges of − 3.80 to 4.00 for Strength, −2.10 to 2.10 for Hand Function, −2.75 to 2.63 for ADL/IADL, and − 4.16 to 3.48 for Mobility.Among the 4 SIS subscales, Mobility has the easiest performable range, but this subscale may not be relevant to improvements in the affected UE because tasks assessed by this subscale do not require the use of the affected hand (e.g."how difficult was it to sit without losing your balance?").Therefore, the Mobility subscale was not considered an appropriate anchor for estimating MCID values for UE functional assessment.Following the Mobility subscale, the Strength subscale has the next broadest and easiest ranges, with questions relevant to the affected UE (e.g."how would you rate the strength of your arm that was most affected by your stroke?").Therefore, stroke patients with moderate or severe UE hemiplegia may experience "clinically meaningful" improvements in the real-life muscle strength of the affected UE when clinical improvements are observed in the MAL-14.The MAL-14 has less accuracy for predicting meaningful improvements in the Hand Function or ADL/IAL subscales in stroke patients with moderate to severe hemiplegia, as indicated by their associated AUC values, because the questions associated with these subscales are more difficult.
This study has several limitations that should be considered when interpreting the results.First, since this study was conducted as a secondary analysis of data from the original prospective study, sample validity and statistical power could not be examined in detail, or a distribution-based method to estimate the MDC values could not be conducted.Second, MCID values were only estimated from patients who showed overall improvements, and different values have been reported for improvements and deterioration 38,39; therefore, the estimated MCID values may only apply only to stroke patients who improve after rehabilitation therapy.Third, the estimated MCID values may not be suitable for other stroke patient groups because previous studies have shown that the estimated MDC and CID values may vary according to baseline symptom severity. 40,41In particular, MCID values may be estimated lower in the elderly with cognitive impairment because of their reduced motor learning ability.Fourth, MCID values could not be estimated for the FMA-UE or ARAT due to low AUC values in ROC analysis, which may be due to the small changes in these UE outcome measures in patients with moderate to severe UE hemiplegia.Fourth, the estimated MCID values for the MAL-14, when anchored by the Strength subscale, had only moderate AUC values (0.71 and 0.73).The assessment of more cases remains necessary to obtain more accurate MCID values.Despite these limitations, this secondary analysis was performed on real-world data from a study that followed a strict protocol. 15Unlike studies that utilize clinician-reported outcomes as anchors, the present study used the SIS to anchor these estimates.The SIS is a PRO that may represent the patients' opinions of "clinically meaningful" changes.Future studies including more stroke cases and using the SIS to anchor estimated MCID values are recommended for UE outcome measures.

Conclusions
The estimated MCID values determined by the present study add information regarding the clinical characteristics of the MAL-14, a QOL measure related to real world arm use, and facilitate interpretations of changes in MAL-14 scores among stroke patients undergoing rehabilitation therapy.Among chronic stroke patients with moderate to severe UE hemiplegia, the mean changes in the MAL-14, or how much (AOU) and how well (QOM) the patients use, were 0.46 and 0.44, while the estimated MCID values were 0.89 and 0.77.Therefore, mean changes beyond these values in the MAL-14 in similar eligible patients are considered clinically significant.Future studies with more cases that use the SIS to anchor MCID estimates should be performed to obtain more accurate MCID values for UE outcome measures.

Figure 1 .
Figure 1.Receiver operating characteristic curves for mean changes in upper extremity outcome measures using stroke Impact Scale subscales as anchors.The y-axis shows the sensitivity values (true-positive rate), and the x-axis shows 1 − specificity values (falsepositive rate).ADL, activities of daily living; AOU, Amount of Use; ARAT, Action Research Arm Test; CID, clinically important difference; FMA-UE, Fugl-Meyer Assessment Upper Extremity; IADL, instrumental activities of daily living; MAL, Motor Activity Log; QOM, Quality of Movement; SIS, Stroke Impact Scale; UE, upper extremity.
Dr. Uchiyama reports nonfinancial support from Teijin Pharma and Tsukuba Clinical Research and Development Organization, University of Tsukuba, during the conduct of the study.Dr. Takebayashi reports personal fees from Teijin Pharma and nonfinancial support from Tsukuba Clinical Research and Development Organization, University of Tsukuba, during the conduct of the study.K. Takahashi reports personal fees from Teijin Pharma and nonfinancial support from Tsukuba Clinical Research and Development Organization, University of Tsukuba, during the conduct of the study.Dr. Amano reports nonfinancial support from Tsukuba Clinical Research and Development Organization, University of Tsukuba, during the conduct of the study.M. Sakai reports nonfinancial support from Teijin Pharma during the conduct of the study.Dr. Hashimoto reports nonfinancial support from Teijin Pharma during the conduct of the study.Dr. Hachisuka reports nonfinancial support from

Table 2 .
Mean changes in UE outcome measures after intervention.
ADL, activities of daily living; AOU, Amount of Use; ARAT, Action Research Arm Test; FMA-UE, Fugl-Meyer Assessment Upper Extremity; IADL, instrumental activities of daily living; MAL, Motor Activity Log; QOM, Quality of Movement; SD, standard deviation; SIS, Stroke Impact Scale; UE, upper extremity.

Table 3 .
Comparisons of changes in UE outcome measures between groups according to the CID values of SIS subscale anchors and the estimated MCID value (cutoff value).Amount of Use; ARAT, Action Research Arm Test; AUC, area under the curve; CI, confidence interval; CID, clinically important difference; FMA-UE, Fugl-Meyer Assessment Upper Extremity; IADL, instrumental activities of daily living; MAL, Motor Activity Log; MCID: minimal clinically important difference; QOM, Quality of Movement; SD, standard deviation; SIS, Stroke Impact Scale; UE, upper extremity.