Clinical interpretation and cutoff scores for manual ability measured by the ABILHAND questionnaire in people with stroke

ABSTRACT Background The ABILHAND questionnaire is recommended to assess perceived manual ability after stroke; however, more knowledge on interpretability is needed to improve the clinical applicability. Objectives To determine clinically meaningful cutoff scores for different levels of perceived manual ability, assessed by ABILHAND, corresponding to established observed and perceived upper extremity assessments post stroke. Methods This cross-sectional study, part of the Stroke Arm Longitudinal Study (SALGOT) at the University of Gothenburg, included 80 participants with upper extremity impairments after stroke. The self-reported upper extremity functioning was assessed with ABILHAND and Stroke Impact Scale Hand (SIS Hand), and the observed functioning was assessed by Fugl-Meyer Assessment for Upper Extremity (FMA-UE) and Action Research Arm Test (ARAT) at 3 months after stroke. Receiver operating characteristic curve, sensitivity, and specificity analyses were used to determine the cutoffs. Results The overall discriminating accuracy was excellent (AUC > 0.90) for most of the cutoffs and sensitivity and specificity values ranged from 0.73 to 1.0. The ABILHAND cutoff score 1.78 discriminated well between low and good functioning resulting in a 95% match with SIS Hand and 87.5% match with ARAT and FMA-UE. Conclusions The determined cutoff scores of the ABILHAND, validated through established upper extremity assessments, will provide a useful tool to clinicians when interpreting the logit scores and when selecting individualized treatment options. ABILHAND matched well with self-reported SIS Hand, but discrepancies found with observed scales implies that self-perceived assessments should be complemented with observed assessments.


Background
After stroke, impairments of the upper extremity are common and affect approximately 50% to 70% 1,2 in the acute phase. About 40% have remaining impairments more than three months after stroke, 1,3 and these impairments may negatively impact the ability to perform daily manual activities. 4 To follow recovery and evaluate effectiveness of interventions after stroke, reliable, valid, and responsive outcome measures capturing different aspects of functioning and disability according to the International Classification of Functioning, Disability and Health (ICF) should be used. [5][6][7] Patient-reported outcomes after stroke, such as the Stroke Impact Scale (SIS) 8 and the ABILHAND questionnaire, 9 are important as they reflect the individual's perceived difficulty to use the upper extremity and perform activities in their usual environment. The ABILHAND questionnaire 9 is an interview-based measure that is recommended to assess perceived manual ability after stroke. 5 The questionnaire is developed using the Rasch measurement model, 10 a method to convert ordinal raw scores into a linear measure that is presented in logits (i.e. log odds units). The logit scale ranges from plus to minus around zero, which defines the center of the scale, but it has no absolute minimum and maximum values. The ABILHAND has proven to be valid and reliable in stroke 9,11,12 but as far as we know, no cutoff scores of the logit scale are established to distinguish between different manual ability levels. More knowledge on interpretability of the logits would improve the clinical applicability of the ABILHAND.
Observational performance-based measures such as the Fugl-Meyer Assessment for Upper Extremity (FMA-UE) 13 and Action Research Arm Test (ARAT) 14 are recommended core measures to be included in stroke trials. 6,15 Previous studies have found that FMA-UE and ARAT are highly related 16 but that these measures have discrepancies in relation to patient-reported questionnaires. [17][18][19][20] A cross-sectional study 17 found that about 20% of persons 6 months after stroke showed good observed function measured by FMA-UE but low perceived ability according to the subscale hand of the SIS (SIS Hand). Discrepancies in clinically meaningful change assessed by ARAT and SIS Hand were observed in about 11% to 30% of participants included in upper extremity intervention trials in the sub-acute stage of stroke. 18,19 Discrepancies between perceived and observational measures demonstrate the need of more knowledge of these relations for selfreported scales.
ABILHAND logits have been found to be significantly correlated to the physical domains of SIS (strength, self-care, and hand functioning). 11 The FMA-UE and ARAT have also been reported to be associated to ABILHAND, explaining about 60% of the total variance of the logit score. 21 However, to our knowledge, no study has specifically investigated how different functioning categories of SIS Hand, FMA-UE, and ARAT correspond to levels of ABILHAND. A better understanding of how these measures are related will improve the interpretation and comparison of outcomes and guide clinicians in setting individualized, achievable, and realistic treatment goals.
The aim of the study was to determine clinically meaningful cutoff scores for different levels of perceived manual ability assessed by ABILHAND corresponding to established observed and perceived upper extremity assessments post stroke.

Participants
This cross-sectional study is a part of the Stroke Arm Longitudinal Study at the University of Gothenburg (SALGOT). The SALGOT comprised a non-selected cohort of 117 adults with first-ever stroke admitted to the largest stroke unit at Sahlgrenska University Hospital during an 18-month period between 2009 and 2010. 22 Inclusion criteria were i) ischemic or hemorrhagic stroke (confirmed by clinical neuroimaging); ii) upper extremity disability at 3 days after stroke onset (Action Research Arm Test, ARAT < 57); iii) resident in Gothenburg urban area; iv) 18 years or older; v) Swedish speaking; vi) no other upper extremity condition that limited the functional use of arm and hand; vii) no severe multiimpairment, diminished physical condition, or short life expectancy due to other chronic or terminal illness prior to stroke. The current study comprised data from the 3-month follow-up and included 80 participants who were able to rate the ABILHAND and SIS and perform the ARAT and FMA-UE. The flowchart of the inclusion process is shown in Figure 1.
The SALGOT study was approved by the Regional Ethical Review Board in Gothenburg, Sweden (225/08), and each participant or next of kin gave their written informed consent prior to their participation in the study. The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines for reporting observational data were followed. 23

Self-perceived manual ability
The ABILHAND questionnaire (stroke version) assesses the ability to perform daily activities that require the use of the upper extremity, whatever the strategies involved. It includes 23 common bimanual tasks that are rated as impossible (0 point), difficult (1 point), or easy (2 points). 9 The ABILHAND Rasch score is presented in logits (i.e. log odds units), and higher logits values represent better self-perceived ability. The logit scores in the current study were obtained by entering the raw scores from each participant into an online data analysis module (http://rssandbox.iescagilly.be). Face validity and content validity have been established for ABILHAND in chronic stroke 9 and adequate concurrent validity and excellent predictive validity with Stroke Impact Scale in subacute to chronic stroke. 11 Excellent internal consistency and test-retest reliability have been found in chronic stroke, 9,12 and ABILHAND has proven to be responsive in subacute and chronic stroke. 9,11 The SIS Hand is an interview-based questionnaire that includes five manual activities, such as carry heavy objects, tie a shoe lace, and pick up a dime, while using the most affected hand. 8 The items are scored on a 5-point scale from 1 (could not do at all) to 5 (not difficult at all). The final score is calculated as a composite mean score and converted into a percentage value (from 0 to 100) where a higher percentage value indicates a better perceived manual ability. 24 Reliability and validity of the scale are established. 25,26

Performance-based observational outcome measures
The ARAT is a performance-based outcome measure that assesses observed activity capacity of the upper extremity. 14,27 ARAT includes 19 items divided into four subscales: grasp, grip, pinch, and gross movement. Each item is scored from 0 (no movement) to 3 (normal movement). The total score ranges from 0 to 57, where a higher score indicates better performance. Validity and reliability of the ARAT have been established for stroke. 28,29 The FMA-UE 13 is a performance-based outcome measure to assess sensorimotor function after stroke. Motor function in the shoulder, elbow, wrist, and hand in the more affected upper extremity is assessed in 33 items. The items are scored on a 3-point ordinal scale (0 = cannot perform; 1 = performs partially; 2 = performs fully) except for reflex activity, which is scored as absent (2 points) or present (0 points). The total motor score ranges from 0 to 66 points, and the higher the score the better the function. The FMA-UE has been found to be valid and reliable after stroke. 13,30,31

Procedures
Clinical admission data were gathered from medical charts. Stroke severity at onset was assessed by the National Institutes of Health Stroke Scale (NIHSS) 32 and global disability at discharge by the modified Rankin Scale (mRS). 33 All assessments of upper extremity were performed according to the standardized protocol of the SALGOT study and were predominantly carried out by two experienced physiotherapists after a joint training period. 22

Statistics
Data were analyzed with the IBM SPSS Statistics version 27 (IBM Corporation, Armonk, New York, United States). Probability values less than 0.05 were defined as statistically significant. Descriptive statistics, such as frequencies, means and standard deviations (SD), and medians (interquartile range) were used to describe the cohort characteristics.
The cutoff scores for ARAT and the FMA-UE were based on previously established categories [34][35][36][37][38][39][40][41] (see Table 1). Corresponding categories for SIS Hand are lacking in the literature. In the current study, cutoff scores for SIS Hand (Table 1) were set through visual inspection of scatterplots between SIS Hand and ABILHAND, ARAT and FMA-UE. The middle cutoffs (SIS Hand ≥ 50, ARAT ≥ 22, and FMA-UE ≥ 32) were used to determine the ABILHAND cutoffs for low and good performance. 17 Cutoffs for the minimum and maximum values of the SIS Hand, ARAT, and FMA-UE were also used to calculate relating cutoffs for the ABILHAND logit scale.
The receiver operating characteristic (ROC) curve analysis was used to determine the optimal cutoff scores. The ROC curve shows the relationship between the sensitivity and specificity for every possible cutoff. The optimal cutoff score is defined by maximizing the sum of sensitivity (true positive rate) and 1 − specificity (false positive rate) from the coordinate points of the ROC curve. The accuracy of the test is measured by the area under the curve (AUC). The AUC equals 0.5 when the ROC curve corresponds to random chance and 1.0 for perfect accuracy. In the present study, the AUC was interpreted as good when >0.80 and excellent when >0.90. 42 Furthermore, matching and non-matching groups for the low and good upper extremity functioning levels were calculated and displayed in scatterplots.
The cutoffs determined for the ABILHAND logit scores corresponding to the perceived (SIS Hand) and observed (ARAT and FMA-UE) upper extremity functioning are shown in Table 3. Sensitivity and specificity values ranged from 0.73 to 1.0, and the overall discriminating accuracy determined by the AUC was excellent (>0.90) for most (14 out of 17) of the cutoffs. The highest accuracy was observed for the cutoff between low and good manual ability related to the SIS Hand (AUC 0.98, 95% CI 0.95-1.0).
The lowest cutoff of 0.47 on ABILHAND corresponded to the FMA-UE cutoff score 4 (the FMA-UE minimum score in the study sample) and the highest cutoff of 5.61 corresponded to the SIS Hand cutoff score of 100 (Table 3) The optimal ABILHAND cutoff score to differentiate between low and good UE performance was determined to be 1.78 logits for all scales (Table 3). This cutoff discriminated correctly 95% of participants (n = 76) assessed with SIS Hand and 87.5% of participants (n = 70) assessed with ARAT and FMA-UE (Figure 2-4). The 12.5% of participants (n = 10) who showed mismatch, identified for the observational scales of ARAT and FMA-UE, had low perceived manual ability but good observed functioning (Figure 3 and 4).

Discussion
The present study is the first to determine clinically meaningful cutoff scores for different levels of perceived manual ability assessed by ABILHAND in people with stroke. Three well established and recommended clinical scales covering both self-    reported and observed upper extremity assessments were used. Even when the accuracy was excellent for merely all determined cutoffs, the highest overall accuracy and best match was noted for the cutoff of 1.78, discriminating between low and good manual ability when assessed with the self-perceived SIS Hand score.
The cutoff score of 1.78 showed excellent accuracy in discriminating between low and good perceived manual ability irrespective of the clinical scales used. This finding was confirmed by the analysis of matching groups that revealed a high match, particularly observable between the ABILHAND and SIS Hand score (95% agreement).  The somewhat lower match (87.5%) between the ABILHAND and ARAT or FMA can possibly be explained by the fact that SIS Hand is a selfreported scale, while the scoring of ARAT and FMA-UE are based on clinicians' observation of performance. Similar kind of mismatch, as seen in the current study, was also found in a study with chronic stroke, in which 19% of the participants showed good observed scores (FMA-UE ≥ 32) but low perceived scores (SIS Hand ≥ 60). 17 The slightly higher percentage of mismatch reported by Essers et al. 17 might have been caused by a higher cutoff of SIS Hand that was used in that study. The good agreement between the two selfreported scales, ABILHAND and SIS Hand, was observed despite the minor differences between the scales. Particularly, in SIS Hand, the scoring only considers the perceived performance of the more affected hand, in contrast to the ABILHAND, in which difficulty to perform common daily activities is scored irrespective to the hand or methods used. One possible reason for this good match could be that two of the five items scored on SIS Hand are bimanual (open a jar/can, tie shoelaces) and that most of the items of ABILHAND require at least some active part from the affected arm (bimanual). Taken together, the cutoff of 1.78 seems to be a stable cutoff that can be used to differentiate between low and good manual ability measured by ABILHAND. Our findings also confirm that self-reported and observational scales might provide somewhat different information on upper extremity performance and therefore both should be included in a clinical assessment battery. 7,43 Self-awareness of upper extremity performance is an important aspect of rehabilitation that should be integrated in treatment.
In the current study, previously reported cutoff categories of well-known clinical scales, ARAT and FMA-UE, were used, and knowledge from these scales can be useful when interpreting the ABILHAND cutoffs. For example, the FMA-UE score 32 or more points has shown to be required to perform a unimanual drinking task with the more-affected arm. 40 Based on this information, it can be concluded that a person scoring at least 1.78 logits on ABILHAND will likely be able to use their more-affected arm at least partly in activities of daily living. However, the same ABILHAND cutoff 1.78 corresponded also to the subsequent lower cutoff categories of the SIS Hand, ARAT, and FMA-UE (Table 3). This indicates that the 1.78 cutoff is not sensitive enough to differentiate between these two adjacent cutoff categories defined by the clinical scales. Similarly, no differentiation could be made between two categories of FMA-UE (48 and 53 cutoffs). One possible explanation to this weaker discrimination can be the limited number of observations available with intermediate range of scores (Figure 3 and 4). Interestingly, the upper range cutoffs of ARAT ≥ 55 and FMA-UE ≥ 53 both corresponded to the same ABILHAND cutoff of 2.31 logits, which confirms its potential practical use, indicating a relatively good motor functioning.
Since the ABILHAND does not have a definite value for the lowest and highest possible scores, the corresponding lowest and highest possible cutoffs of well-known clinical scales can be useful. Our data show that the highest ABILHAND score (5.61 logits) corresponded to the highest cutoff of SIS Hand (score of 100). This finding suggests that a full score on SIS Hand corresponds to a higher perceived manual ability than the full score on ARAT and FMA-UE. This difference can be connected to the items of the SIS Hand that all require some ability to grasp, while ARAT and FMA-UE include items that assess both proximal and distal function of the upper extremity. As also supported by previous research, 20,44 it is important to highlight that even when a person achieves full score on ARAT or FMA-UE, perceived manual difficulties might still be reported.
Interestingly, a lower ABILHAND score (0.47 logits) was noted for the lowest category of the FMA-UE, compared to the ARAT and SIS Hand (1.41 logits). The lowest FMA score (4 points in the current sample) corresponds to almost no active motor function of the more-affected arm, while in ARAT, some active movement of the arm is required to gain a score higher than 0, which can explain this discrepancy. Our results also showed that all determined ABILHAND cutoffs were above 0, even when the ABILHAND logit scale reaches also to the minus side (approximately to −6 logits). A possible explanation for this could be that the participants in this study were included at 3 months after stroke. At this time, it is likely that most of the participants had learned a way to handle and compensate for their upper extremity impairments and limitations in everyday activities. This is an important finding and will help the clinicians to account for this when interpreting the results from ABILHAND.
A strength of the current study is that all available individuals from the SALGOT cohort (unselected longitudinal study) having a completed 3-month assessment were included in the analyses. Well established measures were used to determine the ABILHAND cutoffs, and care was taken to standardize the test situation and trained examiners performed the assessments. The discriminating ability for some defined cutoffs of ARAT and FMA-UE was limited due to the low number of data available in the middle ranges of these clinical scales. Furthermore, different factors such as cognitive deficits and mood might influence the ratings in self-reported outcomes, which may have affected the results. The time point of 3 months after stroke was selected, since it was expected that the participants had at least tried or resumed most of their major daily activities at this time point of stroke recovery. Nevertheless, as relationships of perceived and observational measures might change over time, the ABILHAND cutoffs ought to be verified also in other phases after stroke.

Conclusion
The determined cutoff scores of the ABILHAND, validated through established upper extremity assessments, will provide a useful tool to clinicians when interpreting the logit scores and when selecting individualized treatment options. ABILHAND matched well with self-reported SIS Hand, but discrepancies found with observed scales implies that self-perceived assessments should be complemented with observed assessments.