The Dutch language anterior cruciate ligament return to sport after injury scale (ACL-RSI) – validity and reliability

ABSTRACT The ACL-Return to Sport after Injury scale (ACL-RSI) measures athletes’ emotions, confidence in performance, and risk appraisal in relation to return to sport after ACL reconstruction. Aim of this study was to study the validity and reliability of the Dutch version of the ACL-RSI (ACL-RSI (NL)). Total 150 patients, who were 3–16 months postoperative, completed the ACL-RSI(NL) and 5 other questionnaires regarding psychological readiness to return to sports, knee-specific physical functioning, kinesiophobia, and health-specific locus of control. Construct validity of the ACL-RSI(NL) was determined with factor analysis and by exploring 10 hypotheses regarding correlations between ACL-RSI(NL) and the other questionnaires. For test–retest reliability, 107 patients (5–16 months postoperative) completed the ACL-RSI(NL) again 2 weeks after the first administration. Cronbach’s alpha, Intraclass Correlation Coefficient (ICC), SEM, and SDC, were calculated. Bland–Altman analysis was conducted to assess bias between test and retest. Nine hypotheses (90%) were confirmed, indicating good construct validity. The ACL-RSI(NL) showed good internal consistency (Cronbach’s alpha 0.94) and test–retest reliability (ICC 0.93). SEM was 5.5 and SDC was 15. A significant bias of 3.2 points between test and retest was found. Therefore, the ACL-RSI(NL) can be used to investigate psychological factors relevant to returning to sport after ACL reconstruction.


Introduction
Anterior cruciate ligament (ACL) rupture is a common injury in athletes. An ACL rupture is often surgically treated with an ACL reconstruction to restore the stability of the knee, aiming to allow patients to return to sport after rehabilitation (Marx, Jones, Angel, Wickiewicz, & Warren, 2003;Myklebust, 2005). After ACL reconstruction and the subsequent rehabilitation, many patients report a good knee function but do not return to their pre-injury level of sport participation (Ardern, Taylor, Feller, & Webster, 2014;Ardern et al., 2014;Feller & Webster, 2003;Kvist, Ek, Sporrstedt, & Good, 2005;Langford, Webster, & Feller, 2009;Webster, Feller, & Lambros, 2008). As surgery and functional outcome were satisfactory in these patients, the question remains which other factors may cause the low return-to-sports rate.
Previous research showed that the low rate of return to sports might be caused by psychological reactions to the initial injury, surgery, and following rehabilitation (Ardern, Taylor, Feller, & Webster, 2012Kvist et al., 2005;Langford et al., 2009;Tjong, Murnaghan, Nyhof-Young, & Ogilvie-Harris, 2014). The main reason is fear of re-rupture of the ACL (Ardern et al., , 2014Tjong et al., 2014;Tripp, Stanish, Ebel-Lam, Brewer, & Birchard, 2007). The athlete's psychological response before and after surgery was found to be related to whether or not the athlete returned to preinjury level of sports at 12 months post-operatively (Ardern, Webster, Taylor, & Feller, 2011;Langford et al., 2009;Webster et al., 2008). Furthermore, fear of new injuries, reduced motivation, and other negative psychological reactions are described in athletes who did not return to the pre-injury level of sports after ACL reconstruction (Gobbi & Francisco, 2006;Kvist et al., 2005;Langford et al., 2009;Tjong et al., 2014;Tripp et al., 2007). Consequently, athletes should not only be physically prepared to return to sports but they also have to be psychologically ready (Ardern et al., 2012(Ardern et al., , 2014. Research showed that physical and psychological readiness to return to the sports, do not always coincide (Quinn & Fallon, 1999). Returning to the sport before the athlete is psychologically ready may lead to anxiety, reoccurrence of injury, injuries to other body parts, depression, and performance degradation (Gobbi & Francisco, 2006;Tjong et al., 2014;Tripp et al., 2007). It is suggested that attention to psychological recovery, in addition to physical recovery, after ACL injury and reconstruction surgery may be warranted (Ardern et al., 2011(Ardern et al., , 2014Langford et al., 2009;Tripp et al., 2007;Webster et al., 2008).
To evaluate various psychological factors associated with return to sports, several questionnaires have been used (Gobbi & Francisco, 2006;Kori, Miller, & Todd, 1990;Morrey, Stuart, Smith, & Wiese-Bjornstal, 1999;Thomeé et al., 2008). However, these scales were not developed specifically to evaluate the psychological impact of returning to sport after ACL injury or reconstruction.
Therefore, Webster et al. developed the ACL Return to Sport after Injury scale (ACL-RSI) that measures athletes' emotions, confidence in performance, and risk appraisal in relation to return to sport after ACL injury (Webster et al., 2008). The ACL-RSI has been shown to discriminate between athletes who return and athletes who do not return to sport after reconstruction surgery. The scale was recently validated in Swedish and French (Bohu, Klouche, Lefevre, Webster, & Herman, 2015;Kvist et al., 2013). In the Netherlands, there is need for a questionnaire that measures the psychological impact of returning to sport after ACL injury or reconstruction. Moreover, in general translated versions of the ACL-RSI in different languages are needed in order to be able to compare the results of different studies conducted in countries that speak different languages.
In order to be able to use the ACL-RSI by Dutch speaking patients, the aim of this study was to translate the ACL-RSI into Dutch and to study the validity and reliability of the ACL-RSI(NL).

Methods
The ACL-RSI was translated into Dutch according to international guidelines (Beaton, Bombardier, Guillemin, & Ferraz, 2000). Then, the translated version was tested for clinimetric properties. The local Medical Ethical Committee approved the procedures employed in this study.

Translation
The developers of the original ACL-RSI were informed and gave their consent to a Dutch translation of the ACL-RSI questionnaire (Webster, personal communication, 2013). The English language ACL-RSI was translated according to the method described by Beaton (Beaton et al., 2000). This method recognises 5 stages: (1) translation, (2) synthesis, (3) back translation, (4) expert committee review, and (5) pretesting. The expert committee consisting of 4 translators from stages 1 and 3, 2 human movement scientist/epidemiologist, and a sports physical therapist drafted the final version of the Dutch version of the ACL-RSI (ACL-RSI(NL)) (stage 4), which was pre-tested on 15 patients (4-12 months after ACL reconstruction).

Participants
Patients who had undergone ACL reconstruction at the department of Orthopaedics of the Martini Hospital Groningen or at the department of Orthopaedics or Trauma Surgery of the University Medical Center Groningen in the period from 1 January 2012 to 1 February 2013 were recruited. Patients were eligible for participation when they had undergone ACL reconstruction 3-16 months previous to the start of the study. This time interval was chosen because (1) patients start to perform sport-specific exercises and focus increasingly on return to the sport from about 3 months after ACL reconstruction, and (2) it was expected that 16 months after reconstruction the patients have returned in the sport. Patients who were unable to understand written Dutch were excluded.

Procedure
Eligible patients were sent 6 questionnaires in order to assess construct validity: the ACL-RSI(NL), the Multidimensional Health Locus of Control Scale (MHLC) (Wallston, Stein, & Smith, 1994), the Injury-Psychological Readiness to Return to Sport scale (I-PRRS) (Glazer, 2009), the Tampa Scale of Kinesiophobia (TSK) (Vlaeyen, Kole-Snijders, Boeren, & Van Eek, 1995), the Knee injury and Osteoarthritis Outcome Score (KOOS) (De Groot, Favejee, Reijman, Verhaar, & Terwee, 2008), and the International Knee Documentation Committee Score (IKDC) 2000 (Haverkamp, 2006). Questionnaires were sent by mail with an accompanying letter explaining the study and explaining that all data will be anonymised and that return of the questionnaire will be considered as consent to participate.
Patients were asked to fill in the questionnaires at home and to return them by mail. Patients who did not respond after 1 week were reminded.
Patients who underwent ACL reconstruction 3-4 months before the start of the study are not likely to be in a stable state since these patients are starting at that phase of the rehabilitation with sport-specific exercises and actual changes in status are very likely (Van Grinsven, Van Cingel, Holla, & Van Loon, 2010). Hence, we asked patients who underwent ACL reconstruction at least 5 months previous to the start of the study to complete the ACL-RSI(NL) questionnaire for a second time after an interval of 2 weeks to explore test-retest reliability. We confirmed our assumption by having these same patients provide a Global Rating of Change (GRC) score to quantify a change in the confidence regarding sports resumption during the previous 2 weeks. The GRC was scored on a 5point Likert scale ranging from much more confident (+2) to much less confident (−2). Participants who reported being much more confident or less confident on the GRC were excluded from our test-retest analysis.
Demographic characteristics were retrieved from the electronic patient records (gender, height, weight, age, and surgery date). To measure the current level of work and sports activities, patients were asked to complete the Tegner score in the first mailing.

Questionnaires
The ACL-RSI(NL) scale consists of 12 questions regarding the psychological impact of returning to sports in this population. The original ACL-RSI items were scored on a visual analogical scale (VAS) from 0 to 100. Upon the advice of Webster (Webster et al., 2008), we have used an adapted version of the ACL-RSI, which is also used in several follow-up studies (Bohu et al., 2015;Kvist et al., 2005Kvist et al., , 2013Langford et al., 2009). The items of the adapted ACL-RSI are identical to those of the original version, but the VAS is replaced by an 11-point Numeric Rating Scale (NRS) in the form of boxes to be ticked from 0 to 100. This version of the ACL-RSI was recently validated in the Swedish and French language (Bohu et al., 2015;Kvist et al., 2013) The I-PRRS was developed to assess an athlete's psychological readiness to return to sport participation after injury. The I-PRRS is a valid and reliable questionnaire and consists of 6 items that are scored on a 100-point scale (Glazer, 2009). The total score consists of the raw sum score of the 6 items, divided by 10. A high score (max 60) implies a higher confidence to return to sport. We translated the I-PRRS into Dutch following the international guidelines (Beaton et al., 2000).
The KOOS was developed to assess the patient's opinion about their knee and associated problems. The KOOS contains 42 questions and consists of 5 subscales; pain, other symptoms, function in daily living (ADL), function in sport and recreation (Sport/Rec), and knee-related quality of life (QOL). The KOOS has been validated in Dutch (De Groot et al., 2008). Questions were answered using a 5-point Likert scale. For each subscale, a sum score was calculated and converted into a 100-point scale. Higher scores reflect fewer symptoms and limitations.
The IKDC Subjective Knee Form 2000 is an 18-item instrument designed to measure pain, symptoms, function, and sports activity in patients with a variety of knee conditions (Haverkamp, 2006). A sum score was calculated, which is then transformed to 100-point scale, with 100 indicating no restrictions in daily and sports activities and the absence of symptoms. The IKDC 2000 was translated to Dutch and validated for a heterogeneous population of patients with knee complaints.
The TSK measures fear for re-injury due to movement and physical activity. It contains 17 items scored on a 4-point Likert scale regarding the subjective experience of the injury and physical activity. The sum of the items results in a score between 17-68, where 68 indicates a high level of fear (Kori et al., 1990). There is a Dutch version of the TSK available (Vlaeyen et al., 1995). For this study, the modified Dutch version, adapted for knee injuries was used.
The MHLC measures different dimensions of the Health Locus of Control (HLC), that is, persons' believe whether their health is determined by their behaviour or external events (Halfens & Philipsen, 1988;Wallston, Wallston, Kaplan, & Maides,1976). Forms A and B are general health locus of control scales. Form C is used in place of Form A/B when studying people with an existing medical or healthrelated problem (Wallston et al., 1994). The MHLC-C consists of 4 subsections; Internal, Chance, Doctors, and Other (powerful) People. The internal HLC subsection assesses the extent to which one believes that internal factors are responsible for health/illness. Persons with strong internal HLC believe that the outcome is directly a result of their own behaviour or action. In this validation study, only the internal HLC subsection is used. We translated the MHLC-C into Dutch following the international guidelines (Beaton et al., 2000). For the current study, the word "condition" was replaced by "knee problems".

Construct validity
Validity of the ACL-RSI(NL) was expressed in terms of construct validity. Construct validity refers to the extent to which scores on a particular measure relate to other measures, consistent with theoretically derived hypotheses concerning the constructs that are being measured (Terwee et al., 2007). Construct validity of the ACL-RSI(NL) was determined by evaluating its structural validity and by hypothesis testing. Structural validity was assessed to determine whether the ACL-RSI(NL) is also 1 dimensional (i.e., it does not contain subscales), like the original ACL-RSI. Additionally, according to the guidelines proposed by the COSMIN initiative (Mokkink et al., 2010), we formulated 10 hypotheses about the magnitude of relationships between the ACL-RSI(NL) and the corresponding (sub)scales of the I-PRRS, KOOS, IKDC 2000, TSK, and MHLC (Table 4). According to the COSMIN guidelines, the construct validity of the ACL-RSI(NL) is considered good if ≥75% of the predefined hypotheses are confirmed (Terwee et al., 2007).

Reliability
According to the COSMIN guidelines, reliability was assessed in terms of internal consistency, test-retest reliability, and measurement error (Mokkink et al., 2010). Reliability refers to the extent to which an instrument consistently yields the same scores on 2 successive occasions in stable persons and the extent to which patients can be distinguished from each other despite measurement errors. Internal consistency is a measure of the extent to which items in a questionnaire's (sub)scale are correlated (homogeneous), thus measuring the same concept (Terwee et al., 2007). Test-retest reliability concerns the extent to which scores of patients are the same for repeated measurements (Mokkink et al., 2010). The Bland and Altman method was used to explore repeatability, which reflects the amount of agreement in repeated measurements (Bland & Altman, 1986). Additionally, measurement error was investigated; measurement error is a measure of systematic error of a patient's score what is not caused by actual changes in the measured construct (Mokkink et al., 2010).

Floor and ceiling effects
The presence of floor and ceiling effects may jeopardise the validity and reliability of a questionnaire instrument. It is then likely that extreme items are missing in the lower or upper ends of the questionnaire (Terwee et al., 2007). Floor and ceiling effects are defined as 15% of the participants achieving the minimum or maximum score, respectively (McHorney & Tarlov, 1995).

Statistical analysis
A sample size of at least 100 is considered an adequate sample size for studies regarding measurement properties of questionnaires, and a sample size of 50 is considered adequate for determining test-retest reliability (Terwee et al., 2012). Hence, we planned a sample size of at least 100 participants for assessing the construct validity of the ACL-RSI(NL), and a sample size of at least 50 for establishing the test-retest reliability of the ACL-RSI(NL).

Construct validity
To assess the structural validity of the ACL-RSI(NL), exploratory factor analysis was conducted on all ACL-RSI(NL)items using principal component analyses (PCA) with varimax rotation. Factor analyses with 1-, 2-, 3-factor solution were performed. The Spearman's Rho was calculated between the ACL-RSI(NL) and the other questionnaires. The Spearman's Rho was interpreted according to Hinkle (Hinkle, Wiersma, & Jurs, 1998). Correlation coefficients above 0.6, 0.6-0.3, and less than 0.3 are considered to be high, moderate, and low, respectively.

Reliability
Internal consistency of the ACL-RSI(NL) was assessed with Cronbach's alpha using the data from the first administration of the ACL-RSI(NL). It is widely accepted that Cronbach's alpha should be between 0.70 and 0.95 (Terwee et al., 2007). To determine test-retest reliability of the ACL-RSI(NL), the intraclass correlation coefficient (ICC: 2-way random, type agreement) with corresponding 95% CIs were calculated between the first and second administration of the ACL-RSI(NL). The ICC is generally considered to be good at 0.70 and above (Terwee et al., 2007).
Additionally, a Bland and Altman analysis was performed; the mean difference between the first and second administration of the ACL-RSI(NL) with a 95% CI was calculated. When zero is lying within the 95% CI of the mean difference, it can be seen as a criterion for absolute agreement. When zero lies outside the 95% CI, a bias in the measurements is indicated (Bland & Altman, 1986) The standard error of measurement (SEM), a measure of the instrument's absolute measurement error, was calculated. The value of the SEM can be derived by dividing the SD of the mean differences between 2 measurements (SDdiff) by √2 (de Vet, Terwee, Knol, & Bouter, 2006). The smallest detectable change (SDC) for the individual score and for the group was calculated according to Beaton (SDCind = 1,96x√2xSEM; SDCgroep = SDCind/√n) (Beaton et al., 2000).
All statistical analyses were performed using PASW statistical package (version 20, SPSS Inc, Chicago). A p-value of <0.05 indicated statistical significance. Descriptive statistics were used for patient characteristics and to display outcomes of questionnaires.

Translation
The ACL-RSI was successfully translated into Dutch (ACL-RSI (NL)) according to guidelines (see Appendix 1). No changes were made after pretesting the ACL-RSI(NL); all questions were clear. A detailed report of the translation process can be provided by the authors.

Participants
In total 241 patients were approached. Of these, 150 patients returned the first questionnaires response rate 62%). Two weeks after receiving the first questionnaires, all 122 respondents who were treated with an ACL reconstruction 5-16 months prior to the second questionnaire start of the study, were asked to complete the ACL-RSI(NL) once more. The second questionnaire was returned by 107 patients (response rate 88%). The mean time between receiving the first and second questionnaire was 22.9 days (SD 7.9). A flow diagram of inclusion of participants is shown in Figure 1. Demographic characteristics of the participants are shown in Table 1.

Description of the results
An overview of the scores on the various questionnaires is shown in Table 2.

Construct validity
The factor analysis showed a 1-factor structure of the ACL-RSI (NL), with an explained variance of 59% and an eigenvalue of 7.1. The Spearman correlations between the ACL-RSI(NL) and the I-PRRS, the subscales of the KOOS, IKDC 2000, TSK, and MHLC-C are shown in Table 3.
The ACL-RSI(NL) showed a high correlation with the I-PRRS (r = 0.79) and a moderate correlation with the 4 subscales of the KOOS (r = 0.30-0.48). Only the subscale "activities of daily living" has a low correlation with the ACL-RSI(NL) (r = 0.25). The KOOS subscales "sport and function" (r = 0.48) and "knee-related quality of life" (r = 0.40) had higher correlations than the other KOOS subscales. Moderate correlation was found with the TSK (r = −0.46) and a low correlation with the MHLC-C (r = −0.15). Of the predefined hypotheses to determine construct validity, 9 of the 10 were confirmed (90%) ( Table 4). Only the hypothesis regarding the correlation between the ACL-RSI(NL) and the KOOS subscale "function in daily living" was rejected, since a correlation of 0.25 instead of 0.30-0.60 was found.

Reliability
The Cronbach's alpha of ACL-RSI(NL) was 0.94, which indicates good internal consistency. Of the 107 participants who returned the second administration of the ACL-RSI(NL), 74 (49.3%) participants indicated that no change had occurred in the expectations regarding sport resumption in the 2 weeks prior to completion of the retest. Seven participants (4.7%) reported less confidence, 23 (15.3%) more confidence, and 3 participants (2%) reported to have much more confidence. These 3 participants were excluded from the test-retest reliability analysis. The test-retest reliability analyses are presented in Table 5. The mean score for the ACL-RSI(NL) was 56.1 (SD 22.3) at the first measurement and 59.3 (SD 21.9) at the second measurement. The mean difference was 3.2 (95% CI 1.6-4.6; SD 7.8), with the 95% CI not containing zero, indicating a systematic bias (Figure 2). The ICC was 0.93 (95% CI 0.88-0.96), indicating excellent test-retest reliability. SEM was 5.5; SDCind was 15.3, and SDCgroup 1.5.

Floor and ceiling effects
The mean total score for the ACL-RSI was 56.5 (range 8.3-100, SD 22.2). No significant floor and ceiling effects were found: 0.7% of participants scored below 10 and 10.7% of participants scored 90 or higher.

Discussion
The aim of this study was to translate the ACL-RSI scale from English to Dutch and to examine whether the Dutch version of the ACL-RSI (ACL-RSI(NL)) is a valid and reliable instrument to investigate psychological factors relevant to returning to sport after ACL reconstruction. The ACL-RSI(NL) showed good validity and reliability.

Construct validity
In accordance with the English and Swedish versions, the factor analysis revealed that the ACL-RSI(NL) primarily evaluates 1 dimension (Kvist et al., 2013;Webster et al., 2008). Therefore, the 3 psychological responses that are included in the ACL-RSI, that is, emotions, confidence in performance, and   risk appraisal cannot be separated and 1 total score should be used. Construct validity was examined by testing pre-defined hypotheses. Except for the correlation between the ACL-RSI (NL) and the KOOS subscale "activities of daily living", all hypotheses described were confirmed (90%). As this was above the 75% criterion set in the COSMIN guidelines, it can be concluded that the ACL-RSI(NL) has good construct validity Terwee et al., 2007).
As expected, we found a high correlation between the ACL-RSI(NL) and the I-PRRS. Both questionnaires measure a comparable construct, namely the psychological impact of returning to sport after ACL injury or reconstruction and athlete's psychological readiness to return to sport participation after injury respectively. However, the ACL-RSI is specifically developed for the ACL patient while the I-PRRS developed for injured athletes in general. The different psychological responses measured in the ACL-RSI, such as emotions, confidence in performance, and risk appraisal receive less attention in the I-PRRS. To our knowledge, the relationship between the I-PRRS and the ACL-RSI has not been investigated previously.
Moderate correlations were found between the ACL-RSI(NL) and the subscales of the KOOS. Only the subscale "activities of daily living" had a low correlation with the ACL-RSI(NL). The results of this study with regard to the degree of correlation of the ACL-RSI(NL) with the KOOS are comparable to those of the study of Kvist et al. (2013), who also found moderate correlations. In accordance with Kvist et al. (2013), the ACL-RSI(NL) showed higher correlations with KOOS subscales "sport and function" and "knee-related quality of life" than with the other KOOS subscales. Overall, Kvist et al. found slightly higher correlations between the ACL-RSI and these KOOS subscales compared with this study, which may be explained by the fact that Kvist et al. included participants 2-5 year after ACL reconstruction. We hypothesise that these patients, for those who practice sports, good quality of life is strongly associated with return to sports after ACL reconstruction. A significant part of the participants of our study (3-16 months after reconstruction), still participate in a rehabilitation program after ACL reconstruction. This might have led to less strong correlations.
Probably, the quality of life is partly determined by the possibility of being able to do sports. Possibly, patients still participating in a rehabilitation program, already indicate a higher quality of life, while they still score low on the ACL-RSI. The hypothesis that these 2 subscales had a stronger correlation with ACL-RSI(NL) than the other 3 subscales, within the moderate range, was confirmed.
The ACL-RSI(NL) correlated moderately with the IKDC 2000. This is in line with previous research into the validity of the French version of the ACL-RSI. The IKDC 2000 partially measures aspects, which are related to psychological factors relevant to returning to sport, but are primary developed to measure other constructs, such as symptoms, limitations in function, and sports due to impairment of the knee. Since  1 A high correlation between the ACL-RSI(NL) and the I-PRRS(NL) (r > 0.6).   the IKDC 2000 has similarities with the KOOS, hence the moderate correlation can also be explained from that point of view. The moderate correlation found between the ACL-RSI and the TSK was comparable with a foregoing study (Kvist et al., 2013). Previously, the TSK is used to assess the fear of re-injury in relation to predicting return to sport in recreational athletes with anterior cruciate ligament injuries (Kvist et al., 2005;Tripp et al., 2007). Fear of re-injury has been shown to be a hindrance for return to sports. A higher score on the TSK has previous been associated with not returning to sports after ACL injury (Kvist et al., 2005). The TSK has not been tested for responsiveness in patients with ACL injuries or after ACL reconstruction. It seems that the underlying construct of the TSK does not correspond entirely with the construct of the ACL-RSI. Beside fear of re-injury, the ACL-RSI evaluates other emotions (such as nervousness, frustration, and stress) and confidence in performance and risk appraisal in relation to return to sport after ACL reconstruction. Additionally, as expected and in line with previous research, the correlation between the ACL-RSI(NL) and the MHLC-C was low (Kvist et al., 2013).

Reliability
The ACL-RSI(NL) demonstrated good internal consistency. The magnitude of the Cronbach's alpha (0.94) was comparable with the Cronbach's alpha of the original ACL-RSI (0.92) and the Swedish (0.94) and French versions (0.96) (Bohu et al., 2015;Kvist et al., 2005;Webster et al., 2008). The test-retest reliability of the ACL-RSI(NL) was high (ICC: 0.93), which corresponds with the Swedish (ICC: 0.89) and French (ICC:0.90) versions (Bohu et al., 2015;Kvist et al., 2013). The Bland and Altman analysis showed a significant bias of 3.2 points between the first and second administration of the ACL-RSI (NL). However, this bias is less than the SEM (5.5). Therefore, it cannot be distinguished from the measurement error. The SEM (5.5) and SDCind (15.3) found in this study are in line with those found in the Swedish ACL-RSI study (SEM 7.0 en SDCind 19) (Kvist et al., 2013).
It should be noted that the time period between the 2 measurements of the ACL-RSI(NL) may have affected the results. The mean time between receiving the first and second questionnaire was 22.9 days (SD 7.9). The COSMIN recommendation is that the time interval should be long enough to prevent recall bias and short enough to ensure that patient characteristics have not changed regarding the construct to be measured (Mokkink et al., 2010). A time interval of 2 weeks is often used in repeatability studies (Streiner & Norman, 2008). To check whether the patients were not changed with regard to the construct measured by the ACL-RSI during the period between test and retest, a global rating of change question was included in the second measurement (Kamper, Maher, & Mackay, 2009). The 3 participants who indicated that they received much more confidence during the time interval between the first and second measurement were not included in the analysis.
In this study, no floor or ceiling effects were observed, since less than 15% of the respondents had achieved the lowest or highest possible total score. Like the original English, Swedish and French version the mean score is somewhere in the middle of the scale (56.5) and varies from 8-100. Also in these studies, no floor or ceiling effects were found.

Limitations and strengths
A limitation of this study is that there is no information available with regard to validity and reliability of the Dutch language version of the I-PRRS and the MHLC-C.
The validity and reliability of the original questionnaires has been demonstrated though and both are translated following the international guidelines (Beaton et al., 2000;Glazer, 2009;Wallston et al., 1994).
A strength of this study is the study population used. In contrast to the English and Swedish ACL-RSI studies, using a population being a considerable period of time after ACL reconstruction, the current study was performed in a patient population partly still participating in a rehabilitation program after ACL reconstruction. We have specifically chosen this patient group because in these patients the return to sport is an issue. Moreover, if we want to implement interventions to modify psychological factors based on the outcomes of the ACL-RSI, these will take place in this phase of the rehabilitation.

Conclusion
The ACL-RSI is successfully translated into Dutch (ACL-RSI(NL)), and the ACL-RSI(NL) is a valid and reliable questionnaire to investigate athletes' emotions, confidence in performance, and risk appraisal in relation to return to sport after ACL reconstruction.

Disclosure statement
No potential conflict of interest was reported by the authors.