Test-retest reliability and agreement of the SPI-Questionnaire to detect symptoms of digital ischemia in elite volleyball players

ABSTRACT The Shoulder posterior circumflex humeral artery Pathology and digital Ischemia – questionnaire (SPI-Q) has been developed to enable periodic surveillance of elite volleyball players, who are at risk for digital ischemia. Prior to implementation, assessing reliability is mandatory. Therefore, the test-retest reliability and agreement of the SPI-Q were evaluated among the population at risk. A questionnaire survey was performed with a 2-week interval among 65 elite male volleyball players assessing symptoms of cold, pale and blue digits in the dominant hand during or after practice or competition using a 4-point Likert scale (never, sometimes, often and always). Kappa (κ) and percentage of agreement (POA) were calculated for individual symptoms, and to distinguish symptomatic and asymptomatic players. For the individual symptoms, κ ranged from “poor” (0.25) to “good” (0.63), and POA ranged from “moderate” (78%) to “good” (97%). To classify symptomatic players, the SPI-Q showed “good” reliability (κ = 0.83; 95%CI 0.69–0.97) and “good” agreement (POA = 92%). The current study has proven the SPI-Q to be reliable for detecting elite male indoor volleyball players with symptoms of digital ischemia.

For this purpose, the Shoulder posterior circumflex humeral artery Pathology and digital Ischemiaquestionnaire (SPI-Q) was developed using medical reports of volleyball players with confirmed digital ischemia combined with medical literature on symptoms associated with digital ischemia (Van de Pol et al., 2012). To our knowledge, no disease-specific questionnaires exist for symptoms of digital ischemia in the upper limb.
Before the SPI-Q can be used for periodic surveillance, its measurement properties should be tested. Therefore, the purpose of this study is to evaluate the test-retest reliability and agreement of the SPI-Q for detection of symptoms of digital ischemia in the population at risk: elite male volleyball players.

Study design
A prospective cohort study was performed among elite male volleyball players in the Netherlands. Official approval of this study was waived by the Institutional Review Board (IRB) of our academic hospital.

Participant selection
A power analysis in nQuery advisor 7.0 (Statistical Solutions Ltd, 2007) showed that 62 participants were needed with kappa = 0.7, precision = 0.15 and a one-sided confidence level of α = 0.05. The inclusion criteria were: (1) being an elite male indoor volleyball player active in the Dutch national top or second league in the seasons 2013-2014; and (2) written informed consent. To secure sufficient variety in players with and without symptoms of digital ischemia, 84 volleyball players who reported at least one symptom of cold, blue or pale digits in the dominant hand during or directly after practice or competition were selected from a previous study, and supplemented with volleyball players who reported no symptoms (n = 22) ( Van de Pol et al., 2016).

Study protocol
An electronic questionnaire was sent through a digital e-mail link with a 2-week interval between test (t = 1) and retest (t = 2). This time interval was chosen to prevent both recall bias and change in health condition/symptoms (Terwee et al., 2007). To assure no bias was caused by a change in the prevalence of symptoms between t = 1 and t = 2, for each of the six symptoms the following question and answer categories were formulated in the retest questionnaire: "Does this symptom occurs less often, equally often or more often than 14 days ago when you filled in the first questionnaire?" If a participant answered one of these six questions with "less often" or "more often", the participant was excluded from the analysis.

Questionnaire content
The SPI-Q is a questionnaire specifically developed to detect ischemic symptoms among elite volleyball players. This questionnaire was developed by Van de Pol et al. using medical files and case reports of volleyball players with confirmed digital ischemia due to PCHA pathology, and was also used among professional indoor volleyball players (Van de Pol, Alaeikhanehshir, Maas, & Kuijer, 2015;Van de Pol et al., 2012;Van de Pol, Kuijer, Langenhorst, & Maas, 2014). In the SPI-Q, symptoms of cold, blue and pale digits in the dominant hand during or immediately after practice or competition were considered to be associated with digital ischemia based on the following arguments ( Van de Pol et al., 2012): (1) These symptoms were the only symptoms that were reported as present in at least 50% of the volleyball players with confirmed digital ischemia from the medical literature and/or medical files.
(2) Pain in the digits of the dominant hand was excluded because it is hard to distinguish between ischemic pain and pain resulting from trauma. (3) Jackson (2003) stated that the signs of ischemia that typically distinguish arterial injury from musculo-skeletal injury are those of changes in temperature and colour (Jackson, 2003).
Additionally, these symptoms had to occur during or immediately after practice or competition because symptoms are most likely to occur at high levels of exercise, when during the spiking motion in volleyball, thrombi can be extruded from the aneurysmal PCHA into the axillary artery and embolize to the circulation of the forearm, hand and digits (Durham, Yao, Pearce, Nuber, & McCarthy, 1995;Jackson, 2003;McCarthy et al., 1989).
The SPI-Q comprises six questions concerning symptoms of digital ischemia in the dominant hand, like cold, blue or pale digits, occurring both during and directly after practice as well as in competition (Table 1). A 4-point Likert scale was used for the answer categories of the questions on symptoms: "never", "sometimes", "often" and "always".

Data analyses
The data from the questionnaires were entered in SPSS (version 21.0, 2012, SPSS Inc.) and randomly checked for correct entry. The mean, standard deviation, maximum and minimum of age, body weight, body height, total years playing volleyball and total hours volleyball played in the last 14 days were reported for the group as a whole.
Test-retest reliability and agreement were calculated and expressed in the following ways: (1) Linear weighted kappa (κ) was calculated for each of the individual symptoms of digital ischemia, i.e., cold, blue or pale digits during and after practice and competition. Since even subtle symptoms, like sometimes a pale finger during volleyball, can be a first sign of these emboli in this young and healthy population, we decided to categorise every player who reported one or more symptoms as symptomatic. Κ is the recommended parameter for the estimation of reliability for categorical data (Kottner et al., 2011;Mokkink et al., 2010), and was calculated using a website (Kappa as a Measure of Concordance in Categorical Sorting, n.d.). The linear weights for the weighting matrix were calculated using the following formula: 1 -((ij)/(k -1)), where i is the category rated in test 1, j the category rated in test 2 and k the total number of categories (Sim & Wright, 2005;Vanbelle, 2014). To reflect sampling error, the confidence interval (CI) of the linear weighted κ was also calculated. The following decision criterion was formulated for the interpretation of the values of κ: >0.60 = "good", 0.41-0.60 = "moderate" and ≤0.40 = "poor" (Landis & Koch, 1977). To assess the dependence of κ from the distribution of data, the percentage of agreement (POA) of the measurements classified in the same categories in the test and retest questionnaires was calculated for each of the individual symptoms of digital ischemia, i.e. cold, blue or pale digits during and after practice and competition (Brennan & Hays, 1992 individual symptoms of digital ischemia, i.e. cold, blue or pale digits during and after practice and competition were combined in order to label a volleyball player as symptomatic or asymptomatic. The case definition of symptomatic was: a volleyball player who sometimes, often or always reported one or more of the symptoms cold or blue or pale digits during or after volleyball. Volleyball players who experienced no symptoms at all (e.g., six times "no never") were defined as asymptomatic. The values of the κ were interpreted in the same manner as described above for the linear weighted κ.

Participants
From March to May 2015, a total of 106 volleyball players were invited to participate, of whom 73 agreed to participate and 71 of these 73 completed the questionnaire both times, a response rate of 97%. Six volleyball players were excluded because they reported a change in the frequency of symptoms in the 2 weeks between completing both questionnaires: five volleyball players reported symptoms less often, and one reported symptoms more often. These participants were advised about follow-up. As a result, 65 volleyball players, completing both questionnaires in 15 ± 4 days (range 7-31 days), were included ( Figure 1). On an average, volleyball players were 27 ± 5 years old (range: 19-42 years), had a body height of 195 ± 7 centimetres (cm) (range: 175-207 cm), had a body weight of 88 ± 9 kilograms centimetres (kg) (range: 70-110) and had been playing volleyball for 17 ± 6 years in total (range 6-33 years) and 15 ± 8 hours in the last 14 days (range 0-30 hours).

Prevalence of symptoms of digital ischemia
Cold and pale digits during practice or competition were most prevalent with percentages ranging from 20% to 26%. Blue digits directly after playing volleyball were rarely reported (3%) and none of the volleyball players reported a symptom as always being present (Table 2). During test (t = 1) and retest (t = 2), respectively 37% (n = 24) and 35% (n = 23) of surveyed volleyball players reported at least one of the symptoms of cold or blue or pale during or after volleyball.

Individual symptoms of digital ischemia
Linear weighted κ for the individual symptoms of digital ischemia ranged from "poor" (0.25) for pale digits after practice or competition to "good" for cold digits during practice or competition (0.63). The POA for these individual symptoms ranged from "moderate" (78%) for pale digits during competition to "good" (97%) for blue digits after competition (Table 3).

Symptomatic for digital ischemia
The unweighted κ calculated for the combined score of the six questions to determine whether a volleyball player is symptomatic based on one or more symptoms of digital ischemia was "good" (0.83) ( Table 3).

Discussion
This study shows that the reliability and agreement of the SPI-Q is sufficient for assessing cold and blue digits, and to distinguish symptomatic volleyball players from asymptomatic ones based on one or more symptoms. Only assessing pale digits showed "poor" to "moderate" results for reliability and moderate results for agreement. This is the first study to provide a reliable tool that can assess symptoms of digital ischemia in elite volleyball players: the population at risk. To our knowledge, no comparable questionnaires have been developed, nor tested, for upper limb ischemia in this or any other population of elite overhead athletes at risk, such as baseball pitchers (Duwayri et al., 2011). For lower limb ischemia, questionnaires like the VascuQol have been developed and extensively tested for their measurement properties (Conijn, Loukachov, Bipat, & Koelemay, 2015;F.A. Frans et al., 2014). However, these questionnaires are mostly used for QoL outcomes of patients diagnosed with or treated forchronic critical lower limb ischemia (Morgan, Crayford, Murrin, & Fraser, 2001), while the SPI-Q has been developed for targeted detection and monitoring of symptoms of digital ischemia in healthy elite overhead athletes.
To classify symptomatic players based on one or more symptoms, the SPI-Q showed "good" results for both reliability and agreement. This indicates that the SPI-Q is a reliable instrument for detecting symptomatic volleyball players. In addition, the outcomes for the individual symptoms of digital ischemia were diverse: the linear weighted κ ranged from 0.25 to 0.63 ("poor" to "good" reliability), and the POA values ranged from 78% to 97% ("moderate" to "good"), making these questions seem less suited for detecting elite male volleyball players with symptoms when used individually. This seems especially true for detecting pale digits.

Strengths and weaknesses
A strength of this study was the power analysis via nQuery advisor 7.0 and the inclusion of 65 volleyball players, since a sample size of at least 62 was considered adequate. Another strength is that all symptomatic players from a previous study were selected to secure sufficient variety in the main outcome "presence and absence of symptoms". Little variety would have caused a poor level of reliability, as found in other studies (Vincent Gouttebarge, Plat, Kuijer, Sluiter, & Frings-Dresen, 2015). A possible weakness of the SPI-Q is that none of the participants reported experiencing one of the symptoms as "always". Therefore, one might consider modifying the answer categories "often" and "always" to "regularly" and "often", respectively. On the other hand, if the SPI-Q is to be used in a population of volleyball players with much more severe symptoms, the "always" category might be needed. Of course, the effects of changes on reliability of reporting of individual symptoms should be assessed in future studies.

Clinical implications
The self-reported prevalence of cold or blue or pale digits in the dominant hand during or immediately after practice or competition is 38% among international world-class male and female beach volleyball players, and 31% among elite male indoor volleyball players (Van de Pol et al., 2012, 2015. These symptoms are associated with digital ischemia, which can be caused by a wide variety of vascular and non-vascular pathologies (Van de Pol et al., 2014), including arterial emboli originating from an aneurysmal and thrombosed PCHA in the dominant shoulder (Van de Pol et al., 2012).
Volleyball players with confirmed digital ischemia present themselves late in disease with severely disabling coldness, discolouration, pain and paraesthesia in the dominant hand ( Van de Pol et al., 2016). Active surveillance enables identification of apparently innocuous symptoms, like coldness and discolouration, at an early stage, and might prevent thromboembolic complications, irreversible tissue damage and possibly surgical ligation of the PCHA ( Van de Pol et al., 2012). The exact correlation between symptoms of digital ischemia and PCHA pathology in elite volleyball players has yet to be determined in future research. Moreover, the responsiveness of the SPI-Q needs to  Table 2. Prevalence of symptoms (number and %) of digital ischemia during or directly after practice or competition in the dominant hand among elite male volleyball players during test (t = 1) and retest (t = 2) (n = 65 be determined in future studies. The current study has proven the SPI-Q to be reliable for detecting elite male indoor volleyball players with symptoms of digital ischemia. However, before the SPI-Q can be used for periodic health surveillance to detect and monitor elite volleyball players with symptoms of digital ischemia, other clinimetric properties, such as for instance the responsiveness, have to be assessed.

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
No funding was received for this study to be conducted.