Measuring psychological trauma after spinal cord injury: Development and psychometric characteristics of the SCI-QOL Psychological Trauma item bank and short form

Objective To describe the development and psychometric properties of the SCI-QOL Psychological Trauma item bank and short form. Design Using a mixed-methods design, we developed and tested a Psychological Trauma item bank with patient and provider focus groups, cognitive interviews, and item response theory based analytic approaches, including tests of model fit, differential item functioning (DIF) and precision. Setting We tested a 31-item pool at several medical institutions across the United States, including the University of Michigan, Kessler Foundation, Rehabilitation Institute of Chicago, the University of Washington, Craig Hospital and the James J. Peters/Bronx Veterans Administration hospital. Participants A total of 716 individuals with SCI completed the trauma items Results The 31 items fit a unidimensional model (CFI=0.952; RMSEA=0.061) and demonstrated good precision (theta range between 0.6 and 2.5). Nine items demonstrated negligible DIF with little impact on score estimates. The final calibrated item bank contains 19 items Conclusion The SCI-QOL Psychological Trauma item bank is a psychometrically robust measurement tool from which a short form and a computer adaptive test (CAT) version are available.


Introduction
Most traumatic spinal cord injuries (SCIs) result from potentially life-threatening events, including motor vehicle crashes, followed by acts of violence (e.g. gunshot wound), falls, and diving. 1,2 Given the traumatic nature of SCI, including the event causing the injury, paralysis and loss of sensation, and sometimes witnessing another person's death or injury, increased symptoms of acute and post-traumatic stress are a not unusual consequence. The Diagnostic and Statistical Manual of Mental Disorders, Fifth Ed. (DSM-5) identifies individuals with SCI as being at high risk for post-traumatic stress disorder (PTSD). 3 The DSM-5 criteria for PTSD require a history of exposure to a traumatic event that involves the perceived or actual threat of death or serious bodily injury to oneself or others (Criterion A). PTSD is characterized by intrusive memories of the event through flashbacks, nightmares, or frightening thoughts (Criterion B); persistent affective and behavioral avoidant and numbing responses (e.g. avoiding thoughts, feelings, activities, locations that are associated with the event) (Criterion C); negative alterations in cognition or mood not previously experienced (Criterion D); and manifestations of hyper-arousal (e.g. difficulty falling asleep, irritability, reckless behavior) (Criterion E). A PTSD diagnosis requires this constellation of symptoms to develop within one week of the event, persist over one month (Criterion F), and cause significant emotional distress and functional impairment (Criterion G). 3 Prevalence estimates of PTSD following SCI vary widely. Kennedy and Duff's review found that posttraumatic stress is common, ranging from 10-40%. 4 Two Danish studies reported PTSD prevalence of 7-20% in a sample of 69 adults with paraplegia and tetraplegia averaging 19 years since injury. 5,6 In a study of adults (n = 85) in the United Kingdom whose injuries were more recent (≤6 months), 14% demonstrated high levels of posttraumatic stress. 7 Among 125 American veterans with SCI, Radnitz et al. reported that 12% met the diagnosis criteria for PTSD at the time of assessment and 29% met the criteria at some point postinjury. 8 Within pediatric-onset SCI samples, studies report PTSD rates ranging from 25% 9,10 to 33%. 11 Conversely, other research suggests a SCI-PTSD prevalence that is comparable to the general population (6% to 8%). [12][13][14] Variability in samples, methods, and measures likely accounts for this variation.
Pre-injury disposition, trauma-related factors, and emotional processing of the traumatic event may contribute to PTSD symptoms. [15][16][17] These factors can be grouped into pre-injury, peri-injury and post-injury factors. 18 Pre-injury factors, such as sociodemographic, physiological, and psychological characteristics may contribute to how individuals react to SCI. Younger age, 6,7 female sex, 7 single marital status, 5,7 more education, 19 and greater trauma exposure 17,20 before SCI may predispose and influence responses to trauma. Individuals whose trauma is characterized by greater violence (e.g. gunshot, interpersonal violence) exhibit higher PTSD prevalence than those who were injured in motor vehicle crashes. 11 Post-traumatic factors that contribute to PTSD symptoms include negative cognitions and dispositional factors, time since injury, 16,21 and self-attributions of responsibility for injury. 22 There may be a paradoxical association between injury level and PTSD vulnerability, with higher PTSD incidence in individuals with paraplegia rather than tetraplegia. 8,16 There may be a curvilinear relationship between SCI disability and stress level, such that those with the most and least disability report the highest levels of stress. 17 PTSD symptom severity typically attenuates with time. 2,16 The psychological consequences of trauma have important clinical implications. PTSD symptoms are often persistent if left untreated and are associated with an elevated risk of morbidity. PTSD symptoms are correlated with somatic problems, anxiety, depression, 7,12,16,23 social isolation, 24 substance abuse, and suicide. 25 Individuals with SCI may be at high risk for mental illness because of the negative consequences of SCI on physical and social functioning; [26][27][28] in fact, elevated rates of mental illness have been observed in individuals with SCI. 29 Identifying risk factors for PTSD or subclinical manifestations of pervasive psychological consequences of trauma is important to prevent, monitor, and treat PTSD.
A variety of assessment tools have been developed to screen for PTSD, to assess symptom severity, and diagnose the disorder. The Clinician-Administered PTSD Scale 30 and the Structured Clinical Interview for DSM-IV PTSD module 31 are regarded as the gold standards for diagnosis. 32,33 Use of these measures is limited in clinical and research settings due to their length and training requirements. Del Vecchio et al. published a comprehensive review of peer-reviewed, self-report PTSD assessment tools. 32 The research team identified 41 English-language instruments, including the Harvard Trauma Questionnaire (40 items), 34 Impact of Event Scale-Revised (22 items), 35 PTSD checklist (17 items), 36 and the Trauma Screening Questionnaire (10 items). 37 The lack of a common measurement metric makes it difficult to compare prevalence data and treatment outcomes across studies. The complex nature of psychological trauma experienced by persons with SCI creates a need for a self-report instrument with content specificity and sensitivity to change.
Computer adaptive testing (CAT) allows administration of brief instruments that are reliable and sensitive. CAT relies on item response theory (IRT) methods in which a computer algorithm sequentially selects a small number of items that maximizes test information and can discriminate symptom severity over time and between individuals. 38 As part of a larger project that developed IRT-based quality of life instruments for persons with SCI, we developed an item bank to detect and track psychological trauma in patients with SCI.
As described by Tulsky et al. in the introductory paper of this special issue, 39 our group has worked to unify diverse aspects of quality of life following traumatic SCI using the Patient-Reported Outcomes Measurement Information System (PROMIS) 40 and Neuro-QOL 41 approach. The purpose of this paper is to describe the development, psychometric analysis, and validation of the SCI-QOL Psychological Trauma item banks and short form.

Development of a Psychological Trauma item pool
We began by locating and classifying items from focus groups conducted with SCI patients and providers. 43 Using the transcripts, we wrote 9 items, which we reviewed and modified as a group (authors DV, DT, PK). For example, for the patient statement, 'The other day we saw a huge accident right in front of me, and my own accident flashed before my eyes…it really freaked me out and I couldn't get in a car for a while after that,' we wrote items such as 'I avoided reminders of how my injury occurred' and 'I was nervous when something reminded me of the accident. ' We used another patient's statement, 'It's a feeling like being trap-ped…a prisoner in your own body' to write the item, 'I felt trapped.' One of the project investigators (author DV) had recently developed a measure of injury distress, which included a relevant post-traumatic stress scale. With minor modifications (e.g. changing 'accident' to 'injury,' and tense, i.e. present to past tense), we were able to include 16 Injury Distress Index 44 post-traumatic stress items.
From this initial set of 25 Psychological Trauma items, we removed 6 redundant items and wrote an additional 12 items to fill gaps in what we anticipated would be the item hierarchy. Next, we completed cognitive interviews with 5 individuals with SCI. 45 They read and answered each item, then described the thought process they used to answer the statement. We asked them to report any problems understanding the items. No items needed to be removed or modified based on cognitive interviews. Then, we reviewed items for translatability 46 and reading level. 47 We modified 4 items following the translatability review. For example, we modified 'upsetting thoughts about my injury popped into my mind' to 'I had upsetting thoughts about the event of my injury.' All items were written at or below a 5 th grade reading level.

Calibration study participants and data collection procedures
We administered the 31-item item pool as part of a multisite calibration study (sites included the University of Michigan, Kessler Foundation, Rehabilitation Institute of Chicago, the University of Washington, Craig Hospital and the James J. Peters/Bronx Veterans Administration hospital), in which other item pools of health-related quality of life (HRQOL) items were also administered. The calibration sample was comprised of 716 individuals with traumatic SCI, all of whom were older than 18 years of age and could read and understand English. Injury level and severity (i.e. most recent American Spinal Injury Association Impairment Scale rating) were documented through medical record review. To ensure heterogeneity, the sample was stratified by diagnosis ( paraplegia versus tetraplegia), severity (complete injury vs. incomplete injury), and time since injury (<1 year post injury, 1-3 years post injury, and >3 years post injury). We did not specifically exclude individuals on the basis of any concomitant injuries or conditions.

Data analyses
The psychometric analysis included confirmation of unidimensionality, estimation of IRT parameters using a graded response model, and examination of differential item functioning (DIF). Confirmatory factor analysis (CFA) was used to test model fit using the following standards: CFI >0.90; RMSEA < 0.08, good: CFI > 0.95, RMSEA < 0.06, excellent. Significant loadings on the single factor (values >0.30) and local item dependence (residual correlations >|0.20|) were also examined. DIF was examined using Lordif for six categories: age (≤49 vs. ≥50), sex (male vs. female), education (some college and lower vs. college degree and above), injury location (tetraplegia vs. paraplegia), severity (incomplete vs. complete), and time post injury (<1 year vs. >1 year). Items were flagged when the probability associated with the χ 2 test was <0.01 and the effect size measures (McFadden's pseudo R 2 ) were greater than 0.02, a small but non-negligible effect. After reviewing item content, we removed items iteratively to retain the best fitting items and allow for the most precise estimation. We repeated the analytic steps until an acceptable solution was attained that confirmed a unidimensional model and no misfitting items or DIF were present.

Assessment center programming and short form selection
The IRT parameters for the final set of retained items were then used to program a CAT version of the SCI-QOL Psychological Trauma item bank on the Assessment Center™ platform 48 (available at www.assess mentcenter.net). Final item parameters were also combined with clinical/expert input to create a static short form (SF) version of the measure, which can be administered by paper and pencil or via Assessment Center.™

Patient characteristics
We administered the Psychological Trauma and other SCI-QOL item pools related to emotional health (e.g. Depression, Positive Affect & Well Being, Resilience) to a sample of 716 individuals with SCI. Table 1 provides the sociodemographic and injury-related information.

Preliminary analysis and item removal
We conducted CFA iterations for the 31-item Psychological Trauma item pool. After each iteration we examined item content, model fit, internal consistency (Cronbach's α), corrected item-total correlations, excessive missing data (missing responses for greater than 5 items), sparse cells (fewer than 5 responses), and violations of monotonicity. After the first iteration we removed 6 items that exhibited local item dependence; some items also demonstrated misfit (significant S-X 2 statistic), 2 items; low item-total correlation, 4 items; and category inversion, 3 items. Three of the 25 remaining items demonstrated local dependence or low item-total correlations. DIF analysis on the 22-item set indicated that one item, 'I broke into a sweat when I thought about my accident,' exhibited DIF for level of education and was removed. Finally, two items has sparse data (n < 5) in category 1 (Never) which would have resulted in collapsed categories for those items (i.e. 4 instead of 5) so they were removed from the bank. The following results are based on the final 19-item set.

Dimensionality
The CFA fit to a unidimensional model was good to excellent (CFI=0.952; RMSEA=0.061). For the majority of items, R 2 values were greater than 0.30, however for 1 item the R 2 values was less than 0.30 (Trauma_21= 0.283). While no item pairs met our criteria for LID (residual correlations >|0.20|), two item pairs did have residual correlations >|0.15|. They were the following pairs: I avoided reminders of how my injury occurred (Trauma_33) WITH I was frightened by sudden noises (Trauma_24) (r = -0.157); and I was nervous when something reminded me of the accident (Trauma_19) WITH I felt trapped (Trauma_13) (r = -0.164).

IRT parameter estimation and model Fit
As seen in Table 2, item slopes ranged from 1.16 to 2.55 and thresholds ranged from -0.78 to 3.44.
The measurement precision in the theta range between 0.6 and 2.5 is roughly equivalent to a classical reliability of 0.95 or better ( Figs. 1 and 2).
The S-X 2 model fit statistics were examined using the IRTFIT 49 macro program. All items had adequate or better model fit statistics (P > 0.05) with marginal reliability equal to 0.89.

Differential item functioning
Nine items were flagged for DIF in at least one category based on the χ 2 test; however, when the effect size measures were examined, the DIF was negligible in all cases.

Short form selection and mode of administration
We programmed the 19 items and parameters into the Assessment Center SM48 platform, which enables the bank to be used as a CAT or downloaded as a static short form. In addition, researchers may also wish to select individual items based on item content and parameters to create a customized short form. We compared the reliability of the full 19-item bank with the 8-item short form, a variable-length CAT with the Assessment Center default parameters (i.e. minimum # items to administer = 4, maximum number of items to administer = 12, maximum standard error = 0.3), a variable-length CAT with a minimum of 8 items (and the same maximum, 12), and a CAT fixed to 8 items (i.e. length of short form) to examine the level of measurement precision and error across modes of    administration. Table 3 presents measures of central tendency and dispersion for the various administration modes.
As expected, the full bank demonstrated the highest level of precision (Fig. 2). Measures from all modes of administration correlate highly with the full bank, with the 8-item minimum variable-length CAT demonstrating the highest level of precision (0.99 correlation with full bank, see Table 3) and the short form the least (0.95). Over 50% of the sample received the maximum number of items (i.e. 12), indicating that a large portion of the sample responded indicating no (or very few) trauma symptoms (Fig. 2). Since these individuals were at the floor of the trait distribution (i.e. where the reliability of trait estimates is lowest), the CAT did not reach a low enough standard error value to discontinue before the maximum number of items. While many of the other SCI-QOL CATs have an average number of items administered of 6-7, the average number of items administered in the Psychological Trauma CAT is just over 10. For this reason, the variable-length CATs perform better than the short form or 8-item fixed length CAT since on average the variable-length CATs are administering 2 additional items. We recommend administering the SCI-QOL Psychological Trauma bank as a CAT with a minimum of 4 items and a maximum of 12 items to maximize measurement precision while limiting response burden.
In some cases, however, it is neither possible (i.e. internet unavailable) or practical (i.e. laptop/tablet computer equipment beyond budget of project) to administer items in this way. As such, and as with all other SCI-QOL item banks, the project investigators utilized psychometric and clinical input to develop a fixed, 8-item 'short form' version of the item bank. Short forms may be administered directly within Assessment Center, or may be downloaded from Assessment Center for administration by paper and pencil or an alternate data capture platform or system.

Scoring
SCI-QOL Psychological Trauma scores are standardized on a T metric, with a mean of 50 and a standard deviation of 10, based on the SCI-QOL calibration data (i.e. a mean of 50 reflects the mean of an SCI population rather than the general population). All CAT administrations of the SCI-QOL Psychological Trauma item bank are automatically scored by Assessment Center. When administering the short form, whether via Assessment Center, paper and pencil, or another data capture platform, an individual must complete all 8 component items in order to receive a valid scaled score. The raw score for the short form is computed by simply summing the response scores for the individual component items. The T-score and associated standard error for each raw score value are located in Table 5.

Reliability
As a part of the reliability study described in the Tulsky et al. 42 methods paper in this issue, we compared Psychological Trauma scores at Baseline with those from the 1-2 week retest assessment. In a sample of 245 individuals with SCI, Pearson's r = 0.84 and ICC (2,1) = 0.84 (95% CI = 0.80 to 0.88). Discussion SCI presents considerable challenges to physical, psychological and social well-being. Longitudinal studies have reported that overall rates of injuryrelated distress among individuals with SCI changes little over time, and that individuals displaying clinically significant symptoms of emotional distress in the weeks immediately following injury are significantly more likely to do so in the future, even as far out as 10 years, highlighting the need for early interventions. 50,51 The measurement of psychosocial issues following traumatic injury has improved dramatically over the past 10 years. Measures in areas such as depression, alcohol use/abuse, and social participation have all become central variables in clinical research. Unfortunately, though, measurement of anxiety in general and especially in the more extreme form as PTSD, has received less attention by the medical community.
Traumatic SCI is, by definition, the type of trauma that could cause post-traumatic stress. There seems to be a disconnect between the high rates of PSTD (i.e. between 10-40%) 52 and the fact that it has not been studied more in this population, which leads us to conclude that this is a very understudied area. More research is necessary to determine how to prevent and treat PTSD and also to determine whether this is in response to the event of prior trauma, to the event of SCI, or to the physical disabilities caused by SCI. More research is needed to determine whether these psychological reactions to trauma are universal in persons with SCI. Also, additional research can determine whether the SCI-QOL Psychological Trauma item bank is sensitive to identify individuals who are at risk for PTSD.
It is for all of these reasons that the SCI-QOL Psychological Trauma item bank was developed to measure the constellation of psychological symptoms and issues related to the traumatic and stressful event of an SCI. We removed misfitting items systematically based on psychometric and clinical criteria and the final SCI-QOL Psychological Trauma item bank contains 19 items; users have the option of using an 8item short form or CAT. This measure has been developed to help fill the research gap related to SCI and PTSD and the item bank is intended to serve as an SCI-specific indicator of clinical or subclinical levels of psychological consequences of trauma/post-traumatic stress for SCI research. In the future, it would be helpful to evaluate the clinical potential of this scale in assisting with the formal diagnosis of PTSD and to move the SCI-QOL Psychological Trauma scale from a research tool to a clinical measurement instrument. Several new areas of research are recommended. These would include establishing minimally clinically important differences 53 and developing clinically meaningful standards or scoring cut-points. 54 Additionally, we should view reactions to trauma and PTSD as changing states. It will be necessary to examine clinical trajectories and patterns of change over time 55,56 and develop clinically meaningful standards of these change scores. We should be able to detect when the change represents a clinically meaningful state that health care providers should attend to and implement treatment procedures. An additional area that is currently understudied is the impact of concomitant TBI with SCI, especially given the potential for the confounding of TBI sequelae with PTSD symptomatology. We recommend further research in individuals with concomitant SCI and TBI, perhaps including development of separate clinical guidelines for use in this population. As described in the Introduction, the impact of the psychological consequences of the SCI and its related traumatic event is an understudied area. The item bank described in this manuscript represents a way to diagnose psychological trauma and has the potential to greatly facilitate the future study of trauma/PTSD in individuals with SCI.

Conclusion
With this paper, we offer the SCI research community a new tool to identify individuals' functioning related to the psychological consequences of trauma. We developed this scale using qualitative feedback from individuals with SCI to ensure a patient-centered approach to assessment and then we tested the items in a large sample of individuals with SCI and state-of-the-art quantitative measurement techniques. We have developed a 19-item bank that may be administered using a subset of these items as either a CAT or fixed-length short form. This new tool should help fill the gap in measurement of this important psychosocial outcome in SCI research.

Disclaimer statements
Contributors All authors have contributed significantly to the design, analysis and writing of this manuscript. The contents represent original work and have not been published elsewhere. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors or upon any organization with which the authors are associated.
Funding This study was supported by National Institutes of Health grant number 5R01HD054659 (Eunice Kennedy Shriver National Institute of Child Health and Human Development/National Center on Medical Rehabilitation Research and the National Institute on Neurological Disorders and Stroke).
Conflicts of interest All SCI-QOL items and parameters are © 2015 David Tulsky and Kessler Foundation. All rights reserved. All items are freely available to the public via the Assessment Center platform (www.assess mentcenter.net). There are currently no plans for Dr. Tulsky or Kessler Foundation to profit from the use of the SCI-QOL instrument.
Ethics approval The Institutional Review Board at each site reviewed and approved this project.