A pilot study of user satisfaction and perceived helpfulness of the Swedish version of the mobile app PTSD Coach

ABSTRACT Background: There is a need for accessible interventions in the aftermath of traumatic events with documented efficacy for preventing or reducing negative mental health consequences. The PTSD Coach is a mobile app that has shown to be effective in reducing symptoms of posttraumatic stress (PTSS). Objective: The purpose of the current study was to evaluate the user satisfaction, perceived helpfulness and potential reductions of PTSS and symptoms of depression among participants using the Swedish version of the PTSD Coach. Method: This was an uncontrolled pre-test post-test open trial including participants recruited from the community via advertisement and from an ongoing observational study who had experienced a potentially traumatic event in the last five years. Participants had access to the Swedish PTSD Coach app for four weeks. Results: Eleven participants (mean age = 38.6, female = 8) completed the study. Nine of the participants met criteria for full or partial PTSD. Results from the PTSD Coach Survey indicated that participants found the app slightly to moderately helpful and were slightly to moderately satisfied with the app. Nominal but not statistically significant reductions of medium effect sizes in PTSS (PCL-5) and depression (PHQ-9) from pre- to post-assessment were found. In interviews, participants indicated that they found elements such as learning about PTSD, breathing exercises and monitoring symptoms helpful in managing symptoms. However, several participants indicated that they had not used the app as much as they had intended to. Participants also had suggestions for improvements such as enhanced app structure and better guidance regarding how to use the app. Conclusions: The perceived helpfulness and user satisfaction were slightly lower compared to research on the original version of the app. Experiences from the study are discussed and a future controlled study of the Swedish version of the PTSD Coach is suggested.


Introduction
A majority of individuals will experience a potentially traumatic event during their lifetime (Frans, Rimmö, Aberg, & Fredrikson, 2005). For some individuals, these experiences are associated with the development of mental health problems such as depression (Mandelli, Petrelli, & Serretti, 2015), substance abuse and stress-related disorders such as posttraumatic stress disorder (PTSD; Arnberg, Bergh Johannesson, & Michel, 2013;Arnberg et al., 2015). However, many individuals experiencing mental health problems after overwhelming events do not receive evidence-based support or treatment (Witteveen et al., 2012), or do not receive treatment until many years after the onset of their problems (Goldstein et al., 2016). There is a need for accessible interventions in the aftermath of traumatic events with documented efficacy for preventing or reducing negative mental health consequences after serious events.
In Sweden, 85% of the population had a smartphone in 2017 (The Internet Foundation in Sweden, 2017). These devices can be used to access self-help interventions aiming to improve both physical and mental health. Currently, there are a plethora of apps available for smartphone users that assert that they help people with their physical or mental health. Although there has been a rapid proliferation of psychological interventions delivered via the Internet for a range of mental health disorders (Andersson & Titov, 2014;Arnberg, Linton, Hultcrantz, Heintz, & Jonsson, 2014), most of these apps lack empirical support, i.e. their efficacy is unknown, and it is therefore important to evaluate mobile apps in scientific studies with rigorous designs (Olff, 2015). In addition, before conducting rigorous studies of novel psychosocial interventions, such as a mobile app, it is important to conduct more basic work evaluating development and feasibility of the intervention (Anderson, 2008).
The PTSD Coach is a mobile app developed by the National Center for PTSD-Dissemination and Training Division, Veterans Administration (VA), by means of participatory research, where focus groups with PTSD patients were used to generate input on features and design (Kuhn et al., 2014). The app was designed to be used either as a stand-alone tool providing psychoeducation and self-management, or to supplement varying levels of care provided by a professional. The app provides psychoeducational information about PTSD and strategies for coping with PTSD symptoms. Versions of the app in English have been publicly available on Google and Apple app markets since 2011 and the developers have focused on making the app available to as many as possible. The original English PTSD Coach has also been evaluated in several studies. An open trial conducted in a VA setting suggested that users were very satisfied with the app and that they perceived it as helpful for their PTSD symptoms (Kuhn et al., 2014). To our knowledge, two randomized trials have investigated the efficacy of the PTSD Coach. Miner and colleagues (2016) investigated feasibility, acceptability and potential efficacy by comparing one month of access to the PTSD Coach to a wait-list condition. They reported that participants found the app moderately helpful and that they had learned to use the skills from the app to manage their symptoms, but the investigators found no statistically significant betweengroup effect size in terms of PTSD symptom improvement. Kuhn and colleagues (2017) also recruited trauma survivors from the community and compared three months of access to the PTSD Coach to a wait-list and found significant improvements in PTSD symptoms, depression and psychosocial functioning. Finally, in a small randomized study with primary care veterans, Possemato and colleagues (2016) compared access to the PTSD Coach with access to the app enhanced with guidance from a clinician and suggest that the addition of clinician support appears to increase the effects of self-management alone, but that a larger study is needed to confirm findings.
There has also been an international interest in the app. To our knowledge, work is ongoing in developing and evaluating versions in Danish, Norwegian, German and Dutch languages. Given that the PTSD Coach was developed in the US and within a military context, it is important to investigate if other language versions used in different clinical and cultural contexts also are valid and feasible tools. The purpose of this study was to preliminarily evaluate the user satisfaction, the perceived helpfulness and potential reductions in symptoms of posttraumatic stress (PTSS) and depression among participants using the Swedish version of the PTSD Coach.

Design
This study was an uncontrolled pre-test post-test open trial in which all participants had access to the PTSD Coach app for four weeks.

Participants
Inclusion criteria were: age 18 years or above, Swedish speaker, access to a smartphone connected to the Google Play Store or the Apple App Store and experience of a potentially traumatic event, as defined by the DSM-5 (American Psychiatric Association, 2013), during the past five years. Exclusion criteria were ongoing severe psychiatric problems such as psychotic symptoms, severe depression, mania, ongoing substance abuse and risk for suicide.

Measures
2.3.1. MINI International Neuropsychiatric Interview 6.0 (MINI 6.0) The MINI 6.0 (Lecrubier et al., 1997) is a structured interview developed to screen for psychiatric disorders according to DSM-IV criteria. The interview covers the most common mood disorders, psychotic symptoms, substance abuse, anxiety disorders and suicidality and has demonstrated good inter-rater reliability All modules in the interview were used except the module assessing PTSD.

Clinician-Administered PTSD Scale for DSM-5 (CAPS-5)
The CAPS-5 (Weathers et al., 2017) is a structured interview to assess PTSD according to DSM-5 criteria. In the current study partial PTSD was also assessed, requiring endorsement of one symptom from each of the B, C, D and E criteria.
Initial evaluations indicate that the interview shows strong inter-rater and test-retest reliability, as well as correspondence with earlier versions of the CAPS (Weathers et al., 2017). The Swedish version of the CAPS-5 was developed at the National Centre for Disaster Psychiatry, Uppsala University, and preliminary evaluations indicate similar properties as compared to the original version (unpublished data).

PTSD Checklist-5 (PCL-5)
The PCL-5 (Blevins, Weathers, Davis, Witte, & Domino, 2015) is a self-report scale that assesses symptoms of PTSD and comprises 20 items corresponding to the DSM-5 PTSD symptom criteria. The respondents rate the intensity of each symptom during the past month using a 5-point scale from 0 'not at all' to 4 'extremely'. Initial evaluations of the instrument indicate that it has good test-retest reliability, convergent and discriminant validity (Blevins et al., 2015). The Swedish version was developed using a standard back-translation process and show satisfactory psychometric properties (Sveen, Bondjers, & Willebrand, 2016).

Patient Health Questionnaire-9 (PHQ-9)
The PHQ-9 (Kroenke, Spitzer, & Williams, 2001) assesses symptoms of depression and includes nine items. Respondents rate how much each symptom has been bothering them during the last two weeks using a 4-point scale from 0 'not at all' to 3 'almost every day'. The instrument has shown good psychometric properties and adequate validity as a measure of symptoms of depression (Gilbody, Richards, Brealey, & Hewitt, 2007;Kroenke et al., 2001).

PTSD Coach Survey
This instrument was developed by the original developers and investigators of the PTSD Coach app (Kuhn et al., 2014). The survey comprises 14 items that assess perceived helpfulness of the app and one item asking about user satisfaction. The original English version was translated to Swedish by the authors. Items are rated on a 5-point scale ranging from 0 'not at all' to 4 'extremely'. The instrument was administered via the Internet after the participants gained access to the app. The 14 items measuring perceived helpfulness of the app demonstrated high internal consistency (Cronbach's alpha = 0.96).

Interview
A semi-structured interview via telephone was conducted after access to the app in order to get a more in-depth assessment of user patterns and experiences from interacting with the app. Examples of questions included were: 'How have you used the app?', 'Which parts or functions of the app did you find most helpful?' and 'Do you have suggestions for improvement of the app?' The answers were probed with follow-up questions.

PTSD Coach app
The source code for both IOS and Android versions was made available to the authors from the original developers (Kuhn et al., 2014) with the purpose of developing a Swedish version of the app. The content of the original app was translated from English to Swedish, new audio files were recorded and support information was adapted to match the available resources in Sweden. The material was then compiled into a Swedish version.
The app contains four main sections: 1. Learn. This section provides psychoeducation about PTSD, e.g. prevalence, symptoms and how it develops, and information about treatment options; 2. Self-assessment. This section includes the PTSD-Checklist Civilian version, a well-validated measure of symptoms of PTSD based on the DSM-IV. The respondent is given direct feedback on the severity of their symptoms and how it compares to their previous rating. Results are also presented graphically which enables the user to track their development of symptoms over time. Users can also schedule new assessments at different intervals and also schedule reminders; 3. Manage symptoms. The section provides the user with different coping strategies addressing symptoms of PTSD. Examples of coping strategies include relaxation techniques, positive and adaptive self-statements and pleasant-events scheduling. Before each tool is presented the user is asked to provide a Subjective Unit of Distress Scale (SUDS) rating from 0 to 10 of their current distress. The SUDS rating is presented as a thermometer. This is repeated after the tool has been used and the user is provided with feedback on how their distress level changed. If the level is unchanged or has increased, users are prompted to try another tool; 4. Find support. This section provides the user with contact details to different support functions, e.g. the national telephone number for emergency situations and different helplines that can be accessed via the telephone or the web. The users can also select contacts from their phone's contacts app and add contact details to people of their own choice.
During this study, the Swedish version of the PTSD Coach was not publicly available on the Google Play Store or the Apple App Store. Instead, the participants accessed and downloaded the app via closed beta-testing functions, which made it possible to provide access as necessary.

Procedure
Participants were recruited from the community via advertisement in local newspapers and online. Interested individuals were instructed to e-mail their name and telephone number to a study coordinator. Individuals participating in an ongoing longitudinal observational study including adults with experiences of traumatic events were also asked for interest in participating in the current study. Interested individuals were assessed for eligibility via telephone. The participants who met inclusion criteria were assessed with the MINI and the CAPS-5. Before participants were provided access to the app, they were assessed via the Internet with the demographic questionnaire, the PCL-5 and PHQ-9. After completing the assessment, the participants accessed the app using their own devices via closed beta-testing functions on the Google Play Store and Apple App Store. After four weeks of access to the app, the participants were again assessed via the Internet with the PCL-5, PHQ-9 and the PTSD Coach Survey. Participants were interviewed via telephone about their experiences of using the app. The procedure was approved by the regional ethical vetting board in Uppsala (Reg. no.: 2015/258). All participants provided written informed consent.
During the project, technical difficulties made it necessary to modify the source code and recompile the code along with the content in Swedish several times in order to produce an acceptable working version. These difficulties caused delays in the project; participants who had been screened and completed the MINI and the CAPS interview had to wait several months before eventually accessing the app. However, the self-reported assessments via the Internet (PCL-5 and PHQ-9) were administered just before access to the app and after the four weeks of access.

Data analyses
Descriptive statistics were used to summarize the findings. The Wilcoxon paired samples test (2-tailed) was used to test for differences in PTSS and depression between the two assessments. The effect size Cohen's d, corrected for the correlation between paired samples (Morris & DeShon, 2002), was calculated as a metric of the standardized mean difference between the pre-and the post-assessments. Two participants had missing data on one item each and these two values were imputed with the participants mean score on the observed items. All quantitative analyses were performed with the statistical package IBM® SPSS® Statistical package version 23.
Interviews were conducted via the telephone except for one participant who was interviewed in person. The interviews were conducted by one of the authors (MC) and the participants' responses to the semi-structured interview were noted during the interviews. Data was analysed according to principles from qualitative description (Sandelowski, 2010). Notes were summarized in key units, and these key units were organized according to their common content (by MC). From this organization of key units, categories were identified (by MC). Another author (JS) reviewed the summaries and categories, and inconsistencies were resolved by discussion.

Participant characteristics and type of trauma
See Figure 1 for participant flow through the study. Of the 16 participants who participated in the pre-assessment, 13 were recruited via advertisement and three were recruited via the ongoing observational study. Eleven participants who completed both the pre-and post-assessment were included in the analyses. There was no difference in age between completers and non-completers, however non-completers had significantly lower scores on the PCL-5 (mean difference = 24.3, p < .05) and on the PHQ-9 (mean difference = 5.9, p < .05). Among completers, five participants met criteria for full PTSD and four met criteria for partial PTSD. Among non-completers, one participants met criteria for full PTSD and the remaining four did not meet criteria for partial PTSD. In the included sample the mean age was 38.6 years (range 23-55 years). Participant characteristics are shown in Table 1. Three participants reported that they had experienced an interpersonal trauma as their index event in the CAPS-5 and eight participants reported a non-interpersonal trauma as their index event.

Psychiatric morbidity
According to the CAPS-5 interview, five individuals met criteria for full PTSD and four met the criteria for partial PTSD at the time of the interview. Partial PTSD was defined as meeting one or more criteria in each of the three symptom groups, i.e. B, C and D (Breslau, Lucia, & Davis, 2004). According to the MINI 6.0, two participants met criteria for one psychiatric diagnosis and eight participants met criteria for more than one diagnosis (Table 2).

PTSD Coach Survey
The item ratings on the PTSD Coach Survey ranged between 0.91 and 1.91, indicating that the participants were, in general, slightly to moderately satisfied with the app and found it slightly to moderately helpful (see Table 4).

Interviews
Nine participants were interviewed by telephone shortly after completing the post-assessment with semi-structured questions and one participant was interviewed in person. The data from the interviews were summarized in the following descriptive categories.

Helpful components
Exercises such as breathing and relaxation techniques were highlighted by participants as helpful (n = 4). Furthermore, reading and learning more about PTSD was also perceived as helpful (n = 3). One participant described that the information provided in app had a normalizing effect on her experiences. Also, the advantage of having the information always available and being able to go back at any time was perceived as helpful (n = 1). The RID-tool (Relax, Identify, Decide; n = 3) and the SUDS rating before and after an exercise (n = 2) were also put forth as helpful. One participant valued the option to schedule reminders for planned activities and exercises.

Changed coping strategies
Some participants felt that the app had helped them to reflect on their situation and identify difficulties (n = 3) and to understand their difficulties in a better way (n = 1). Participants also reported that knowing that they had the app had given a sense of support and a sense of feeling calmer (n = 2).   Table 3. PCL-5 and PHQ-9 scoring distribution at pre-and post-assessment (n = 11).  Helpfulness and satisfaction ratings: 0 = not at all; 1 = slightly; 2 = moderately; 3 = very; 4 = extremely.
3.5.3. Using the app when alone Some of the participants preferred to use the app at home (n = 3) and in quiet situations (n = 2). For example, one participant used it when having trouble falling asleep and another used it around bedtime.

Less usage than intended
Several individuals said that they had not used the app as much as they had intended to (n = 5) and some of the reasons stated were that it did not meet expectations (n = 1), eye problems (n = 1) and difficult app structure (n = 1).

Technical problems
Two participants experienced technical problems, in which the playback of an audio track during an exercise ended prematurely or did not start properly.
3.5.6. Suggestions to improve the app Some participants suggested that the app could benefit from more structure (n = 4), that it would make it easier to follow, and suggested more personalized settings (n = 3), such as being able to change colour and voices in the app (n = 1) or create shortcuts from the homescreen to specific content in the app (n = 1). Two participants stated that the app was difficult to figure out and that it could benefit from a more explicit overall purpose and additional guidance throughout the different parts of the app. One participant stated that they had not understood the purpose of some of the content and suggested improved rationales for some of the exercises. Another participant suggested that the app could be used in combination with other services rather than as a stand-alone intervention.

Discussion
The overall purpose of this study was to evaluate the user satisfaction, perceived helpfulness and potential reductions of PTSS and symptoms of depression when using the Swedish version of the PTSD Coach.
To this end, a Swedish version of the PTSD Coach was developed based on the source code from the original English language version. Participants from the community were recruited and invited to use the app. The participants were in general slightly to moderately satisfied with the app and found it to be slightly to moderately helpful. In addition, nominal reductions in PTSS and symptoms of depression with moderate effect sizes were observed, although these failed to reach statistical significance. According to interviews, the participants found specific functions of the app helpful such as relaxation exercises, the RID-tool and the SUDS rating. They also had suggestions for improvements such as a clearer structure and linearity in the app, and being given more possibilities of personalizing the app. Several participants indicated that they had not used the app as much as they intended to, or thought that they would be, when signing up for the study. The current results both departs and coheres with previous investigations of other language versions of the app. In their preliminary investigation, Kuhn and colleagues (2014) generally found higher ratings on the PTSD Coach Survey, rating it as moderately to very helpful or being moderately to very satisfied with the app. One reason for the lower ratings in the present study may be that the participants in the current investigation were recruited from the community whereas the participants in the Kuhn et al. (2014) study were attendees at a VA residential treatment programme. Thus, participants in the Kuhn et al. (2014) study may have had increased contact with the research staff, and hence were more positive to the content of the app. In addition, in the current project, several technical difficulties were encountered in the process of producing the Swedish app. The resulting delay between recruitment and access to the app may have led to participants having a more negative attitude towards the app, which, in turn, may have contributed to the lower helpfulness and usability scores. It is important that research aiming to evaluate technical solutions such as mobile apps find and maintain reliable technical competencies required for successful implementation of the technology. Also, several of the participants indicated that they had not used the app as much as they thought they would have. Although the usage data were based on self-report, this observation is in line with a previous investigation of the use of the original English version of the app. Owen and colleagues (2015) analysed user data from their apps accessed on the Apple App Store and Google Play Store and found that 61% of users who downloaded the app returned to the app the next day, and that 52% of users continued to use the app or used it at least one time beyond the first week of download. This indicates that it is important that mobile apps targeting mental health problems are designed so that they engage users in accessing the app content and that use is maintained over time. One such example would be providing tutorials or examples for how to use the app. Another method to increase usage and adherence to key components could be to combine access to the app with therapist support via e-mail, phone or instant messaging as suggested by the Supportive Accountability model (Mohr, Cuijpers, & Lehman, 2011).
Despite these difficulties, there was a nominal reduction in PTSS with a moderate effect size, albeit not reaching statistical significance, which is in line with a recent investigation (Miner et al., 2016). Using a controlled wait-list design with a sample recruited from the community, Miner and colleagues (2016) found a non-significant between-group effect size of 0.25 after one month of access to the app. However, they found a moderate within-group effect size of 0.59, indicating a reduction in PTSS of a magnitude similar to the reductions found in the current study. A more recent investigation increased the time of access to the app to three months of use and found a larger between-group effect size (Kuhn et al., 2017). Future studies may therefore benefit from extending the trial period beyond one month of access to the app, as used in the current study, as it might be a too short period of time to produce marked reductions in symptoms. Future studies of the PTSD Coach and other similar apps may also focus on investigating whether usage of certain features in the apps are related to specific reductions in distress. Combining rigorous trial designs where app-usage data is continuously collected (logins, duration of usage, duration at certain elements, etc.) with, for example, ecological momentary assessment (Shiffman, Stone, & Hufford, 2008), in which data on symptoms/distress are collected with high frequency, could yield more detailed information about the potential mechanisms of change related to app usage.
The current investigation has several limitations. First, the project was hindered by technical difficulties which caused delays in participants accessing the app, and this may have had a negative influence on the attitude of the participants towards the app. Second, the study was small with participants recruited from the community via advertisement and from an ongoing study, which may negatively affect the representativeness of the included sample. The small sample size also limits the power to adequately detect change over time in terms of the symptoms assessed. In addition, the study lacks a control group so we do not know whether the changes in PTSS and depression were due to access to the app or merely a result of time. In the current study, negative effects were not formally assessed. Negative effects such as deterioration, adverse events, novel symptoms and non-response are understudied in psychological/ psychosocial intervention research (Jonsson, Alaie, Parling, & Arnberg, 2014). In future trials, we intend to also assess any negative effects associated with using the Swedish version of the PTSD Coach. Finally, it may be argued that participation in a beta-testing programme leads to the overall user experience not mimicking real-world usage and may thus negatively influence the ecological validity and generalizability of the findings.
To conclude, the current investigation suggest that the Swedish version of the PTSD Coach was perceived as slightly to moderately helpful and that participants were slightly to moderately satisfied with the app. Furthermore, participants experienced substantive but not statistically significant reductions in PTSS and depression during access to the app. Participants also indicated that they had not used the app as much as intended, and had suggestions for improvements. As such, it is a first step towards investigating the efficacy of the PTSD Coach in a Swedish context. As a next step, the experiences gained in the current study will be used in modifying and refining the Swedish version of the PTSD Coach. Modifications will include using the updated code from the developers including a refreshed user interface and clarification of some of the rationales for the different exercises. Examples of different use cases will also be included to provide guidance when using the app. This new version will then be subjected to evaluation in a more rigorous study using a controlled design.

Funding
This project was funded by grants from the Swedish National Board of Health and Welfare (Socialstyrelsen).