Reliability and validity study of the Turkish version of the remission from depression questionnaire

ABSTRACT Objective: In current psychiatric practice, the principal goal in the treatment of patients with Major Depressive Disorder (MDD) is to achieve remission. Symptom-based definitions of remission do not adequately reflect the treatment expectations of depressed patients. The “Remission from Depression Questionnaire (RDQ)” evaluates not only symptoms of depression but also patients’ other psychiatric symptoms, positive mental health, ability to function in various contexts, sense of well-being, life satisfaction and ability to cope. The first aim of this study is to assess the reliability and validity of the RDQ’s Turkish language version. Secondly, it purposes to compare the scale with the Hamilton Depression Rating Scale (HDRS) and the Beck Depression Inventory (BDI) in psychometric terms. Method: 106 outpatients who met MDD diagnosis criteria according to Diasnostic and Statistical Manual of Mental Disorders-IV (DSM-IV) were included in the study. After the translation and back-translation processes, the RDQ, HDRS, BDI, Beck Anxiety Inventory and World Health Organization Quality of Life survey (short version) were administered. Results: All item-scale correlations were found to be statistically significant, and the median value for correlations was .534. The RDQ Turkish version demostrated excellent internal consistency with Cronbach’s alpha coefficient of .945. The test-retest reliability value for the overall RDQ was .908. The correlations between the RDQ subscales and psychosocial functioning, quality of life areas, anxiety severity and depression severity were evaluated and convergent validity analysis was performed, revealing significant relationships between the RDQ total and subscale scores and the other applied scales. For a discriminant validity assessment, the RDQ scores and subscales of two groups made up of the patients who were in remission and those who were not in remission were compared with the HDRS score (with a cutoff point of 7 points) or BDI (with a cutoff point of 17 points) score. We conducted a ROC curve analysis to determine the most appropriate cutoff score for the RDQ. When the cutoff value 7 of HDRS were used, sensitivity was found to be 86.7% for a cutoff value of 43 for the RDQ, and specificity was 73.5%. When the cutoff value 17 of BDI were used, the sensitivity was 85.5% and the specificity was 75% for a cutoff value of 43 for the RDQ. Conclusion: Our findings show that the Turkish version of RDQ is a reliable and valid scale.


Introduction
In current psychiatry, the primary goal in the treatment of Major Depressive Disorder (MDD) is to achieve remission [1][2]. Although both the existence of symptoms and some impairment of function are necessary for the diagnosis of MDD, according to the evaluation criteria currently in use, complete or nearly complete relief from symptoms is sufficient to establish remission [3]. However, studies have shown that a significant proportion of patients with residual symptoms in remission cannot return to their old functionality [4].
In studies of patients' responses to MDD treatment, symptoms-based metrics applied by clinicians, such as the Hamilton Depression Rating Scale (HDRS), are used. At the cutoff point at which the HDRS, which is commonly used in studies, defines remission, residual symptoms may still be clinically observed in patients [5][6]. Studies on this subject have showed that patients with residual symptoms experience greater psychosocial dysfunction and lower quality of life than those without residual symptoms, and that there is a discrepancy between patient-reported remission and remission as indicated by measures like HDRS [6][7]. While relief or elimination of depressive symptoms is an important goal, it has been shown that this goal does not meet the expectations of patients in treatment for depression [4].
Since current definitions of remission do not reflect the patient's treatment goals precisely and are based solely on symptoms, these definitions can be said to be insufficient. In the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project, it has been shown that remission in patients with depression is determined by several factors in addition to relief from symptoms [4]. In the results of the MIDAS project, it was reported that the patient's evaluation of symptom severity, depressionrelated loss of ability to function, and quality of life were each related to the subjective perception of the patient's remission status in a meaningful and independent manner [8]. The Remission from Depression Questionnaire (RDQ) was developed by Zimmermann et al. in 2011 as part of the MIDAS project [9]. The aim of this scale is to identify the areas patients consider to be relevant to remission. As a whole, it aims at the general condition of the patient, their goals for treatment, their response to treatment, and determination of remission [10]. It further aims at evaluating positive mental health characteristics, coping skills, functionality, life satisfaction and perception of general well-being, as opposed to the solely symptomsbased approach of metrics like the HDRS and Beck Depression Inventory (BDI). In the first study of the scale, it has been shown that it has appropriate psychometric properties in terms of validity and reliability [9].
The aim of this study is to evaluate the Turkish version of DRQ in terms of validity and reliability. Secondly, it purposes to compare the scale with HDRS and BDI in psychometric terms.

Materials and method
Translation Approval for the Turkish translation of the scale was obtained from the developer of the scale, Mark Zimmermann, via e-mail. The initial translation of the RDQ from English to Turkish was performed independently by two bilingual translators. The forward translations were further evaluated by the researchers of this study. The forward translation was then back-translated into English by two independent back-translators who were unaware of the concepts and the purpose of the scale. The backward translation was reviewed by a group consisting of the researchers of this study, an English lecturer and a Turkish lecturer. Subsequently, a final version of the Turkish RDQ was approved.

Sample
Out of those patients who had agreed to participate in the study on a volunteer basis and had read and signed the written informed consest, those who were over the age of 18, diagnosed with MDD, and of sufficient cognitive ability to answer the scales were included in the study, without regard to gender, while patients who were under the age of 18, intellectually disabled, uneducated, disoriented, psychotic, catatonic or unwilling to volunteer were not included in the study. Accordingly, between June and November of 2013, a total of 106 outpatients with MDD at Dışkapı Yıldırım Beyazıt Additionally, 30 adults, who were randomly selected, were administered a second RDQ evaluation within a week of the first.

Evaluation tools
Remission from depression questionnaire (RDQ) This metric is the subject of the validity and reliability study. It is made up of 41 items. The areas of evaluation are: symptoms of depression, symptoms of anxiety and irritability that are commonly seen in depression patients, characteristics of positive mental health, coping skills, functionality, life satisfaction, and perception of general well-being. These items assess the preceding week in the life of the patient according to these areas of evaluation, using a three-rating scale (never or very infrequently, sometimes, often or all the time). The items are scored as 0, 1, or 2; except the first two sub-scales, the scores of the sub-scales are reversed and the total score is obtained for the sub-scales and RDQ. Higher values reflect more severe pathology. The original version of RDQ demonstrated excellent internal consistency (Cronbach's alpha of .97 for the total scale) [10].

Beck depression inventory (BDI)
The BDI is a 21-item self-assessment scale that evaluates the patients' perceived depressive symptoms quantitatively, and measures the symptoms that occur in the vegetative, emotional, cognitive, and motivational areas of depression. The severity of the patient's depression is indicated by how high the total score is. It has been suggested that those who score 17 and above on the BDI may constitute an at-risk group. The Cronbach's alpha value for the BDI-Turkish version was .80 [11].

Beck anxiety inventory (BAI)
The BAI evaluates the frequency with which an individual experiences symptoms of anxiety. It is a selfevaluation made up of 21 items and scored from 0 to 3. The internal consistency of BAI-Turkish version was high (Cronbach's alpha = .93) [12].

Hamilton depression rating scale (HDRS)
The HDRS is a questionnaire consisting of 17 questions and administered by the clinician. It evaluates the level of depression and the severity of depression in the patient. Scoring is between 0 and 4. The highest score is 51. In the study of Turkish validation of the HDRS, Cronbach alpha coefficient was .75 [13].

World health organization quality of life instruments (WHOQOL-BREF-Tr)
The WHOQOL, a general-purpose quality of life profile scale, consists of 26 questions. With the addition of a national question during the Turkish-language validity studies of this scale, WHOQOL-BRIEF-TR has 27 questions. WHOQOL-BRIEF-TR is a self-reported scale. Field scores related to physical, pyschological, social, environmental, and national environment can be calculated using the questions after first two general questions. The higher the score of the scale areas, the higher the quality of life related to that area. In the validity and reliability study of the WHOQOL-BREF-Tr, Cronbach's alpha values for the sub scales were between .20 and .82 [14].

Statistical analysis
Statistical analyses were performed using the computer package statistical programme SPSS (Statistical Package for the Social Sciences version 11.5; SPSS; Chicago, IL). The correlation matrices of the RDQ subscales were evaluated in order to evaluate the amount of shared variance on subscales. The test-retest and internal consistency reliability of RDQ and its subscales were evaluated (Cronbach's alpha). The correlations between RDQ subscales and psychosocial functioning, quality of life areas, anxiety severity and depression severity were evaluated and convergent validity analysis was performed. For a validity assessment, the RDQ scores and subscales of two groups made up of the patients who were in remission and those who were not in remission were compared with the HDRS score (with a cutoff point of 7 points) or BDI (with a cut-off point of 17 points) score (Mann Whitney U Test). The cut-off point for the RDQ was calculated by taking a cut-off point of 7 points for the HDRS and 17 points for the BDI. The specificity and sensitivity values were determined for these breakpoints. In addition, inter-measure agreement was assessed by Kappa reliability analysis. p value < 0.05 was considered statistically significant.

Item-scale correlations, internal consistency and test-retest reliability
All of the item-scale correlations were significant and the median value of the correlations was 0.534 (0.212-0.733). The Cronbach alpha coefficient for the RDQ was found to be 0.945 (Table 2).
In our study, 30 adults retook the RDQ an average of 5.63 (± 1.45) days after their initial evaluation. All of the test-retest reliability values of the RDQ and its subscales were high and the test-retest reliability of the RDQ was 0.908 (Table 2).

Intercorrelation values and validity of RDQ subscales
The intercorrelation values of the RDQ subscales are shown in Table 3. Correlations between the RDQ and subscales and HDRS, BDI, BAI, WHOQoL are presented in Table 4. Of the patients, 40 (37.7%) had an HDRS score > 7 and 66 (62.3%) had an HDRS score ≤ 7. The RDQ and subscale scores of individuals in the category of HDRS > 7 were significantly higher than individuals in the HDRS ≤ 7 category (Table 5).
Of the patients, 35 (33%) had a BDI score > 17 and 71 (67%) had a BDI score ≤ 17. The RDQ scores and sub-scales of the individuals in the BDI > 17 category were statistically significantly higher than those of the   Table 6.
The EAA values of the ROC curves, which were formed according to the HDRS = 7 cut-off point for the RDQ and subscale scores, were significantly higher than 0.500 b2 (unit square). The highest EAA value (0.819 b2) was obtained with an RDQ total score (p < 0.001) (Figure 1). 43 points was the best cut-off point in RDQ, and, according to this cut-off point, the sensitivity was 86.7% and the specificity was 73.5%. For this cut-off point, the fit between HDRS (cut-off point 7) and RDQ was calculated as 81.9% (kappa = 0.606). (Figure 1).
The EAA values of the ROC curves, which were formed according to the BDI = 17 cut-off point for the RDQ and subscale scores, were significantly higher than 0.500 b2 (unit square). The highest EAA value (0.852 b2) was obtained with an RDQ total score (p < 0.001) (Figure 2). 43 points was the best cut-off    point in RDQ, and, according to this cut-off point, the sensitivity was 85.5% and the specificity was 75%. For this cut-off point, the fit between BDI (cut-off point 17) and RDQ was calculated as 81.9% (kappa = 0.600). (Figure 2).

Discussion and conclusion
In current psychiatry, there is a consensus that the desired outcome in the treatment of depression is remission [1][2]. In studies related to treatment for depresison, remission indicates scores below the agreed cut-off point on a scale that determines symptom severity. However, there have been studies that show that indicators of remission that are based on symptoms and developed by the clinician do not adequately meet the patient's expectation of treatment [10]. Within the scope of the MIDAS project, it has been shown that various factors in addition to the regression of symptoms may be important to depression patient  in defining the transition from depression to remission [4]. In fact, a return to normal functionality and the presence of positive mental health seem to be more important indicators of the patient's transition from depression to remission than relief from symptoms. In accordance with this premise, the results of another study within the scope of the MIDAS project suggest that functional impairment and quality of life are more important than depressive symptoms. It has been found that each of these two parameters is correlated with the patient's subjectively perceived remission status at a meaningful and independent level [8]. The results of these studies revealed the necessity of developing a new scale including the other factors important in determining the remission status in depression, and this gave rise to the RDQ scale.
In our study, the first aim was to make a Turkish adaptation of the RDQ and our findings showed that the Turkish version of RDQ is a reliable and valid scale. All item-scale correlations were found to be statistically significant. The RDQ Turkish version demostrated excellent internal consistency with Cronbach's alpha coefficient of .945. All of the test-retest reliability values of the RDQ and its subscales were high. Discriminant validity was also high. When the cutoff value 7 of HDRS and 17 of BDI were used, the highest total value for sensitivity and specificity in the RDQ was 43 points.
Secondly, our study purposed to compare psychometric features with the HDRS and BDI, which are not included in the original version and are frequently used in MDD studies. In the validity and reliability study of the original version of RDQ, "Quick Inventory of Depressive Symptomatology" (QIDS) was used as a self-report scale for the purpose of subjective evaluation of the severity of depressive symptoms [10]. This scale consists of 16 items that assess only depressive symptoms [15]. One item was used to evaluate both indecisiveness and concentration disorder, one for both guilt and worthlessness, one for both desire to die and suicidal ideation. No validity and reliability study has been performed for QIDS in Turkish. For this reason, we used the BDI to determine the subjective perception of depression symptom severity in our study. In the study that developed the original version of BDI [16], the cut-off point for remission was not reported; in the Turkish validity and reliability study of BDI [11], the cut-off score of the scale was accepted as 17. According to the aforementioned cutoff point, those with high BDI scores had significantly higher RDQ and all subscale scores than those with low BDI scores. This outcome supports the thesis that a 17point cut-off in the BDI may be functional in the evaluation of remission.
In the discriminative validity assessment, the total score and subscale scores of the patients who were considered to be in remission according to the HDRS score were found to be significantly lower than those not judged to be in remission. In the convergent validity assessment, we found significant positive relationships between the RDQ scores and subscales and the HDRS, BDI, and BAI scores, and negative relationships between the RDQ scores and subscales and the physical, psychological, social, environmental and environmental-TR scores of the WHOQoL scale.
In the study of [10], there was a significant correlation between all RDQ subscales, and, on average, these subscales shared about 50% of variance [10]. In our study, the strongest relationships were found between the "Symptoms of depression" and "Other symptoms," between "Functionality" and "Life Satisfaction," and between "Life Satisfaction" and "Perception of general well-being" subscale scores. The subscales were found to share an average of 38.4% of variance. In this context, the scale was sufficient to distinguish between different structures.
In the study that developed the original version of the RDQ scale, the cut-off point for HDRS was taken as 7 points, and it was observed that the scale could differentiate the cases in remission [10]. Our results showed that the Turkish version of RDQ and its sub scales had features that distinguished cases in remission.
In the convergent validity analysis, the relationships between the RDQ and its subscales and the scales we used were weaker than those in the analyses with similar scales in the original study that developed the RDQ [10]. The weaker relationships in our study may stem from the fact that we used different scales in our study than those used in the original study. However, all the relationships in our study were significant, and we obtained findings in favour of the convergent validity of the Turkish version of RDQ.
We found that when we took 7 for the HDRS cut-off point, or 17 for the BDI cut-off point, the highest total value for sensitivity and specificity in the RDQ was 43 points. When 43 points was taken as the cut-off point in the RDQ, according to both the 7-point cut-off point in the HDRS and a 17-point cut-off point in the BDI, the level of fit with HDRS and BDI was quite high. As we know there is not any other version of RDQ except the original version in the literature. The cut-off point we found for the RDQ (43 points) is well above the calculated value (27 points) for the original version of the RDQ. In the validity and reliability study of Zimmerman et al., patients with socioeconomic security who were being treated at a private clinic were included in the evaluation [10], whereas a large proportion of the patients included in our study were not working and their income levels were low. It is therefore possible that this difference between the two studies may be due to socioeconomic and cultural differences.
One of the limitations of our study is that we evaluated a relatively small sample from a low socioeconomic stratum. Another limitation is that we do not have information about premorbid functionalities of the cases included in our study. In addition, considering that the RDQ consists of 41 items, it may be appropriate from the standpoint of the clinician to develop a short scale, which would be easier to use and evaluate for the clinician, and more understandable and less tedious for the patient. However, it should be borne in mind that the RDQ evaluates a number of different areas. Considering the difficulties created for the practitioner or the respondent when scales are used to evaluate all of these areas separately, the RDQ may even be considered a relatively short scale.
The results of our study showed that the Turkish version of RDQ was valid and reliable for our outpatient sample at the psychiatry outpatient clinic. Our study is a cross-sectional study evaluating cases under continuous treatment. Future studies that obtain premorbid functionalities for patients or that obtain RDQ scores for patients at initial admission into treatment will more clearly reveal the validity of the scale with respect to treatment outcomes.