Development of self-stigma inventory for patients with schizophrenia (SSI-P): reliability and validity study

ABSTRACT OBJECTIVES: Internalizing the public stigma in patients with schizophrenia leads to self-stigmatization associated with a number of negative consequences such as depression, low self-esteem, hopelessness, impairment of social adaptation, unemployment, and treatment non-adherence. No instruments have been developed to assess the self-stigmatization for patients with schizophrenia living in Turkey. The purpose of this study was to develop a culturally-sensitive and easy-to-use instrument to measure self-stigma of the patients with schizophrenia and schizoaffective disorder. METHODS: After examining the existing stigma and self-stigma scales for people with mental illnesses, a 19-item self-stigma inventory was formed. Focus group interviews were conducted with patients with schizophrenia and the items were reviewed and rephrased into more comprehensible statements for the patients. The pilot study was conducted with a sample of 15 patients with schizophrenia, and the inventory was given its final form, self-stigma inventory for patients (SSI-P). Outpatients with schizophrenia and schizoaffective disorder were given sociodemographic form, SSI-P, Beck Depression Inventory (BDI), Internalized Stigma of Mental Illness (ISMI), Rosenberg Self-Esteem Scale (RSES), Beck Hopelessness Scale (BHS), Positive and Negative Syndrome Scale (PANSS), Clinical Global Impression-Severity (CGI-S), and Global Assessment of Functioning (GAF). For reliability analyses; internal consistency, item-total correlation, and test-retest reliability were assessed. Validity analyses were conducted with Explanatory Factor Analysis and convergent validity. RESULTS: The sample of the study was 162 patients with schizophrenia and schizoaffective disorder of which 77% were males, 70% were single, mean age was 37, and level of education was 10 years. Cronbach’s alpha coefficient of the scale was 0.93 ranging among the subscales between 0.60 and 0.91. Kaiser-Meyer-Olkin value was 0.91, and the Barlett test was significant (p < 0.001) for explanatory factor analysis and three factors were found (perceived devaluation, internalized stereotypes and social withdrawal, concealment of the illness) that can explain 63.5% of the total variance. Two items were removed because of their low factor value, and the final form consisted of 17 items. SSI-P was highly correlated with commonly-used stigma scale ISMI (r = 0.73), and moderately correlated with BDI (r = 0.53), BHS (r = 0.40), and RSES (r = −0.59). It also showed low correlation with PANSS negative score (r = 0.19). The test-retest reliability coefficient of the scale was 0.83. CONCLUSION: SSI-P is a reliable and valid instrument for assessing the self-stigmatization of patients with schizophrenia and schizoaffective disorder. The scale is an easy-to-comprehend, user-friendly, and culturally-sensitive tool with its 17 items.


Introduction
Stigma is generally defined as possession of some negative attributes causing the individual to be labelled as a member of a discredited social category which results in loss of social status and experience of discrimination [1]. People from diverse social groups or ethnic background struggle with stigmatization in everyday life and one of the most stigmatized group is people with mental illnesses [2]. Patients with mental disorders have sometimes been labelled as "nuts", "insane", "crazy", "psycho", etc. in public view, and they are challenged by the heavy burden of these negative labels [3]. Mental illness stigma is especially destructive for people with severe mental illness (SMI) since they need to cope with both the symptoms of the disease and also the discrimination and prejudice rooted against them in the community [4].
Stigmatization of people with SMI (i.e. schizophrenia, schizoaffective disorder, bipolar disorder, etc.) has several adverse outcomes for the patients [5,6]. Researchers indicate that social stigma restricts the patient in terms of finding a job, living independently, and establishing interpersonal relationships [7]. Because of discrimination, negative and low expectations, employers do not want to hire a person with a past or current diagnosis of SMI, landlords are reluctant to rent their apartments out to a patient, and people, in general, are less likely to be engaged or interact with a mentally ill patient [8]. Consequently, patients feel excluded from society, become depressed and lonely, and espouse the stigma of others [9,10]. When patients internalize these public attitudes, selfstigma occurs with numerous negative consequences. Self-stigma is another major challenge for the patients with SMI which includes not just being aware of the public stigma but also accepting the stereotypes and agreement with the prejudiced beliefs which in the end results in negative emotional reactions (e.g. low self-confidence) and behaviours in response to discrimination (e.g. failing to continue school or job) [11,12]. Studies revealed that self-stigma is related to personal autonomy, quality of life, negative symptoms, unemployment, recovery, self-esteem, self-efficacy, and treatment adherence [13][14][15][16][17][18][19]. Since the stigmatization is associated with one's insight into the illness [20], this phenomenon is essential concerning the therapeutic alliance and adherence to treatment.
Among the SMI, schizophrenia is one of the most discriminated and stigmatized mental illnesses [21,22]. Because of its chronic nature, compliance and adherence to the treatment are crucial for the patients with schizophrenia [23]. As an example, researchers conducted a study investigating the factors related to the medication compliance of the people with schizophrenia and found a significant relationship between self-stigmatization and medication compliance. It was revealed that patients with high level of self-stigma concerning the agreement with the stereotypes of mental illness had poorer medication adherence [24]. In that sense, assessing the self-stigma of the patients with schizophrenia is essential regarding illness prognosis and psychotherapy practices [25].
Self-stigma comprises several components such as: stereotyping, labelling, discrimination, social withdrawal, and status loss [2]. In this regard, scales assessing the self-stigma focus on one or more of those factors. Some of the most widely used scales are the Internalized Stigma of Mental Illness (ISMI) [26,27], Self-Stigma of Mental Illness Scale (SSMIS) [28], Self-Stigma Questionnaire (SSQ) [25], Perceived Devaluation and Discrimination (PDD) Scale [8], Discrimination and Stigma Scale (DISC) [29], and Consumer Experiences of Stigma Questionnaire (CESQ) [30]. These scales were originated from various countries; thus, every stigma scale includes some cultural statements along with universal concepts. Concerning the stigma term, it was argued that stigmatization has specific cultural aspects in which it should be evaluated by considering the community and the culture it is originated [31].
Since the stigmatization is a culture-specific phenomenon [32,33], researchers from different countries tried to develop a new scale or adapted the existing scales into their culture. However, most of the current scales were developed in Western-European cultural group and researchers from other cultures mainly translated those commonly used scales in their languages [34]. Relatively few studies focused on the culture-specific aspect of stigmatization and tried to construct a measurement that contains cultural features of its society [35].
In Turkey, the most commonly used measure assessing internalized stigma in mentally ill patients is a translated scale originating from the Western culture [27]. A culturally specific self-stigma scale developed for this community is still lacking, to our best knowledge. To be able to evaluate the self-stigma properly, it is crucial for the patients to understand the questions and that the items should reflect the stigma they experienced in their cultural context. The purpose of this study was to develop a culture-sensitive self-stigma inventory for the patients with schizophrenia living in Turkey and to measure its psychometric properties.

Participants
In this study, 162 outpatients diagnosed with schizophrenia or schizoaffective disorder according to DSM-5 criteria [36] were recruited from Kocaeli University School of Medicine Psychiatry Polyclinic between September 2016 and July 2017. The participants of this study were the patients of the families participated in our previous study of the "development of Self-stigma Inventory for Families" that was approved by the ethical committee for both the patients and their relatives [37]. Ethical permission of the study was taken from Kocaeli University Ethical Committee of Non-invasive Clinical Research (KÜ GOKAEK 2016/61). Researchers gave information to the patients about the study, and informed consent form was given to those who agreed to participate.

Inclusion criteria
Participants were included to the study from those who were 18-60 years of age, not having mental retardation or any neurological disease that can affect their judgment, at least primary school graduate, continuing to their regular medical treatment, and not receiving ECT in the last 6 months.

Sociodemographic form
Patients were given a form developed by the researchers containing their sociodemographic information such as gender, age, marital status, education, employment, the age of onset of the illness, duration of the illness, and the number of hospitalizations.
The Global Assessment of Functioning (GAF) is an instrument included in the axis V of DSM-IV-TR mainly aims to assess the patients' general functioning on a scale range from 0 to 100. Higher scores indicate better performance [38].
Positive and Negative Syndrome Scale (PANSS) is a scale for rating the symptoms of the schizophrenia (and other psychotic disorders). It assesses the symptoms of the patients as positive, negative, and general psychopathology with its 7-point rating and 30-items [39,40].
Clinical Global Impression-Severity (CGI-S) is a clinical global assessment scale where the clinician rates the severity of the illness following seven-point scale: 1 = normal, not at all ill; 2 = borderline mentally ill; 3 = mildly ill; 4 = moderately ill; 5 = markedly ill; 6 = severely ill; 7 = among the most extremely ill patients [41].

Internalized stigma of mental illness (ISMI) scale
The scale was developed by Ritsher et al. [26] and adapted to Turkish culture by Ersoy and Varan [27]. It is a 4-point Likert type questionnaire with 29-item assessing the self-stigma among the people with mental illness. Cronbach's alpha coefficient of the scale was 0.90 and 0.93 for original and Turkish versions, respectively. It has 5 subscales defined as Alienation, Stereotype Endorsement, Discrimination Experience, Social Withdrawal, and Stigma Resistance. The total score is calculated by grading the "strongly disagree" as 1, "disagree" as 2, "agree" as 3, "strongly agree" as 4, and reverse the coding for the items in the last factor. Minimum and maximum scores of the scale are between 29 and 116. Higher scores indicate a higher level of internalized stigmatization.
Beck Depression Inventory (BDI) was developed by Beck et al. [42] to assess physical, emotional, and cognitive symptoms observed in depression and the study of its Turkish adaptation was conducted by Hisli [43]. It is a 21-item self-assessment scale. Cronbach's alpha coefficient was found as 0.90. Higher scores indicate a higher level of depression.
Beck Hopelessness Scale (BHS) is a 20-item scale developed by Beck, Lester, and Trexler [44], and adapted to Turkish culture by Durak [45]. Internal consistency of the scale was found as 0.86. Higher scores from the scale indicate the higher level of hopelessness.

Rosenberg self-esteem scale (RSES)
In this study, the 10-item Rosenberg Self-Esteem Scale [46] was used. Turkish reliability and validity study of the scale was conducted by Çuhadaroğlu [47]. Internal consistency of the scale was 0.71. Higher scores indicate higher self-esteem.

Self-stigma inventory for patients (SSI-P)
Existing stigma and self-stigma scales for people with mental illnesses in both English and Turkish language were examined, and items were evaluated. By taking into consideration the Turkish culture and stereotypes embedded in, a 19-item self-stigma questionnaire was formed. Then, focus group interviews were conducted with 20 patients with schizophrenia in which patients talked about their opinions, thoughts, and beliefs about the stigma in the society and the ones that they accepted for themselves as a form of internalized stigma. Collecting the information from patients themselves and integrating the statements of the patients about their self-stigmatization into the scale, the items were reviewed and rephrased into more comprehensible terms for the patients. The pilot study was conducted with a sample of 15 people with schizophrenia, and the inventory was finalized after reevaluating the incoherent phrases. Answers to each item were rated with a 5-point Likert scale: 1 = do not agree, 2 = slightly agree, 3 = moderately agree, 4 = generally agree, 5 = totally agree. Higher scores indicate the higher level of stigmatization.

Procedure
The SSI-P was given to the outpatients with schizophrenia and schizoaffective disorder who gave informed consent to the study together with a sociodemographic form, ISMI, BDI, RSES, and BHS. A psychiatrist also evaluated the patients with the PANSS, CGI-S, and GAF scales. To assess the test-retest reliability, the SSI-P was given to a group of the selected patients 2-3 weeks after the first administration.

Statistical analysis
Analyses were conducted with SPSS version 22.0. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett's tests of sphericity were carried out to check preliminary assumptions, and the Principal Components Analysis and Direct Oblimin Rotation were utilized for factor analysis. For internal consistency, corrected item-total correlations, Cronbach's alpha, and Cronbach's alpha if item deleted were calculated. For test-retest reliability and concurrent validity, Pearson or Spearman correlation was used based on the distributional features of the variables.

Results
One hundred and sixty-two outpatients with schizophrenia and schizoaffective disorder participated and completed the study. Majority of the patients were males, single, and living with their parents. Sociodemographic information of the patients was given in Table 1.
The validity of the SSI-P

Construct validity
Explanatory Factor Analysis (EFA) was conducted to examine the construct validity. The KMO test indicated excellent sampling adequacy (KMO= 0.920), and Bartlett's test of sphericity showed that a factor analysis might be useful for the data (X 2 = 1877.47, df = 171, P < 0.001). In the EFA of the 19-item scale, 3 factors were found which can explain 61.47% of the total variance and has eigenvalue greater than 1. Factor analysis was conducted with Direct Oblimin technique, and the items with factor value less than 0.40 (item number 9, 12) were removed. Thereby, the scale had three factors and 17 items. It was revealed that first factor explains 49.51%, the second factor explains 7.10%, and the last factor explains 6.87% of the total variance, and; together they explain 63.47% of the total variance. In terms of the items constitutes the factors; first factor (1, 2, 3, 4, 6, 7, 10, 11) was labelled as "perceived devaluation", second factor (8,14,15,16,17,18,19) described as "internalized stereotypes and social withdrawal", and third factor (5, 13) entitled as "concealment of the illness" (Table 2).

Content validity
Content validity was assessed with the correlation between the total score of the scale and its subscales ( Table 3). A high correlation was found between the SSI-P total score and perceived devaluation subscale (r = 0.92), internalized stereotypes and social withdrawal (r = 0.90), and concealment of the illness subscale (r = 0.66).

Concurrent validity
Concurrent validity of the scale was calculated by its correlation with the BDI, BHS, and RSES ( Table 4). The SSI-P and its subscales were significantly   Table 3. Correlations between the SSI-P total score and factor scores. correlated with all the scales; positively with BDI and BHS, negatively with RSES. Spearman correlation was carried out to assess the correlation of the scale with clinical assessment tools. The SSI-P was not significantly associated with PANSS total, PANSS positive symptoms, PANSS general psychopathology, GAF, and CGI-S scores (P > 0.05). The only correlation was found between PANSS negative symptoms and SSI-P total scores (P < 0.05), yet the magnitude of the association was quite low (r = 0.20).
The other scale used as convergent validity was ISMI. Table 5 shows the correlations between the ISMI and SSI-P total scores and subscales. A high positive correlation was found between ISMI and SSI-P total scores. The correlations between the subscales were also quite high except the "concealment of the illness" factor from the SSI-P and "stigma resistance" subscale from the ISMI.

Internal consistency reliability
In the internal consistency analysis, Cronbach's alpha internal consistency coefficient of the scale was calculated as 0.932 for the SSI-P with 17 items. Cronbach's alpha coefficient of perceived devaluation factor was 0.91, internalized stereotypes and social withdrawal factor was 0.87, and concealment of the illness was 0.60. Data concerning the item-total correlations and Cronbach's Alpha coefficients calculated for each item through if item deleted technique (Table 6). Item-total score correlation coefficients were between 0.31 and 0.76, and all were statistically significant (P < 0.001).

Test-retest reliability
For the test-retest reliability analysis, the SSI-P was given to 30 patients in 2-3 weeks after the first   administration. The results were analyzed through the Spearman correlation test. The test-retest reliability coefficient of the scale was r = 0.829 (P < 0.01).

Association between the SSI-P and sociodemographic variables
The mean SSI-P scores for the patients with schizophrenia and schizoaffective disorder were calculated as 37.6 (SD = 16.58), and 39.1 (SD = 20.40), respectively, and the average total scores did not differ statistically according to the diagnostic groups (P = 0.68). Moreover, the mean SSI-P total scores were not statisti- .60, P = 0.09). Furthermore, SSI-P was not significantly correlated with the age (P = 0.58), level of education (P = 0.44), the onset of the illness (P = 0.10), duration of the illness (P = 0.29), and the number of hospitalizations (P = 0.79).

Discussion
The results of the study showed that the SSI-P is a reliable and valid instrument for assessing the self-stigmatization of people with schizophrenia and schizoaffective disorder. The Cronbach's alpha correlation coefficient value of the scale was 0.93 indicating high reliability. The factor analysis revealed that the scale has excellent construct validity with its 3 factors called perceived devaluation, internalized stereotypes and social withdrawal, and concealment of the illness. The SSI-P total score was highly correlated with its factors indicating the content validity of the scale.
Concerning the convergent validity, the SSI-P was significantly correlated with the BDI, BHS, RSES, and ISMI. The relationship of self-stigma with depression, hopelessness, and self-esteem was also revealed in previous studies [28,[48][49][50] which supports our results of the moderate correlation. These studies found that selfstigma is positively correlated with depression, hopelessness, and negatively correlated with self-esteem. Some studies also found a relationship between the severity of the illness (CGI-S) and self-stigma [51]. However, we did not find a significant correlation between SSI-P and CGI-S in this study. The reason might be the difference between SSI-P and other scales concerning the content and context of the stigma items; also, the cultural differences might affect the relationship between the severity of the illness and self-stigmatization of the patients. Additionally, while some researchers found that both positive and negative symptoms of schizophrenia were associated with selfstigma [52]; others found that only negative symptoms directly affected the hopelessness, depression, and selfstigma [48]. Our study supported the latter research by revealing a positive correlation only between PANSS negative symptoms and SSI-P total score. Although it was a small correlation, it is noteworthy that the correlations differ according to different kind of stigma scales. Furthermore, we could not find any association between GAF and SSI-P. In SSQ study, they found a small correlation between GAF and SSQ at a p-value of 0.047 (r = 0.23), which is very close to non-significance level. In that sense, it can be concluded that general functioning is not strongly associated with the level of self-stigmatization of the patients. Apart from that, SSI-P did not statistically change according to the sociodemographic information of the patients such as gender, marital status, employment, etc.
In Turkey, the most commonly used scale for selfstigma is the ISMI, and our new scale was strongly correlated with this scale (r = 0.73, P < 0.01). It was revealed that both ISMI and SSI-P have Cronbach's alpha value of 0.93; meaning the new scale is as reliable as the commonly used one. The advantage of the SSI-P is that it has high internal consistency with much fewer items compared to ISMI. Additionally, the SSI-P was constructed together with the patients by considering of their beliefs and thoughts; thus, the items are more comprehensible for the patients, and it is the first culturally sensitive scale that was developed for this culture. In our country, existing stigma scales were all translated versions of Western-origin scales. Therefore, this new scale is imperative in terms of containing culture-specific items and being sensitive to collectivistic social virtues [53]. Since the scale has only 17 items and no reversed items, its calculation is also quite easy. One of the most significant advantages of the SSI-P is its reader-friendly and easy-to-comprehend structure; because we included the statements directly from the patients themselves to the scale as items. One difference between the ISMI and SSI-P was that in SSI-P there were no items related to the Stigma Resistance factor in ISMI. We did not include any statement from this specific factor since that kind of topic was never brought up by the patients during the meetings or interviews we conducted about their self-stigmatization. We concluded that statements related to stigma resistance need not be included in the scale since our patients did not have the concept of "resistance" when they think about the self-stigma. That was the reason that every factor of the SSI-P and ISMI were moderate to strongly correlated with each other except the Stigma Resistance factor of ISMI.
The explanatory factor analysis revealed that the SSI-P has three factors, and the factors could explain 63.5% of the total variance. The first factor comprised of the items related to the patients' thought of inability to take responsibilities, feelings of uselessness, feeling like a burden to their family, having low self-confidence, thoughts about people avoiding them, thinking that others see them as "sick", thinking that they cannot take proper decisions on their own, and the feeling of a drug addict. These items were categorized as perceived devaluation. Although some of the items were placed in "social discrimination" and "perceived capabilities" factors in another scale [24]; they gathered into one factor in our study. We concluded that those negative feelings and attitudes of the patients were derived from the perception of devaluation they had.
In the second factor, items about patients' unemployment, thought of being unable to get married, social withdrawal from their environment, and thoughts of others not caring them clustered into same factor merging with the items including the feelings of being unsuccessful and unhappy. This factor was labelled as internalized stereotypes and social withdrawal. The fact that stereotypical thoughts could not be differentiated as a separate factor might be the result of that the failure and unhappiness are closely associated with social withdrawal in our culture. It was interpreted that stereotype endorsement and internalization also lead to withdrawal from the society as well for the patients with schizophrenia in Turkey. In the third factor, hiding the name of the illness from their close circle, environment, and their friends are defined as concealment of the illness. These items were also included in other self-stigma scales as in the same factor [25]. Moreover, in our previous study, we similarly found three factors (social withdrawal, concealment of the illness, and perceived devaluation) that could explain 66.8% of the total variance of the Self-Stigma Inventory for Families (SSI-F) [37].

Limitations of the study
This study has some limitations. Firstly, the study was conducted in a single-centre facility and with the patients who are compliant with their medical treatment. Thus, it may have affected its external validity, and the study might not be generalized for those unstable and non-adherent patients. Secondly, the study was carried out only with the people with schizophrenia and schizoaffective disorder. Its reliability and validity should also be investigated for other psychotic disorders and severe mental illnesses in general. Thirdly, we used the ISMI to evaluate the convergent validity of the scale; however, it would be better if the scale was compared with another reliable, valid, and culture-sensitive instrument developed in the same culture. Nonetheless, since this was the first self-stigma scale developed for this culture, we had to use a translated stigma scale.
Despite the limitations, the SSI-P was the first culture-sensitive, a user-friendly scale that was aimed to assess the self-stigmatization of the patients with schizophrenia and schizoaffective disorder living in Turkey. We believe that this scale can also be adapted to different cultures and languages since it includes the essence of the stigma concept and comprehensible to the patients' self-evaluations. Furthermore, future studies should be done to replicate the results and further validate the inventory. Although this study stands for the preliminary results of the new scale, it can be said that it is a useful and practical tool in assessing self-stigmatization of the patients with schizophrenia regarding clinical practices and therapeutic purposes.