Construct validity of the perceived stress scale (PSS-10) in a sample of early childhood teacher candidates

ABSTRACT OBJECT: The 10-item Perceived Stress Scale (PSS-10) is one of the most widely used psychological instruments for measuring stress perception in practice and research but has sparked some controversy regarding its factor structure. Further, no study has been conducted to date using a sample of Korean university students to test the reliability and validity of the PSS and the factorial structure of this instrument. Likewise, more data on the psychometric properties of the Korean version of the scale (K-PSS-10) are also needed. Hence, the purpose of the present study was to examine the construct validity of the K-PSS-10 among Korean university students. METHODS: A total of 250 university students currently attending the Department of Early Childhood Education (a four-year university curriculum) completed the K-PSS-10. The collected data were subjected to confirmatory factor analysis, which tested three alternative plausible models suggested by the extant literature: single factor, correlated two-factor and bifactor. RESULTS: The results indicated that the bifactor model best fit the data, with one general factor reflecting the overlap across all 10 items, and 2 additional specific factors representing perceived distress (6 negatively worded items) and perceived coping (4 positively worded items). Internal consistencies of the total scale and its two subscales were adequate (α > 0.70). CONCLUSIONS: The findings of this study support the use of the K-PSS-10 as a reliable and valid measure to assess perceived stress in a sample of early childhood teacher candidates.


Introduction
Stress was initially defined as a set of physiological and psychological reactions to any environmental threat to an individual's equilibrium [1]. Stress occurs when individuals face a situation which they perceive to be overwhelming and which they cannot cope with [2]. Increased exposure to stressors is often associated with a higher risk of physical and psychological impairment [3]. Perceived stress can be defined as individuals' feelings and thoughts about the level of stress they are experiencing currently or over a period of time. It generally incorporates feelings about the uncontrollability and unpredictability of one's life, with these irritants bringing about changes in one's life as well as one's confidence in his/her ability to deal with challenges or problems [4].
Academic stress and the psychological well-being of university students have attracted great scholarly interest for many years. The transition to university has proved to be a highly stressful life event for many young adults, as it comes with increased exposure to stressors and subsequent behavioural reactions [3]. Stressors permeate most, if not all facets of tertiary education life, including increased academic workload, new responsibilities, financial difficulties, interpersonal relationships, the long distance from home, and independence and autonomy [2,3]. Students who major in early childhood education (ECE) (i.e. referred to as "early childhood teacher candidates") have the same academic stressors as other university students, such as examinations and assignments. However, early childhood teacher candidates have especially unique stressors beyond those in university education. That is, they are required to undertake teaching practicum on top of a theoretical study load which worsens the already stressful environment. It has been frequently reported that the teaching practicum was rated as the most stressful study task for early childhood teacher candidates, which needs to be seen in the context of their stress levels strongly influencing their decision to stay in the teaching profession [5]. Practicum-related stressors experienced by early childhood teacher candidates include the gap between theoretical knowledge and field experience in the ECE profession, which includes the discrepancy between what is taught in the classroom and how nurture and support are actually provided in practice, the associated psychological burden, and the lack of preparation as a prospective early childhood teacher. Moreover, early childhood teacher candidates reported feeling inadequate in dealing with child behaviour and guidance issues and having difficult relationships with supervisors and colleagues [6,7].
Early childhood teachers are expected to be competent in supporting and guiding children in the early development of good physical and mental health habits. However, international research indicated that early childhood teacher candidates are at risk of both physical and mental health issues, such as increased levels of depression, anxiety, stress, and frustration [8]. Such issues might well jeopardize teacher candidates' pursuit of becoming positive role models for young children [8]. In fact, it has been well documented that early childhood teachers' psychological wellbeing shapes the nurturing and learning classroom climate in early care and education as well as children's development [9,10].
Because perceived stress is a risk factor for depression in teacher candidates and corresponds to reduced quality of nurturing and teaching, an appropriate assessment of perceived stress is critical for preventing further mental health issues. Hence, a reliable standardized measure of stress would be useful for clinicians or health professionals in university counselling services to alleviate teacher candidates' early feelings of stress and to assist in development of prevention programmes that enhance coping skills [3,11].
Several screening measures have been designed specifically to detect perceived stress in students, for example, the Perceived Stress Scale (PSS) [12]. Although stress measurements are required for screening and investigating interventions' effects, choosing the most appropriate measure for a specific application depends on many factors, including the study sample's characteristics and practical issues, such as respondent burden and the instrument's psychometric properties. Since these properties can vary among populations, results obtained from one population are not guaranteed to measure another accurately. Therefore, systemically assessing an instrument's psychometric properties prior to its widespread use on a specific population, differing from the original development population, is important [13]. Additionally, it is necessary to identify whether the shorter version's (i.e. PSS-10) psychometric properties are appropriate, in this case, for use with Korean university students experiencing stress, especially those undertaking ECE. Identifying the PSS-10 scale's psychometric properties could also help health professionals undertake a comprehensive assessment of early childhood teacher candidates who will potentially suffer or are already suffering from stress. Therefore, an informed measurement of perceived stress is necessary for understanding one's susceptibility to and treatment for pathological disorders [3].

PSS factor structure
The Perceived Stress Scale (PSS) is a global measure of stress that evaluates the degree to which individuals perceive aspects of their lives as unpredictable, uncontrollable, and overburdened [12]. The original version of the PSS was shortened from 14 to 10 items to improve completion rates and ease of scoring. Although both the 14-item (PSS-14) and 10-item (PSS-10) versions demonstrate strong psychometric properties, Cohen and Williamson (1988) revealed that the shorter version has higher internal consistency and construct validity and is thus psychometrically superior [14]. Consequently, Cohen and Williamson (1988) recommended that researchers use the PSS-10 over the original version.
The PSS-10 has indeed exhibited strong psychometric properties, possessing good reliability and construct validity among older adults [15,16], multicultural populations [17], and university students [3,11]. Theoretically, the PSS is a unidimensional measure of perceived stress [14], but analyses of the underlying factor structure of the PSS-10 have been generally inconsistent. For example, factor analyses on the PSS-10 have revealed a two-factor structure as the best model, with the PSS reflecting a two-dimensional construct consisting of perceived stress (distress) and perceived self-efficacy (coping) [3,18]. The twofactor model is consistent with confirmatory factoranalytic studies in clinical populations [19,20], older adults [15], and university students [11]. In contrast, a single-factor structure emerged from exploratory factor analysis (EFA) of data from a nonclinical US sample [14]. Studies have recently suggested that as an alternative to a two-factor model, a bifactor model comprising a two-factor structure with a single underlying general factor of perceived stress would best describe the PSS structure [20]. Bifactor modelling can assess whether the PSS-10 is sufficiently unidimensional to warrant the use of total scores, while allowing for additional specific variance resulting from reverseworded items [21]. The bifactor PSS-10 model has an underlying general factor representing commonality among all scale items (in this case, perceived stress) and two separate factors of perceived distress and perceived coping that explain variance independently from the general factor.
These contradictory findings may be partly due to (1) the use of different statistical techniques, (2) sample heterogeneity, (3) age differences in the study sample (older adults vs. college students), (4) participant characteristics (e.g. a clinical sample compared with a community-based sample), or (5) sample size. Methodologically, in such situations, confirmatory factor analysis (CFA) for this validation study is more appropriate, as it may provide a more definite and conclusive explorative factor analysis that assumes no a priori hypotheses. CFA is a statistical procedure used to verify a set of observed variables' factor structure and provide evidence of measurement validity. In CFA, alternative measurement models are constructed based on relevant theories and empirical research. The goal is to replicate the scale's original factor structure from the original study and retain the original model as much as possible. This involves testing and comparing several alternative (i.e. competing) models, all grounded in theory [22]. Alternative models are then explicitly tested statistically against sample data and fit indices are compared to assess how well each model fits the data according to goodness-of-fit indices. Thus, CFA allows a researcher to test how well measured variables constitute a given factor [23].
The Korean version of the PSS-10 (K-PSS-10) has been validated and used in a small number of studies on Korean adults [16,24]. However, to date, no research has been conducted on a sample of Korean university students from the ECE profession to provide data about the PSS-10's factorial structure. Thus, this study aimed to examine the factor structure of the K-PSS-10 with a Korean university ECE student sample (i.e. early childhood teacher candidates). This would involve testing three competing models, based on the preexisting ones, to determine which provides the most parsimonious fit to the data using a range of fit measures and CFA.

Participants
The study used a total sample of 250 undergraduate students enrolled in a private university in South Korea. All students in grades 1-4 were from the ECE Department. Students were between 18 and 31 years old, with a mean age of 21.4 (SD = 1.29). The majority of participants (90%) were females.

Procedure
This study was conducted in accordance with the provisions of the Declaration of Helsinki. Data were collected in the spring 2018 semester during normal instructional delivery. Before the start of the class, students were given access to a survey link and were asked to complete the online survey. The principal investigator and the second author were present at each administration to provide instructions and collect consent forms from the participants. The subjects were informed about questionnaire content and that participation was voluntary. They were not given any compensation for their involvement and were allowed as much time as they needed to complete the task. It took about 10-15 min on average for the participants to complete the questionnaires in class. No identifying information was collected, and the participants were assured of confidentiality.

Measure
The PSS-10 asks respondents about their thoughts and feelings over the last month using a five-point Likert scale (0 = never, 1 = almost never, 2 = once in a while, 3 = often, 4 = very often). Sample items include "in the last month, how often have you felt that things were going your way" and "in the last month, how often have you found that you could not cope with all the things that you had to do." Six negative items are scored in the non-reversed direction (i.e. "how often have you felt that you were unable to control the important things in your life"). Four positive items are scored in the reversed direction (i.e. "how often have you felt that things were going your way"). The total scores range from 0 to 40, with higher scores indicating greater perceived stress.

Statistical analysis
The statistical analyses in this study were performed with IBM SPSS and AMOS v20 [25].
The amount of missing data was minimal in this study, less than one percent of the total number of cases in the data set. Missing data occurred for two participants who either intentionally or unintentionally skipped or refused to answer one question. Missing data was replaced using the expectation maximization (EM) algorithm. The EM algorithm is a method of finding the maximum-likelihood estimates for model parameters from incomplete data that has missing values or unobserved latent variables [26].
CFA is a method used to test a specific hypothesis a priori by assessing whether there is a relation between observed variables and their underlying latent constructs [27]. This method is particularly useful in the early development of a questionnaire in refining items and assessing a questionnaire's construct validity [28].
The appropriate sample size for CFA is a complex issue, as it is generally considered for large sample methods (N > 500) due to the large number of parameters being evaluated (i.e. means and variances for all observed and latent variables, as well as regression weights and intercepts for all path weights) [29]. Although CFA can be viable with smaller samples, there is a greater risk that improper solutions such as ultra-Heywood negative error variance or inter-correlations (>1.00) will occur [30]. Nevertheless, a small-sample CFA can be facilitated by several possible approaches. One of the approaches used in the present study was a maximum likelihood (ML) estimation, which is robust for non-normality and can ensure estimable models. Hence, CFA with a maximum likelihood procedure was used to test a series of alternative (i.e. competing) models for the structure of the PSS-10 based on the relevant theories and empirical research.
The fit quality for each CFA model was determined by reference to a number of fit indices, including the chi square (χ²) and its subsequent ratio with degrees of freedom (χ²/df); comparative fit index (CFI); goodness-of-fit index (GFI); root mean square error of approximation (RMSEA); and standardized root mean square residual (SRMR). The chi-square test divided by the degrees of freedom should be less than 5 [31]. Although CFI values close to 1 indicate a very good fit, it is generally accepted that a value equal to or greater than .90 indicates an acceptable model fit [31]. GFI produces values ranging from 0 to 1, with values closer to 1 indicating a better fit [31,32]. RMSEA values of less than .05 indicate an excellent fit, though values between .05 and .08 indicate acceptable or reasonable errors of approximation [33]. SRMR values below .08 are considered indicative of an adequate fit [32].
Three models were tested and the fit indices were compared to assess how well each model fit the data. The first model is a one-factor model with all 10 PSS items loaded onto a single factor. The second model is a two-factor oblique solution (i.e. factors are correlated with one another) where six negatively worded items are loaded onto one factor as the "perceived distress" factor (items 1, 2, 3, 6, 9, 10), and the four positively worded items are loaded onto a separate factor as the "perceived coping" factor (items 4, 5, 7, 8). The final model is a bifactor model where all 10 items were specified to load onto a general factor with two additional specific factors (orthogonal) reflecting the unique variance not accounted for by the general factor. The bifactor model tested whether the PSS is a general measure of perceived stress with two specific underlying dimensions.
Chi-square difference tests were used to determine whether the models were significantly different from one another. Finally, after demonstrating that the measurement models worked, their descriptive statistics were computed.

Descriptive statistics
The means and standard deviations of the K-PSS-10 items are described in Table 1. The mean K-PSS-10 score for the sample was 19.7 (SD = 10.1). Pearson correlation coefficients between perceived distress and perceived coping were −.30. These low to moderate correlations suggest that the two subscales represent two related yet distinct components of perceived stress. Cronbach's α was calculated for the total K-PSS-10 items, perceived distress and perceived coping items. Reliability estimates (α) of the total scale in the current sample was .79, .82 for the perceived distress factor and .70 for the perceived coping factor. All skewness and kurtosis values for all of the study variables were found to be well within acceptable ranges (i.e. +/−3.00). Table 2 summarizes the fit indices for the alternative CFA models. The results indicate that the single factor for the PSS-10 scale demonstrated an unacceptable fit to the data (χ 2 = 473.2, df = 35; χ 2 /df = 13.5; CFI = 0.60; GFI = 0.74; RMSEA = 0.224; SRMR = 0.194). The two-factor model fits the data marginally better than the single-factor model, as evidenced by the decrease in the χ 2 -value and the improved CFI, GFI, RMSEA, and SRMR; however, the fit indices did not meet the accepted fit criteria (χ 2 = 356.5, df = 34; χ 2 / df = 10.5; CFI = 0.71; GFI = 0.79; RMSEA = 0.103; SRMR = 0.097). In contrast to the single-and two-factor models, the bifactor model was judged to fit best with the early childhood teacher candidate sample (χ 2 = 125.6, df = 26; χ 2 /df = 4.8; CFI = 0.90; GFI = 0.91; RMSEA = 0.069; SRMR = 0.075). The χ 2 /df value for the bifactor model was smaller than those for the single-and two-factor models-another indication of a good fit. Furthermore, the χ 2 difference tests revealed that the bifactor model was a significantly better fit to the data than the single-factor model (χ 2 (3) = 347.6, p < 0.001) and the two-factor model (χ 2 (3) = 230.9, p < 0.001). Table 3 shows that factor loadings were all in the expected direction, and all standardized factor loadings were statistically significant exceeding 0.30. Further inspection of the factor loadings for the two subscale factors (perceived distress and perceived coping) provides critical information regarding the appropriateness of incorporating these subscales in the scoring of the PSS-10. In situations when items load highly or strongly onto a general factor but simultaneously load onto subscale factors to a lesser degree, this supports the consideration of a unidimensional scoring method [21]. Alternatively, when items load more strongly onto subscale factors than they do on the general factor, the creation of subscales is appropriate. In terms of subscales, factor loadings for perceived distress were weaker than for the general factor, but perceived coping showed adequate factor loadings. These produced convergent results supporting the superiority of a single latent factor underlying the PSS-10 and, to a lesser extent, favour the presence of two subscales. Figure 1

Discussion
The growing concern over the impact of perceived stress on early childhood teacher candidates (i.e. students who are majoring in ECE), coupled with the popularity of the PSS-10 as a generalized stress measure for other populations, motivated the current effort. The current study sought to investigate the construct validity of a Korean translation of the PSS-10. A total of three competing models were specified and tested: a single-factor model, a two-factor model, and a bifactor model. In general, the results of the CFA provide preliminary support to the validity and reliability of the PSS-10 in a Korean university sample.
On the basis of fit indices from the CFA, the bifactor model, which reflected a unidimensional construct with two specific factors (perceived distress and perceived coping), was considered to provide a sufficient Table 2. Goodness-of-fit indices of models for the PSS-10 (N = 250).   and better fit to the data than the alternative models. Furthermore, higher item loadings were evident for a general factor than perceived distress and, to a certain extent, perceived coping factor. These findings suggest that while perceived distress and perceived coping factors exist separately, the PSS-10 is driven by a single underlying dimension of the former. Ultimately, these results are consistent with those of previous studies [20,34] and indicate that both the total and subscale scores of the PSS are usable, as the measure is found to be sufficiently unidimensional. This is a significant outcome that substantiates the unidimensionality assumption of the PSS-10, supporting Cohen and Williamson's (1988) original validation report confirming its unidimensionality. Additionally, the unidimensionality assumption of the PSS-10s has been backed by previous studies that used both EFA and CFA [3,18], as well as alternative psychometric methods. For example, Medvedev et al. (2017) conducted Rasch analysis and provided clear evidence of the scale's unidimensionality [35]. As previously mentioned, past examinations of the PSS factor structure have favoured and consistently supported a two-factor solution and have posited that the factor distinction is insignificant because of the superficial nature of reversed wording (four positively worded items) [14]. However, none of the fit indices showed an improvement over those observed for the bifactor model. These results conflict with those of Martin et al. (1995), who asserted that the PSS measures two substantially distinct facets of stress [36]. An outcome that should be acknowledged, however, is the observation that the items established relatively high factor loadings on perceived coping in the bifactor model. This may be because reversed wording (i.e. perceived coping), which is intended to reduce biased responses, can act as a nuisance variable that introduces some undesired source of variation during PSS scoring [17]. This may explain why perceived coping items had high loadings in the current study. To deal with this, Perera et al. (2017) suggested that obtaining a score based on all 10 items and scores based on the four reverse-worded items may help control for this potential variance when administering the PSS.
The reliability estimates of the PSS-10 (all 10 items) and its two subscales demonstrated adequate internal consistencies, meeting the cut-off of α > 0.70 in the current sample. The reliability estimates of the PSS and its two subscales are consistent with those in previous studies and even higher than those reported in others.
The study supports the ease of use of the PSS-10 in a university student sample. Although it is not a measure of psychological symptomatology, the PSS-10 may be used to determine those at risk for particular clinical disorders [14]. For example, the measure can be applied within the university setting as a screening tool to identify those who need further support [3]. Students and early childhood teacher candidates, in particular, deemed at risk because of high perceived stress, can then be referred to university counselling services. Using the PSS-10, both researchers and counsellors working with early childhood teacher candidates would obtain data to address many different issues associated with the impact of stress on them. Universities should aim to reduce stressors and provide psychosocial and academic support systems to alleviate the stress of early childhood teacher candidates. Prompt diagnosis and early detection certainly plays an important role in such an endeavour.
The results need to be interpreted in light of several limitations. First, the construct validity examination was limited to the PSS-10 and not the original PSS-14. However, the psychometric properties of the former were found to be superior to those of the latter and its use has increased globally in recent decades [14,17]. Next, the small sample of students from only one ECE Department in Korea may have affected the study's generalisability. Therefore, our findings are not applicable to students in other degree programmes. Replication with a larger, randomized sample would expand the knowledge on perceived stress among early childhood teacher candidates. In line with many other studies on psychometric properties, the data was based on self-report questionnaires, the responses to which might have been swayed by social desirability. The findings would therefore be strengthened through interviews and physiological assessments.
In conclusion, these findings provide further support for the reliability and validity of the K-PSS-10 and contribute to research literature regarding its factorial structure. Considering that many early childhood teacher candidates continue to experience high levels of stress, the K-PSS-10 could potentially be used in a wide range of future studies on stress experienced by teacher candidates. Moreover, in agreement with the previous research [20,34], the support for a bifactor model in the current study provided evidence for sufficient unidimensionality of the PSS-10. This suggests that the K-PSS-10 and its total scores can appropriately assess stress of the early childhood teacher candidates.

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.