Psychological Resources Program - An intervention to foster psychological resources: Evaluation of results in the Brazilian population

Abstract A quasi-experimental trial was conducted to evaluate the effects of the Psychological Resources Program, an intervention to foster psychological resources for adults, on self-reported psychological-resources-related measures. Participant’s satisfaction with the intervention and program attrition were also specific aims. The Psychological Resources Program, which claims to be a preventive tool, aims to improve psychological resources. It consists of 10 weekly group sections that each last approximately 120 min. Before delivering the program, the intervention manual was translated from the original in Spanish to Brazilian Portuguese and facilitators were trained. Twenty-seven undergraduate students completed the program at university and were evaluated before, after, and at a three-month follow-up. The results showed that (a) perceived mood state improved week-by-week, and (b) improvement in positive affect [PANAS], gratitude [GQ–6], optimism [LOT—R], resilience [RS], general mental health [GHQ–12], assertiveness [E3], and satisfaction with life [SWLS], and decreases in perceived stress [PSS-14] and negative affect [PANAS] (effect sizes from 0.44 to 0.75). These changes, although attenuated, were maintained at the follow-up (3 months after the end of the intervention). The participant satisfaction with the program was high. The attrition rate (35%) was within the range expected for the type of intervention and context. The results support that the program represents a potential tool for prevention and health promotion in the university context. Continued evaluation of the intervention effects are recommended.


PUBLIC INTEREST STATEMENT
An intervention to foster psychological resources was evaluated to find its usefulness to improve mental health in Brazilian university students. The intervention consists of 10 weekly group sections that each last approximately 120 min. The sessions focus on specific activities to develop a particular psychological resource. Activities are pre-established following the program manual. Twenty-seven undergraduate students completed the program at university and were evaluated before, after, and at a 3-month follow-up. The results showed that the perceived mood state improved week-by-week. Improvement in positive affect, gratitude, optimism, resilience, general mental health, assertiveness, and satisfaction with life was observed. Decreases in perceived stress and negative affect were also reported. These changes, although attenuated, were maintained at the 3 months follow-up. The participants' satisfaction with the program was high. The results support that the program represents a potential tool for prevention and health promotion in the university context.

Introduction
Decades of research have found that psychosocial factors and personal resources are key-points to effectively increase behavioral health and prevent illness (Hodges & Clifton, 2012;Idan et al., 2017;Weismann & Hannoch, 2011). In the past decades, research have implemented interventions built on these positive psychological constructs to aim health promotion and prevention. Although many of these interventions developed lacked of empirical contrast of their effects (Seligman et al., 2005), with the empirically supported intervention movement growing, studying of the effectiveness of an intervention showed to be essential. One classical example is found in Seligman et al. (2005) when they conducted a randomly-assigned, placebo-controlled study to test several positive interventions that they had developed. Also, the study of Senf and Liau (2013) using positive interventions, which focus on strengths instead of deficits, evaluated the effects of gratitude and strengths-based interventions on happiness and depressive symptoms against a control group. A detailed review on strengths-based psychological interventions to foster health and wellbeing can be found elsewhere (see Hervás et al., 2008).

The Psychological Resources Program antecedents
The Psychological Resources Program: an intervention to foster psychological resources (Programa +Recursos: Programa para la potenciación de los recursos psicológicos, in the original) is a manualized intervention developed by Remor and Amorós-Gomez (2013) in Spain. It aims to empower and foster psychological resources. Elaboration of the Psychological Resources Program commenced in 2003 through an extensive literature review performed by the authors. The review focused on determining what psychological resources are most related to protective effects on mental and physical health (Remor & Amorós-Gomez, 2013). Psychological resources are skills, beliefs, and individual personality factors that influence how people manage stressful events. These psychological resources become especially important when people are faced with challenging or threatening events (Taylor et al., 2000). Additionally, the theoretical perspectives that guided the program development included the health education and promotion model (Albee, 1982), salutogenic model (Antonovsky, 1996), and psychosocial resources model (Taylor & Broffman, 2011). After the literature review, the authors composed the intervention and submitted it to a panel of experts who provided constructive feedback on the content. The intervention was then reviewed, and the first version of the program manual was prepared (details on these procedures can be found in Remor & Amorós-Gomez, 2013). In its final version, the chosen psychological resources were: (a) empathy (the ability to understand the emotional state of another person and to imagine the feelings experienced by it, as well as the ability to place oneself in another's position; Eisenberg, 2000); (b) assertive behavior (refers to interaction and communication with other people; the person is able to express feelings and desires through proper communication, and this endeavor results in elevated satisfaction and a greater chance of achieving the desired goals; it includes several components such as voice tone, body posture, and facial expression; Nunes & Hutz, 2007); (c) optimism (refers to generalized outcome expectancies that good things, rather than bad things, will happen; Brissette et al., 2002); (d) positive coping (the use of cognitive and behavioral resources to deal with a challenging situation and development of strategies to deal with the triggered emotional demands; Folkman et al., 1986); (e) forgiveness (a process in which those who felt offended adopt a benevolent posture rather than nourish feelings of resentment or hatred; also refers to the process in which the person acts benevolently with her-/himself, forgiving eventual faults and perceived errors; Allen & Leary, 2010); (f) gratitude (understood as an experience, feeling, or attitude towards life, it makes the person more prone to perceive the positive aspects of her/his life and thus experience positive feelings; Emmons & McCullough, 2003); (g) relaxation skills (a method, process, procedure, or activity that helps a person to relax, attain a state of increased calmness, or otherwise reduce levels of negative emotions; Amutio-Kareaga, 1999).
The Psychological Resources Program claims to be a preventive intervention. It consists of 10 weekly group sessions that each last approximately 120 min. It targets adults older than 18 years who are capable of reading and writing; it can be delivered by professionals with different backgrounds (e.g., psychologists, educators, or other health-care professionals) as long as they receive proper training regarding the program. The intervention can take place in different locations, including hospitals, schools, universities, companies, or community centers. This program also incorporates group dynamics, interaction among participants, feedback, positive reinforcement on behavior, activities that involve creativity, problem solving, assertive communication, personal reflection, learning through experiences, relaxation training, and tasks completed during the week by the participant (action plans). Each session has specific topics (described in detail in the program's manual) that over the course of 10 weeks provides knowledge and develops psychological resources (Remor & Amorós-Gómez, 2018). A more detailed description of the program development process and background can be consulted in its published manual.
Existing Psychological Resources Program results indicate its effectiveness in improving physical and psychological well-being-related measures. Although the performed studies were quasi-experimental, they included different populations and replications, factors that minimize potential threats to internal validity associated with the design. The first published study consisted of three different groups that replicated the intervention (Remor et al., 2010). All groups showed an improvement in the mood state after completing all sessions. The first group consisted of 22 Spanish university students. The group showed significant changes in optimism, sense of humor, vitality, anxiety, insomnia, social dysfunction, and satisfaction with life (all outcomes were measured by validated instruments). The second group consisted of 14 patients with multiple sclerosis; they exhibited a decrease in stress experience, anxiety, insomnia, depression, and subjective health complaints, although these effects did not remain in the three-month follow-up. The third group consisted of 24 El Salvadoran university students. In this replication, a nonequivalent control group was added to the research design. The group that received the intervention presented a significant decrease in perceived stress, depression, and somatic symptoms when compared to the control group. A second study (Remor & Amorós Gómez, 2012) was performed with 25 Spanish university students, with pre-test and post-test measures. As in the previous study, the participants experienced an improvement in their mood state after the program sessions. There was also a significant improvement in dispositional optimism and satisfaction with life and a reduction in subjective health complaints. The described evidence highlight the potential of the program as a tool to improve psychological resources and buffer negative outcomes.
The adaptation process to apply the Psychological Resources Program in Brazil included translation of the manual by the author (E.R.) and a pilot study that included 13 university students. This pilot study evaluated the feasibility of the intervention, acceptance, and satisfaction of the participants through a questionnaire applied at the end of the intervention. Furthermore, interviews were conducted with the participants 3 months after the program end (Loss & Remor, 2018). Seventy-five per cent of the participants reported being "very satisfied" and 25% "satisfied". Among the participants, 87.5% reported being "very satisfied" with the facilitators of the program and 12.5% "Satisfied". When considering the learning derived from the program, the reported satisfaction was 100%. Finally, 100% of the participants claimed to have fully understood the contents of the session (Remor & Amorós-Gómez, 2018). Qualitative analysis of interviews identified that the participants perceived benefits from the intervention and reported the use of personal resources nurtured by the program in real-life situations (Loss & Remor, 2018). Based on these findings, we decided that the intervention is feasible and we should plan a formal evaluation study about the outcomes related to the intervention.
According to the American Psychological Association Presidential Task Force on Evidence-Based Practice (2006), the continuous evaluation of results obtained with interventions will allow practitioners to offer preventive treatments and programs that actually fulfill their objectives. Evaluation of programs through replication is of paramount importance to establish the quality and effectiveness of interventions offered in the field of psychology. The present work aimed to report (a) the evaluation of effects of the Psychological Resources Program on self-reported psychological-resource-related outcomes; (b) the participant's satisfaction with the intervention and (c) program attrition.

Design
The study was quasi-experimental, with pre-test and post-test, and 3-month follow-up assessments; there was no control group (Shaughnessy et al., 2015). The intervention protocol (independent variable) consisted of a preventive psychological intervention program.

Intervention protocol
Briefly, the preventive psychological intervention program aims to empower individuals and foster psychological resources. It involves 10 weekly group sessions that each last approximately 120 minutes. It is administered by two facilitators, and the suggested number of participants ranges from 8 to 14. It targets the young and adult population capable of reading and writing and can be administered by trained professionals with different backgrounds (e.g., psychologists, educators, or other health-care professionals). The program intervention is manualized and includes activities such as group dynamics, written exercises, interaction among participants, tasks that involve creativity, assertive communication, emotional self-regulation training (e.g., breathing and relaxation techniques, mindfulness exercises), and tasks completed during the week by the participant (i.e., action plans). Each session of the program has specific topics and objectives, described in detail in the program manual (Remor & Amorós-Gomez, 2013). Over the course of 10 weeks, these activities provide knowledge and foster the development of psychological resources. Table Table 1 provides a summary of the program, following the guidelines of the Template for Intervention Description and Replication (TiDiEr; Hoffmann et al., 2014).

Participants
Forty-eight undergraduate students from a university in Brazil were enrolled to participate in the program. The participants enrolled consisted of 36 women (75%) and 12 men (25%), aged between 18 and 51 years, with a mean of 27.2 years (standard deviation [SD] 7.6). However only 42 attended the first session (6 students did not show up). The participants were students from different disciplines, including biology, biological sciences, nutrition, museology, social sciences, dance, dentistry, physiotherapy, physical education, chemistry, art history, architecture, visual arts, accounting, advertising, and mathematics. The intervention occurred across four consecutive groups delivered from 2017 to 2018. Twenty-seven students completed the program (22 women and 5 men, aged between 19 and 51 years, with a mean of 28.1 years [SD 8.1]), and 26 responded to the follow up measure. Figure Figure 1 presents a flow diagram of the study and the dropouts throughout the intervention process and assessment.

Outcomes measures
In order to evaluate the results of the intervention, the following outcome measures were used (in parentheses are the Cronbach's alpha values for the instruments considering the present sample at baseline):

WHY
Psychological resources are skills, beliefs, and individual personality factors that influence how people cope with challenging or threatening events; they may be manipulated in a intervention. Thus, such an intervention may be useful for promoting mental health and well-being.

WHAT
Materials: The program is a curriculum-based skills training approach, delivered in 10 workshops that are conducted weekly in groups. The manual of the program is available for sale in Spanish (Remor & Amorós-Gomez, 2013). It includes sheets that contain the exercises for each session of the program. These exercises are delivered to participants at the session.

4.
Procedures: The contents for each session are different; they focus on specific activities to develop a particular psychological resource. Activities are preestablished following the program manual.

WHO PROVIDED
Each group was led by a facilitator, with the assistance of an auxiliary. The facilitators (psychologists) received training to lead the program and studied the manual. The training consisted of participating in an intervention as observers when one of the authors (E.R.) performed the program in a pilot intervention. When they performed the intervention, they received regular supervision with one of the authors of the program (E.R.).

HOW
The program was delivered in 10 face-to-face sessions conducted weekly in groups. The total time ranged from 17.5 to 20 hours per group (four intervention replications) with 8-12 participants per group.

WHERE
Sessions were held in a classroom inside a university. The infrastructure included a private meeting room with chairs and auxiliary tables.

WHEN AND HOW MUCH
The intervention was composed of 10 weekly sessions. The duration of each session ranged from 105 to 120 minutes for groups with 8-12 participants.
The interventions occurred at different times (according to the monitor availability). Two groups met in the mornings (10:00 to 11:45) and two met in the evenings (19:00 to 20:45) on different weekdays. The intervention was offered free of charge.

TAILORING
The intervention was delivered according to the manual for all groups by the same facilitators.
(Continued)  Davis, 1983), in its Brazilian version by Sampaio et al. (2011). It consists of 26 items with responses in a 5-point Likert format. It evaluates cognitive aspects (perspective taking; fantasy)

MODIFICATIONS
The intervention was not modified during the course of the study.

HOW WELL
Planned: Based on the program manual, a checklist was prepared that contained all the procedures expected in each session. Facilitators (principal and auxiliary) checked after the session if all listed points were delivered.

12.
Actual: During supervision, the checklist was reviewed with facilitators. There was high fidelity to the program manual for the development of the sessions.

Follow-Up 1
Completed full assessment and started intervention (n = 42)

Follow-Up 2
Lost to follow-up (did not answer follow-up 2 assessment; n = 1)

Figure 1. Flow diagram of sample composition.
and affective aspects of empathy (empathic concern; personal distress). In the present study, we considered the global empathy score as the outcome. A higher score indicates greater empathy.  Scheier et al., 1994), in its Brazilian version by Bastianello et al. (2014). It consists of 10 items, of which four are distracting items. Respondents use a 5-point rating scale (0 = strongly disagree; 4 = strongly agree) to show how much they agree with 10 statements about positive and negative expectations. A higher score indicates greater dispositional optimism.  Diener et al., 1985) in its Brazilian version by Zanon et al. (2013). The SWLS is a short, five-item instrument, answered in a Likert format from 1 (strongly disagree) to 7 (strongly agree), designed to measure global cognitive judgments of satisfaction with one's life. Higher scores indicate more satisfaction with life and wellbeing.  Watson & Clark, 1994), in its Brazilian version by Zanon and Hutz (2014). The scale consists of 20 descriptive items, 10 to evaluate positive affects (Cronbach's alpha: 0.91) and 10 items to evaluate negative effects (Cronbach's alpha: 0.87). Each item is rated on a 5-point  Goldberg & Williams, 1988), in its Brazilian version by Pasquali et al. (1994). The scale is used to measure the level of mental health (non-severe psychiatric illness) of adults with regards to depression, anxiety, and social dysfunction. Lower scores indicate better mental health.  (2012) for the present study, consists of a single question applied at the end of each session that aims to evaluate the clarity of the information and understanding of the contents delivered in the program. In order to assess the clarity of the content, participants answer the question "Do you think the content developed by the facilitator(s) in the session were clear?" With options of response between 1 (unclear) and 10 (very clear). For understanding of the contents, the question is "In your opinion, the contents and activities covered in the session were well understood?" The response options are between 1 (nothing) and 10 (everything).

Scale of generalization of the intervention.
It was adapted from Remor and Amorós Gómez (2012) for the present study, consists of a single question, applied at the end of each session, that aims to evaluate the generalization (application) of the contents of the intervention, i.e., "How much did you apply the contents worked in the previous session?" The response is from 1 (nothing) to 10 (a lot).

Satisfaction with the intervention questionnaire. It was adapted from Remor and Amorós
Gómez (2012) for the present study. Satisfaction with the intervention was evaluated after the end of the program through a self-reported questionnaire. It contains nine self-assessment questions (see items in Table 5) and a descriptive qualitative open question (i.e., "Please, include suggestions and comments you would like to make to improve the intervention program").

Procedures
The invitation to participate in the program was disseminated through e-mail and digital media within university channels and websites. Participants should be over 18 years old, not had previous experience with psychotherapy, and not be under psychological treatment throughout the evaluation period. Those who contacted the investigators by email and met the inclusion criteria were informed that participation was voluntary and received the Informed Consent Form. Before starting the intervention, the participants answered the pre-test through an online platform (Survey Monkey ® ) 1 week or at least 3 days before the program started. Personal data were kept confidential, and participants could withdraw from the program at any time.
The intervention consisted of 10 weekly sessions (105 min each). At the beginning and end of each session, participants completed the perceived mood scales. The measures of clarity, understanding, and generalization were applied only at the end of each session. One week before the program and 1 week after the end of the 10 sessions, the participants answered the online assessment protocol questionnaires to evaluate outcomes. Three months after the end of the program, a second follow-up assessment was performed online.
Examination of satisfaction with the intervention aimed to determine how the participants who completed the program perceived the intervention, their satisfaction, and their evaluation of the facilitators and program content; it also collected suggestions to improve the intervention. This questionnaire was completed as a pencil-and-paper task at the end of the last session. Participants were given approximately 10 minutes to complete the questions.

Data analysis plan
The data were examined for missing information, outliers, and normality. The values for seven aleatory answers were lost in the entire database. To impute these missing values, the sample mean was calculated in the item, according advice from Field (2009). Furthermore, between 1 and 5 participants skipped completing one or more questionnaire during the online assessment, and, consequently, these scores were not included in the statistical analysis. Thus, the number of participants (n) are stated for each statistical analysis. Continuous variables were expressed in terms of the mean (M), SD, range, and median (Mdn). Categorical variables were expressed as frequencies and percentages. Considering the small sample size and the fact that the variables were not normally distribution, all statistics were calculated using nonparametric tests (i.e., Wilcoxon Signed-Rank test, Friedman's analysis of variance [ANOVA]). For the Wilcoxon Signed-Rank test, the effect size was calculated for all comparisons according to the recommendations of Field (2009). Specifically, the effect size was calculated using an equation to convert a z-score into the effect size estimate (r) according to Rosenthal (1991, p. 19). Effect sizes cannot be calculated directly for a Friedman's ANOVA test, so alternatively, Kendall's W (coefficient of concordance; k samples) can be employed (Marshall & Marquier, 2016). Kendall's W looks at the agreement between subjects and assigns a value between 0 and 1. A Kendall's W of 1 indicates that all subjects ranked the four methods in the same way and thus they were in complete agreement. Kendall's W uses Cohen's interpretation guidelines. Cohen provided rules of thumb for interpreting these effect sizes, namely an r of |.1| represents a "small" effect size, |.3| represents a "medium" effect size, and |.5| represents a "large" effect size.
For the dropout analysis, Student's t-test was performed to compare data from dropouts and completers. Cronbach's alpha was used to determine the internal consistency reliability for the scales. All statistical analyses were conducted using SPSS 20.0 version software (SPSS, Inc., Chicago, IL, USA).
Qualitative analysis of the answers regarding the open question in the satisfaction with the intervention questionnaire will be part of a future study with a qualitative approach (not presented here).

Ethics approval and consent to participate
All aspects of the project and study were conducted in compliance with ethical standards for research and publication. The study was approved by the Research Ethics Review Committee of the University (CAAE 8031692017.9.0000.5334). All participants were informed about the purpose of the study and they entered into the study only after their agreement and providing written informed consent.

Outcomes related to the "Psychological Resources Program" intervention
To evaluate the effects of the program session on perceived mood (1 to 10 scale), comparisons before and after each of the sessions were made using the Wilcoxon Signed-Rank Test. The results are presented in detail in Table Table 2. The mean mood state improved week-by-week. In session 1, the mean of mood state was 6.6 before and 7.8 after the session. For session 5, the mean increased from 7.1 (before) to 8.5 (after) the session. For the last week of the program, the mean for mood state was 8.1 at the beginning of the session and 9.1 at the end. The observed effect sizes on perceived mood change ranged from medium to large.
To evaluate benefits related to participation in the program, the pre-test and post-test scores of participants who completed the intervention were compared using Wilcoxon's non-parametric Abbreviations: M, mean; SD, standard deviation; Mdn, median. Notes: *p < 0.05; **p < 0.01; #Effect size was calculated using an equation to convert a z-score into an effect size estimate (r) according to Rosenthal (1999;p. 19). Cohen's guidelines for r: a large effect is 0.5, a medium effect is 0.3, and a small effect is 0.1 Abbreviations: M, mean; SD, standard deviation; Mdn, median. See "Method" section for details on the measures. Note: *p < 0.05; **p < 0.01; #Effect size was calculated using an equation to convert a z-score into an effect size estimate (r) according to Rosenthal (1999;p. 19). Cohen's guidelines for r: a large effect is 0.5, a medium effect is 0.3, and a small effect is 0.1 statistics; corresponding effect sizes of differences were calculated. The results are shown in detail in Table Table 3.

Z (p) Effect size#
Additionally, to ascertain if the obtained results were maintained over time, a non-parametric Friedman's ANOVA statistical analysis was performed with the pre-test, post-test, and follow-up scores. Effect sizes were also calculated and reported, and all results are shown in Table Table 4.

Assessment of the intervention process and satisfaction
According to the answers to the satisfaction questionnaire, all participants felt "well" or "very well" during the intervention (question 1) and reported being "satisfied" or "very satisfied" with the program (question 2). All respondents positively assessed the intervention facilitators (question 3) and were satisfied with the lessons learned from the program (question 4). The majority of the participants (72%) had no difficulty attending program sessions (question 5). Almost all (96%) also indicated that the time chosen for the sessions was adequate (question 6). All the participants liked the group activities and stated that they understood the contents and topics covered in the intervention (questions 7 and 8). The duration of the sessions seemed to be adequate for most participants (92%; question 9). Table  Table 5 shows, in detail, the frequency and percentage of responses marked for each option.
To assess the clarity, comprehension, and generalization of program content, participants completed a 10-point rating scale for all sessions that evaluated these three dimensions. The averages obtained in each session of the program are presented in Table Table 6.

Dropout analysis
Dropout is common and threatens the validity of results, because completers may differ from people who dropped out. In the present study, dropouts occurred between sessions 1 and 4. To ensure that those who dropped out of the intervention were not different from the participants who completed the program, a baseline comparison was made between these groups. This analysis was performed with a Student's t-test to compare their means in each evaluated dimension. Comparisons were conducted between dropouts (n = 15; women = 10) and completers (n = 27; women = 22) with regard to participants' baseline characteristics. There were no significant differences between completers and dropouts ( Table Table 7).

Discussion
The purpose of this study was to replicate the "Psychological Resources Program" intervention in the Brazilian population and report the outcomes. Previous studies reported that participation in the program generated positive results, including improved mood and development of psychological resources (Remor & Amorós Gómez, 2012;Remor et al., 2010).
According to the results reported here, it is possible to assume that the program was implemented properly. Thus, participants' mood was improved and there were statistically significant changes in several variables related to psychological resources. There was an increase in positive affect, gratitude, optimism, resilience, assertiveness, satisfaction with life, and general mental health. Participants also reported decreases in negative affect and perceived stress. Additionally, the participants evaluated that the intervention content was clearly explained. They reported being able to adequately understand the content and to put into practice what was learned during the sessions (measured through rating scales for clarity, comprehension, and generalization).
Participants in the intervention reported improved mood state for all sessions. These changes were statistically significant, and the effect sizes were medium or large (except for session 8). Session 8 focuses on forgiveness. In order to promote the development of this personal resource, participants are asked to remember past situations that involved resentment. Getting in touch with these memories can foster negative feelings, and this phenomenon may explain the lack of mood state improvement during this session. This effect was also observed in past studies with the program (e.g., Remor & Amorós Gómez, 2012;Remor et al., 2010). Kendal's W uses Cohen's rules of thumb for interpreting these effect sizes. Specifically, an r of 0.1 represents a "small" effect size, 0.3 represents a "medium" effect size, and 0.5 represents a "large" effect size. On the other hand, the mood state notably improved week-by-week. It is possible that experiencing this mood improvement is an important factor that motivates participants to engage in the intervention from beginning to end. Moreover, dropouts occurred between the first and fourth session. Perhaps the participants who adhered to the intervention were more prone to be reinforced and motivated by the positive mood experience.

Friedman's ANOVA
The participants who completed the program also presented a decrease in perceived stress and an increase in assertiveness and resilience. Assertiveness and resilience are linked to the variable "positive coping", which is one of the intervention's targets. These results suggest that the program effectively taught participants to deal with the demands in a way that does not lead to overload, a phenomenon that is reflected in lower levels of perceived stress. The same relationship between increased problemsolving ability and decreased perceived stress is documented in a previous study with the program (Remor et al., 2010).
When considering these results, it is important to note, as limitations, the small sample size, non-existence of a control group, and selection of participants based on non-probability sampling. Furthermore, the expected significant changes in empathy and forgiveness were not observed (but revenge motivations decreased significantly at the follow-up).
Of the 42 participants enrolled in the program, 27 completed the intervention. This number represents an attrition rate of 35.7% (15). This number is within the range expected for an intervention, especially when considering a review by Salmoiraghi and Sambhi (2010) that puts dropout rates in cognitive-behavioral interventions between 19% and 50%. It is also important to consider the characteristics of the participants (university students). This factor is relevant because, among the dropouts, 40% indicated as a reason for abandonment "time conflict". Six participants interested in the intervention did not attend the first session of the program and did not respond to contact from the researchers. Since college students have many demands, the academic environment may contribute to overload that would lead to dropout. Therefore, the intervention contents are likely of interest to the participants and implementation of the intervention in a university context is feasible. The intervention satisfaction survey performed at the end of the program also supports this assumption. Participants demonstrated satisfaction with the program, its facilitators, duration, and the topics addressed.    To evaluate one of the potential threats to the internal validity known as attrition bias, a baseline analysis of characteristics of completers and dropouts was conducted. Pre-test scores for the dropouts and completers were compared; there were no significant differences in the evaluated characteristics between the groups. Since dropouts were similar to completers, abandonment may not be related to the individual characteristics of the participants. Moreover, observed outcomes by completers also cannot be attributed to a selection bias.
Measures have taken 3 months after the end of the intervention (follow-up) showed that changes in gratitude, optimism, perceived stress, positive and negative affect, resilience, general mental health, and assertiveness persisted over time.
With regards to forgiveness, there was no change after intervention in any of the TRIM-18 instrument sub-scales. However, revenge motivation significantly decreased from baseline to three-month follow-up. Therefore, it is important to consider the construct of forgiveness. Participants are taught during the program that forgiveness cannot be rushed (Remor & Amorós-Gomez, 2013). One must respect the time necessary for her/him to forgive. This concept applies to situations where one person wants to be forgiven by another. This aspect is relevant, because it may be related to the fact that there were no significant changes in the improvement of this resource after the program. It is possible that the time participants need to deal with issues that require forgiveness exceeds the time when program evaluations were conducted. Another hypothesis is that forgiveness is a psychological resource that requires more time for development. Examples of interventions that focused exclusively on forgiveness and had positive results lasted between 6 ( Harris et al., 2006) and 12 weeks (Coyle & Enright, 1997). Another intervention that fostered this resource was composed of two meetings (3.5 hours each) dedicated exclusively to the matter of forgiveness (Allemand et al., 2013). Participants in this study did not show significant changes in empathy or forgiveness. Future evaluations of the program with different participants will help to determine whether the intervention can generate substantial changes in forgiveness and empathy. Regarding the results in the forgiveness and empathy variables, it is important to consider aspects related to the psychometric properties of the utilized instruments. As far as IRI (empathy) and TRIM-18 (forgiveness) are concerned, there are divergences in the literature regarding their structures and psychometric quality (Fernández-Capo et al., 2017;Formiga, 2015). Considering the above factor, an alternative for future studies could be to use different instruments. This change would help determine whether the changes were observed due to the lack of instrument acuity or if the content of the intervention are not enough to achieve changes.
According to Baker and McFall (2014), despite the importance of evaluating interventions, most of the results are not evaluated further. Thus, their efficacy and clinical utility cannot be accurately stated. The present study is an example of continuous evaluation of results, where the "Psychological Resources Program" was delivered to four consecutive groups, in order to compose a sample, and submit it to evaluation in order to verify its benefits. Further, the present work follows the American Psychological Association recommendation to seek to unite the best evidence of research with practice (American Psychological Association Presidential Task Force on Evidence-Based Practice, 2006).
Finally, the changes registered in most of the outcomes indicate that the Psychological Resources Program may represent a potential tool for mental health prevention and health promotion. Additionally, the changes observed after the intervention endured for up to 3 months, a finding that indicates that such changes might last over time. However, further developments such as replication in a larger sample using a control group, ideally in a randomized controlled trial, which is the gold standard in the evaluation of psychological treatments is recommended.