The large-scale implementation and evaluation of a healthy lifestyle programme in residential out-of-home care: study protocol

Young people living in out-of-home care (OoHC) are widely considered to be one of the most vulnerable groups in the community. This article describes the evaluation and implementation study protocol of a healthy lifestyle programme developed to improve the eating and physical activity habits of young people living in residential OoHC. The study will employ a longitudinal multi-group pre-post process evaluation to assess our implementation strategy in the dissemination of HEALing Matters at the: 1) system-level; 2) organizational-level; 3) residential home-level; and 4) individual-level (N = approximately 60 residential homes [approximately 400 – 500 sta ﬀ and 240 young people]). Implementation-and program-related outcomes will be assessed using self-report surveys, semi-structured interviews and focus groups, house audits and website usage data collected prior, during and after implementation. Results are expected to inform and help guide the further development and training of residential care sta ﬀ ’ s understanding and application of health literacy.


Introduction
In this paper, we describe the implementation and evaluation study protocol of an Australian healthy lifestyle programme developed to improve the eating and physical activity habits of young people living in residential out-of-home care 1 (OoHC).This programme has been specifically designed to strengthen the therapeutic and healthy lifestyle capacities of residential care staff responsible for providing care to young people placed in residential homes.Residential OoHC refers to placements designed for young people who are removed from their families by the state due to the presence of neglect, abuse and/or unsafe and unstable family environments (where care is hence provided by paid staff in a residential home).OoHC is most commonly provided to young people up to 18 years old (usually aged 12-17 years) and is often associated with developmental difficulties and negative consequences.Indeed, young people living in the OoHC system are widely considered to be one of the most vulnerable and disengaged groups in the community (Ainsworth & Hansen, 2005; Australian Institute of Health and Welfare [AIHW], 2016; Barber & Delfabbro, 2003), experiencing a wide range of adverse physical and mental health outcomes (Ford, Vostanis, Meltzer, & Goodman, 2007;Nathanson & Tzioumi, 2007;Osborn & Bromfield, 2007;Skouteris et al., 2011).

Background: HEALing matters
Our formative research established that young people living in residential OoHC are at increased risk of disordered eating, overweight and obesity, and associated morbidities compared to typically developing peers in the wider community (Cox et al., 2014;Skouteris et al., 2011).This body of work (Cox et al., 2017a(Cox et al., , 2014(Cox et al., , 2015(Cox et al., , 2017b;;Skouteris et al., 2014) also included a pilot evaluation of the Healthy Eating, Active Living (HEAL) programme.HEAL was a 12-month programme (inclusive of 6-months maintenance) that aimed to provide young people in residential OoHC with practical opportunities to improve their eating and physical activity habits.The programme targeted young people's direct-residential care staff, providing professional development and resources to facilitate healthy lifestyle behaviour changes among the young people.Findings (both quantitative and qualitative) from the evaluation of HEAL revealed trends towards decreased sugary drink consumption and lower BMI, increased healthy food consumption, improved participation in community sports, adolescent self-esteem and independent living skills (e.g.increased meal preparation by the young people), and increased availability in healthy snacks and meals provided by staff.
Our data also revealed several limitations of the HEAL programme.These included a lack of: (1) a trauma-informed philosophy to guide residential care staff's understanding of the lifestyle behaviours of young people; and (2) cultural connectedness relating to Aboriginal and Torres Strait Islander young people in the OoHC system.Therefore, with funding from the Victorian Government (Australia) to implement HEAL across Victoria, we addressed each of these limitations and revised the programme, renaming it HEALing Matters (i.e.Healthy Eating Active Living Matters).Importantly, the codesign of the HEALing Matters programme involved a team of multi-disciplinary experts and Community Service Organization (CSO) collaborations.This included consultation with Eating Disorders Victoria (EDV), Nutrition Australia, the Department of Health and Human Services [DHHS] (State Government of Victoria, Australia [including Sport and Recreation Victoria and Prevention, Population Health and Place]) and the Centre for Excellence in Child and Family Welfare (CFECFW).Throughout the redevelopment of the programme's content, young people with lived experienced were also consulted; this was accomplished via collaboration with the CREATE Foundation, a consumer body representing the voices of young people with an OoHC experience.This multi-disciplinary and highly collaborative process permitted us to engage key stakeholders to identify scope, priorities, gaps, needs and optimal processes to drive evidence translation and programme sustainability and represents true codesign (i.e.genuine partnership).
To address the previously mentioned limitations of the HEAL programme (i.e. the lack of a trauma informed philosophy and cultural connectedness) we also formed international and local collaborations with experts and keys stakeholders able to provide targeted information and assistance relevant to these limitations.For example, in order to ensure a trauma-informed philosophy was embedded within the HEALing Matters programme (i.e.HEAL's successor), we worked with colleagues from Stirling University (Stirling, Scotland) who are regarded as international leaders in the provision of therapeutic care to young people in OoHC.Consequently, HEALing Matters is now delivered within a framework informed by attachment (Bowlby, 1982), trauma (Emond, Steckley, & Roesch-Marsh, 2016) and resilience (Rak & Patterson, 1996) theories.In doing so, HEALing Matters recognizes that food and physical activity are a powerful way of demonstrating trust, care, predictability, flexibility, and attuned parenting.Furthermore, we worked closely with local Victorian Aboriginal organizations, to ensure HEALing Matters is culturally sensitive, relevant and meaningful to Aboriginal and Torres Strait Islander young people living in residential OoHC.
Finally, the implementation and large-scale roll-out of the programme was also codesigned with key stakeholders, including the Victorian DHHS and major CSOs in the OoHC sector.Hence, the next stage in this line of research is to evaluate the implementation of HEALing Matters across multiple CSOs in Victoria, Australia.Below we outline the methodology and design that will facilitate our evaluation of both implementation-and programme-related outcomes.

Current study
Little work has been carried out within the field of implementation science to understand the barriers and facilitators to the implementation of preventative health strategies in OoHC (e.g.Hanson, Self-Brown, Rostad, & Jackson, 2016;Herrman et al., 2016), with no previous work, to our knowledge, in relation to embedding healthy lifestyle programmes in residential OoHC.The primary aim of this paper is to outline the study protocol for the evaluation of both the implementation and impact of HEALing Matters.Our key implementation-related outcomes are adoption and reach, programme characteristics, outer and inner settings, implementation climate, priority, determinants of implementation behaviour, and programme appropriateness and application.Our key programme-related outcomes are changes in the physical environment of participating residential homes and in the understanding of health literacy (and the application of that information [i.e.knowledge attainment]) of residential care staff.While the majority of these programme-related outcomes will speak to the programme's impact on residential care staff's competencies, knowledge attainment and changes in provision of care, the health-related quality of life (HRQoL) of the young people living in residential OoHC will also be assessed.

Ethical review
The study protocol detailed below was approved by the Monash University Human Research Ethics Committee (MUHREC, Project ID: 12116).All procedures will be executed in compliance with the Australian National Statement on Ethical Conduct in Human Research (2007).

Study design
Implementation of HEALing Matters state-wide, and across multiple CSOs, presents challenges and potential barriers to uptake, adherence and study design.For example, CSOs are likely to demonstrate variability (between and within organizations) in terms of their input and leadership at each level of administration, the implementation climate that currently exists, and the extent to which collaboration occurs.Furthermore, due to the optional and participatory nature of HEALing Matters, we are not able to mandate the number of CSOs that choose to participate, nor control (and thus manipulate [e.g.randomization and/or a control group]) the timing of each residential home's programme start.
In order to address these challenges, the current study will employ a longitudinal multigroup pre-post process evaluation that includes the simultaneous assessment of implementation and programme outcomes.This design is especially pertinent for evaluating programmes in the OoHC system because such a design can be used to evaluate the success of the implementation strategy in the adoption and uptake of HEALing Matters at the: 1) systemlevel/divisional government level; 2) organizational-level; 3) residential home-level; and 4) individual-level.Specifically, we will employ a combination of pre-and post-programme assessments and repeated measures that will be undertaken by groups of individuals involved in various aspects of the programme's implementation.This design is most appropriate for the current study and is pertinent to sustainability considerations helping us to answer: what works, for whom, why, and in what circumstances?

Participants and recruitment strategy
CSOs that provide residential care services in Victoria, Australia will be proactively invited to participate in the HEALing Matters programme and evaluation.Estimates for the number of residential care homes to be included in the HEALing Matters program over the course of 12 months is 60which includes 400-500 residential care staff (aged between 18-65 years) and approximately 240 young people 2 (aged between 14-17 years).These values represent approximately half of the total number of residential care homes currently active in Victoria.Participants will be recruited across all levels of the CSO, including: 1) senior level management (i.e.key stakeholders, recruited via email and through their involvement in deciding whether the CSO they are employed by will participate); 2) residential care home managers (recruited via email and through their involvement in organizing access to the HEALing Matters online programme for residential care staff); and 3) residential care staff who will be completing the HEALing Matters online programme as part of their employment.Residential care staff will have the option to optout of the evaluation component of the online training prior to commencing the first online training module (thus, access to the programme is not contingent on study participation).Additionally, young people living in residential care homes who are participating in the HEALing Matters programme and evaluation will also be invited to participate in the evaluation of the programme.

Programme components
The HEALing Matters programme consists of: 1) an e-learning training/professional development component; 2) additional resources (e.g.video and printable recipes, links to community sport and recreation bodies, and module glossaries); and 3) an online community discussion board that provides residential care staff with the opportunity to participate in a network of practice and knowledge exchange, and also ask questions about the content (the discussion board will be moderated and answered by the developers of HEALing Matters [i.e.health and psychology researchers]).HEALing Matters will be completed via an online platform and contains six training and information modules (that include: Attunement; Shaping Routines; Food for Thought: Physical Activity for Thought; Health Literacy; and Take a Moment for Yourself, see: https://healing-matters.org/home) that each take approximately 45 minutes to one hour to complete (i.e. three to four 15-minute sections).At minimum, residential care staff will be required to complete one section of each module on a weekly basis.The maximum time-frame for completion of all six modules is six months from implementation (i.e.online registration).

Implementation strategies
To enhance the delivery of HEALing Matters a combination of codesigned implementation strategies were selected to support wide-spread adoption, adherence and programme sustainment.Specifically, we will provide face-to-face and phone-based facilitation support to all participating CSOs, idiosyncratic dissemination protocols (to ensure the unique needs of each CSO can be recognized and addressed), utilize quality monitoring and user feedback tools, provide implementation toolkits, and employ e-learning as the key method of delivery.This tailored dissemination approach is responsive to the interests of each CSO and will allow for changes based on each organization's evolving needs and experience of HEALing Matters.In addition, activities to increase buy-in (at all CSO levels) will also be undertaken (e.g.information sessions and e-newsletters).
An e-learning platform was the most suitable delivery method due to its potential for higher levels of efficiency, flexibility, and cost effectiveness (Derouin, Fritzsche, & Salas, 2005;Means, Toyama, Murphy, Bakia, & Jones, 2009).This platform is also the most efficient way to capture user feedback (e.g.reflections on designated learning tasks) and activity (i.e.behaviours in using the online programme).Furthermore, e-learning provides residential care staff with the opportunity to explore new information at their own pace and in a private setting (and through this autonomous training, may also lead to improvements in self-management).Importantly, and in line with Anderson's model (Anderson, 2008) for how to effectively integrate e-learning, HEALing Matters is designed with a strong appreciation for residential care staff's prior experiences and work context.Specifically, learning is reflective, often via Socratic questioning, authentic and actively constructed.Participation in data collection related to online activity (i.e.such as the frequency of resources being accessed or e-learning activity completion time) will be collected as part of participation in the online programme.Participation in all other aspects of data collection will be voluntary.

Measures
Below we describe the measures selected for the evaluation of both the implementation and impact of HEALing Matters.A summary of the data collection procedures can be found in Table 1.

Implementation-related outcomes
The measures presented below will be used to evaluate the implementation strategies selected to assess key implementation outcomes including adoption and reach, program characteristics, outer and inner settings, implementation climate, priority, determinants of implementation behaviour, and program appropriateness and application.

HEALing matters uptake (adoption and reach)
Adoption will be assessed at the systems/divisional-, organizational-and residential home-level.In doing so, stratified estimates of HEALing Matters uptake at these various levels will be calculated.Furthermore, because accurate numbers of residential care staff employed in OoHC are available, a similar proportional calculation of reach, based on the number of residential care staff who participate in the programme versus the number of residential care staff currently employed in Victoria, will also be conducted.Finally, website usage data (e.g. the number of times a particular resource is accessed or the time it takes to complete a module) will also be collected and used to inform a fine-grained adoption analysis of participation in HEALing Matters.

Stakeholder surveys (outcomes: programme characteristics, outer and inner settings, implementation climate and priority)
Pre-and post-programme surveys (completed online) will be conducted with key stakeholders (i.e. senior management) within each participating CSO and at the DHHS.Based on the Consolidated Framework for Implementation Research (CFIR) domains (Damschroder et al., 2009), these surveys will assess a broad range of implementation factors including programme characteristics (e.g.complexity and design quality), outer and inner settings (e.g.compatibility, tension for change and priority), and implementation climate and priority.Questions will be answered using a combination of Likert scales and open-ended responses and will take approximately 10 minutes to complete.The pre-and post-programme surveys will be emailed to participants.

Residential care staff surveys (outcomes: implementation climate and priority)
Based on CFIR domains (Damschroder et al., 2009) pre-and post-programmem surveys (completed online) will be conducted by residential care staff to assess implementation climate and priority.Surveys will be integrated into the HEALing Matters online training programme.Residential care staff will complete the pre-programme survey immediately prior to commencing the first module and will complete the post-programme survey immediately after the conclusion of the final module (i.e.Module 6).Questions will be answered using a combination of Likert scales and open-ended responses.Each survey will take approximately 15 minutes to complete.Formative module assessmentsresidential care staff (outcomes: determinants of implementation behaviour) Formative assessments will also be completed by residential care staff at the completion of each of the six training modules.Based on the Determinants of Implementation Behaviour Questionnaire [DIBQ] (Huijg et al., 2014) formative assessments will be used to assess knowledge and skills, selfefficacy and confidence, attitude, implementation intentions and motivation relevant to each module's content, organizational and implementation climate, and organizational morale.In addition, the assessment will also measure anticipatory affective cognitions (affective responses to putting new practices into action often reflect enthusiasm or trepidation related to the programme and thus also serve to inform implementation behaviour (Damschroder et al., 2009)).Questions will be answered using a combination of Likert scales and open-ended responses and it is anticipated that the surveys will take 10 to 15 minutes to complete.
Post-programme focus groups (outcomes: programme appropriateness and application) To assess the implementation of HEALing Matters at scale, it's appropriateness (i.e.perceived fit within residential homes) and application (i.e. the degree to which staff have used HEALing Matters practices and learnings) residential care staff will be invited to take part in audio-recorded focus groups.Focus groups will occur four to six weeks after the completion of HEALing Matters, will be semi-structured, and based on the programme characteristics domain of the CFIR (see: Damschroder et al., 2009).Additionally, residential care staff will also be asked to elaborate on what worked and what did not work.It is estimated that focus groups will run for approximately 30 minutes.

Programme-related outcomes
The programme-related outcomes examined in this study lean strongly towards changes in either the physical environment of participating residential homes or in the enhanced understanding of health literacy (and the application of that information) by residential care staff.The reasoning for this is that while HEALing Matters is aimed at improving the eating and physical activity habits of the young people living in residential OoHC (in which young people are seen as the beneficiaries), the end users of the program are residential care staff.Therefore, the outcomes selected for assessment are representative of the individuals who actively participated in the programme and focus on the extent to which trainees apply the knowledge, skills and attitudes gained from the HEALing Matters program to the job (Baldwin & Ford, 1988;Gilpin-Jackson & Bushe, 2007).

Residential home survey (outcomes: food quality and physical environment)
To assess the impact of HEALing Matters on each residential home's food quality and physical activity engagement, a pre-and post-programme survey will be completed by the home manager within each residential home.The surveys are aimed at assessing changes in the physical environment and will include the following components: 1) grocery and take-away receipts collected for 7 days pre-and post-implementation (calculations based on the proportion of the home's budget that is spent on different types of food); 2) a list of the current equipment in the residential home that supports physical activity and the frequency of physical or recreational activities that each young person in the residential home is engaged in; and 3) estimates of both the proportion of meals that are eaten together (i.e.young people and care staff together) versus those that are eaten individually and meals prepared and cooked by staff versus the proportion of meals not prepared or cooked by staff.
Summative programme assessmentresidential care staff (outcomes: staff competencies, knowledge attainment [health literacy] and young people's health related quality of life)  KIDSCREEN-10 (Ravens-Sieberer, 2006) will also be included in the Residential Care Staff Surveys.The KIDSCREEN-10 provides a singular index of global health-related quality of life (HRQoL) covering physical, psychological and social facets of the construct (10 items).Residential care staff are required to rate quality of life from the perspective of the young people (i.e. a proxy-patientperspective).
Post-programme interviewsyoung people (outcome: provision of care) Young people will be invited to take part in a 15-to 20-minute face-to-face interview with a member of the research team in a location that they are comfortable with (e.g.either at their residential home or at the office of their home's CSO).Interview questions are designed to assess the extent to which young people perceive changes in their residential home relative to healthy food options and opportunities to engage in physical activity.House managers in each residential care home will help to facilitate recruitment of young people in their care by seeking consent from the young person's parent or legal guardian (or acting as the young person's legal guardian in the case of wards of the state).House managers will also assist the young person to read the explanatory statement, answer any questions they may have, and ensure they are informed about what participation in the research involves.

Analytic considerations
As is often the case with process evaluations, data analysis will be ongoing and iterative, allowing for both apriori and post-hoc testing.Quantitative data analyses will include descriptive statistics (i.e. frequencies and measures of central tendency) and mean difference tests.In particular, at each time point (e.g.pre-and post-programme and between each CSO) differences in both implementation and programme outcomes will be calculated by comparing post-programme data to baseline measures.Qualitative data will be analysed using thematic analysis to identify the recurring themes in the dataas outlined by Braun and Clarke (Braun & Clarke, 2006).Specifically, all audio recordings will be transcribed verbatim and systematically double-coded independently by two trained researchers.Following an in-depth review of the coded data, independent themes will be developed based on recurrent content that reflect the CFIR and KTA frameworks.Both coders will then engage in a cooperative discussion of themes to decide on the most pertinent and recurrent aspects of the coded data.This process of refining and reviewing themes will be iterative until themes are representative of the data and saturation is achieved.

Discussion
To our knowledge, no one has reported on the implementation of a healthy lifestyle programme in residential OoHC at this scale or depth of evaluation.The HEALing Matters state-wide roll-out provides an opportunity for examining multi-level facilitators and barriers to implementation in OoHC and has the potential to advance the quality of implementation science within OoHC by adding to the limited research examining implementation strategies in this context.We plan to use the findings from this research project to inform the future and sustainable implementation of HEALing Matters in others states of Australia and internationally and across various forms of OoHC (i.e.kinship and foster care).The lack of a control or comparison group (due to the optional and participatory nature of the programme) is a limitation of our study design; we cannot provide unambiguous conclusions regarding programme-related outcomes as we would if a randomized controlled trial design was included.However, randomized controlled trials are not necessarily appropriate for this cohort of vulnerable young people and the sector that is responsible for their wellbeing (Landsverk, Hendricks Brown, Rolls Reutz, Palinkas, & McCue Horwitz, 2011;Proctor, 2012;Victora, Habicht, & Bryce, 2004).In relation to the implementation-related outcomes, the measures selected, the methods and the data collection procedures are not contingent on randomization or the comparison to a control group.To further tackle the limitations of these (unavoidable) methodological constraints, we will use a combination of pre-and post-programme assessments and repeated measures which allow for comparisons within and between each participating residential care home, CSO and divisions across the State of Victoria, Australia.Furthermore, we will collect data from a variety of sources (e.g.key stakeholders, residential carers, young people, home audits and website usage data) to ensure not only evaluation breadth, but also to analyse the convergence between these various participant groups.

Conclusion
Researchers and policy makers often fail to engage with stakeholders and end users and instead rely solely on academic 'expert opinion' (current authors not excluded) that is often far removed from the context in which implementation occurs and where complexity is significant (Robinson et al., 2018).Additionally, relying on linear dissemination strategies for knowledge translation is likely to fail compared to strategies that, through an iterative and codesigned process, aim to identify and address contextually specific barriers and facilitators to implementation and knowledge translation.Given the complexity of implementing programmes within the residential OoHC system (see Cox et al., 2017aCox et al., , 2017b)), we expect that the findings of this study will contribute significantly to our knowledge of how best to develop and scale up OoHC programs.

Table 1 .
Summary of the data collection procedures.
(Dominick, Saunders, Dowda, Kenison, & Evans, 2014) Surveys mentioned above (completed online by residential care staff pre-and post-programme) are items that form a summative test of the HEALing Matters programme.Specifically, these items relate to HEALing Matters' training competencies and changes in both the residential home's provisions/environment (based on the Physical Activity and Dietary Environment Assessment [PADEA](Dominick, Saunders, Dowda, Kenison, & Evans, 2014)) and residential care staff's knowledge and practices relevant to providing care that is healthy and physically active.For example, changes in the understanding and application of Australian Dietary (Department of Health and Aging, 2013) and Physical Activity (Australian Government Department of Health, 2017) Guidelines.Finally, a proxy version of the