Do widening participation interventions change university attitudes in UK school children? A systematic review of the efficacy of UK programmes, and quality of evaluation evidence.

ABSTRACT University participation gaps between free school meals (FSM) and non-FSM students have remained largely stable in the UK since 2006. The efficacy of UK widening participation (WP) interventions in changing attitudes towards university and increasing participation of disadvantaged students is uncertain. Some approaches have shown to be effective in other countries e.g. the US, but the educational experience of pupils and markers of disadvantage may be different to the UK. Previous reviews have indicated poor-quality evidence and a lack of peer reviewed experimental evaluation in a UK context. The range and quality of evidence using a non-experimental or qualitative approach is unclear. Understanding how effective WP interventions are in changing attitudes and behaviours towards university in the UK is an important part of addressing the participation gap. This study used a systematic review approach to examine the evidence base of peer-reviewed evaluations of UK WP programmes for school children aged 16 and under in the last 20 years. Inclusion criteria encompassed a range of quantitative and qualitative study designs to gain a picture of the efficacy of interventions and the type and quality of the evaluation evidence. Findings indicated that controlled experimental evaluation of UK interventions remains very limited. Evaluations were largely mixed methods. Quality of evidence was judged moderate overall due to multiple issues with bias and lack of clarity in methods. Approaches that appeared to offer the most potential were those that helped students to develop new skills with tangible outcomes, and exposure to role models.


Introduction and background
The efficacy of the UK's widening participation (WP) interventions in changing attitudes and behaviours around university is unclear.Gaps in university attendance between free and non-free school meal students are currently 18.8 percentage points, the highest since 2006/2007 (Department for Education, 2020).The need to address gaps in university attendance across different groups has been acknowledged by government papers and policies particularly since the Dearing report "Higher Education in the Learning Society" (1997), with the Labour government in 1999 setting a target of 50% of young people attending university (House of Lords Economic Affairs Committee, 2017).However, despite over two decades of focus on widening participation by multiple governments, inequalities persist and students from disadvantaged backgrounds are less likely to go to university than their more advantaged peers.A raft of widening participation interventions has been employed in the UK to address this issue.Approaches range from changes to access pathways such as degree apprenticeships (Education and Skills Funding Agency, 2019), bursary initiatives (Harrrison & Hatt, 2012), contextualised admissions policies (Boliver et al., 2015), and multi-faceted university-run programmes such as Aimhigher, which aimed to raise student aspirations (Doyle & Griffin, 2012).This latter type of approach, university-run interventions for school students, is the focus of this study.There is limited causal evidence as to the efficacy of individual components of such programmes (Robinson & Salvestrini, 2020).Understanding how effective such approaches are in supporting UK students to go to university is an important step in addressing the gap in attendance between socio-economic groups.
There is a wealth of literature published around widening participation programmes in the UK, but much of it comprises opinion pieces and theoretical papers rather than empirical evaluation, or is lacking in appropriate design, methods, analysis, or interpretation of findings (Gorard & Smith, 2006).A later review identified a lack of experimental evaluations persisted when searches were conducted in 2012/2013 (Younger et al., 2018).Younger et al. (ibid) found some evidence of effective strategies such as some financial incentives, although this evidence was not from the UK.Yet, another review of the efficacy of financial supports and bursaries suggests inconsistent evidence as to its positive effects, with a lack of empirical data (Kaye, 2021).Similarly, Robinson & Salvestrini's (2020) wide-scope review of both peer-reviewed evidence and grey literature of WP interventions found limited causal evidence of intervention efficacy, particularly in a UK context.Findings from an international context must be considered cautiously when applied to the UK education system, as the demographic profile of students and markers of disadvantage may differ between countries, though there is a strong pool of causal evidence particularly from US evaluations, from which inferences may be drawn.Additionally, Moore et al. (2013) point to the limited availability of empirical evidence for the efficacy of information, advice, and guidance-type widening participation interventions and highlight the need for a robust, integrated approach to evaluation between practitioners and researchers to fill this gap.In sum, quality evaluation based on causal, empirical data of the efficacy of UK widening participation interventions is limited.
Additionally, the impact of student attitudes upon actual participation is unclear.Student aspirations (the desire to attend university) appear to be high across all socioeconomic groups (Croll & Attwood, 2013;Goodman & Gregg, 2010;Harrison & Waller, 2018;St. Clair, Kintrea, & Houston, 2013) and there is little evidence that aspirations actually influence university participation (Gorard et al., 2012).However, the related but distinct attitude of expectations (whether students think they will be able to go to university), may be a more important influence upon attitudes and decision making towards university (Harrison & Waller, 2018), and expectations have been found to be lower among disadvantaged students (Boxer et al., 2011).Thus, the potential impact of widening participation interventions is uncertain; even if programmes designed to raise aspirations to university are successful, they may not impact actual university attendance unless other factors like expectations and academic attainment are also increased.

This review
This review provides an overview of the evidence base on UK widening participation interventions.The lack of peer reviewed causal/experimental evaluations in the UK (Robinson & Salvestrini, 2020;Younger et al., 2018) could mean that evaluations are simply not taking place.However, this is unlikely given that approximately £800 million is spent by universities on outreach work (Office for Students, 2019).The more probable explanation is that evaluations of widening participation interventions are happening in the UK, but they are not of experimental design.Thus, this study used a systematic review methodology to examine the efficacy and outcomes of UK-based widening participation programmes, the range of study designs used and the quality of that evidence.Given the varying quality of widening participation intervention evidence highlighted previously (Gorard & Smith, 2006), a systematic methodology was chosen to enable a thorough assessment of evidence quality and bias across multiple study designs.This approach allows us to answer our specific research questions by illustrating what is known about UK WP interventions for school children and what is not known (Gough & Thomas, 2016).
Inclusion criteria were kept broad to capture studies using descriptive, qualitative, or mixed methods approaches.To maintain a quality control, only peer-reviewed studies were included, and grey literature was not searched.The review examined studies over the previous 20 years, from 1999 up to 2019, updating and checking the evidence base to see if a lack of experimental evaluation of UK-based interventions remains.
This study comprised a systematic search and review of the evidence published on interventions carried out by UK universities, either within academic departments or through specific widening participation and outreach teams, or equivalent.The included interventions were all aimed at motivational or attitudinal changes towards university and had explicit outcome measures or objectives related to such factors.Participants were school children in the UK, aged 16 or below, as they are in compulsory full-time education.Students above age 16 in the UK have already chosen options, be it A-level or alternatives, which will have some bearing upon their desire or likelihood of attending university.Post-16 children may follow many pathways after secondary school, including vocational qualifications or apprenticeships which route students directly into a career rather than university.Thus, this review focussed upon interventions that target students before this point in their education, prior to such choices being made.A large-scale review of Australian and international evidence on widening access for disadvantaged students found that early, and sustained intervention through pre-Year 11 1 schooling was a key feature of successful programmes (Gale et al., 2010), and intervention evidence that is available in the UK is largely based on older secondary and post-secondary pupils (Robinson & Salvestrini, 2020).Thus, this review adds to the knowledge base on the efficacy of UK-based widening participation interventions for UK school children pre-16, highlights the different forms of evaluation that have been subject to peer review, and assesses the quality of the evidence available.

Research questions
The aim of this review was to gain a picture of the available peer reviewed evidence in the UK, its quality, and the evidence of intervention efficacy.The research questions were: (i) What evidence is there for the efficacy of UK-based widening participation intervention programmes in changing attitudes and/or behaviours towards Higher Education in school children?(ii) What form does this evidence take, and what is the quality of the evidence available?

Eligibility criteria
Inclusion and exclusion criteria for the review were developed in accordance with a PICOS (Perry, 2012) framework, and addressed the participants, intervention type, comparisons, outcomes and study design.Information, advice and guidance-based widening participation intervention evaluations aimed at UK school children up to age 16 from 1999 to 2019 were included.Full inclusion and exclusion criteria are listed in online resource 1.

Information sources
Search terms were developed through referencing terms used in previous reviews (Younger et al., 2018;Gorard et al., 2012), and refined using a series of scoping searches.The databases employed were ERIC, PsycInfo, British Education Index (BEI) and Web of Science, to provide a substantial range of coverage across psychological, sociological and education literature.All searches were carried out between July and August 2019.Papers were collated and reviewed using Zotero (Roy Rosenzweig Center for History and New Media, 2021).

Search strategy
Initial scoping searches were carried out to refine the search terms across all four databases.Once search terms and inclusion/exclusion criteria were determined, searches were conducted consecutively across the four databases, starting with ERIC, PsycINFO, BEI and last Web of Science.Search strategies for each database are in supplementary Table 2. Study selection followed the PRISMA checklist (Page et al., 2021) and was logged in a flow diagram (Figure 1) (Moher et al., 2009).
A first screen of papers using title and abstract was conducted using the established criteria.Inclusion was kept broad at this stage where there was ambiguity.The initial search returned a sample of n = 2419 after de-duplication.An additional study was discovered through review of a linked paper (n = 1).A 20% sample was double screened by a research assistant for quality control.Following discussion of inconsistencies in inclusion, inter-rater reliability was deemed appropriately high with a Cohen's kappa (Landis & Koch, 1977) value of .982obtained.Studies remaining for next-stage screening were n = 235.
A second screen of remaining studies was carried out by examining the full text of studies, using the same inclusion/exclusion criteria.A second stage double screening of a 20% sample was carried out.Following discussion with an additional reviewer, a Cohen's kappa (Landis & Koch, 1977) of .846was determined.Following this second stage of screening, studies retained for inclusion were n = 11.

Data collection process
Quantitative data were extracted using Cochrane Developmental, Psychosocial and Learning Problems data collection form for intervention reviews for Randomised Control Trials (RCTs) and non-RCTs (Cochrane, 2019).Controlled experimental studies were summarised and quality assessed using elements adapted from RCT components of the Cochrane data extraction form (Cochrane, 2019) and study characteristics and quality assessment tables from Younger et al. (2018).Uncontrolled quantitative and mixed methods studies were extracted using components of the same developed form.Qualitative studies were extracted, and quality assessed using guidance from the Cochrane Qualitative and Implementation Methods Group (Noyes et al., 2018) and CerQUAL approaches (Lewin et al., 2018).

Risk of bias in individual studies
The aim of this review was to systematically extract all the available studies carried out in the UK related to widening participation in UK school children.The study inclusion criteria deliberately contain all study designs, whether experimental or not, that have been peer reviewed.This was to gain a picture of the type and quality of evaluations being carried out in the UK, as it is already established that there is a lack of high-quality, experimental evaluations or studies providing causal evidence in a UK setting (Robinson & Salvestrini, 2020;Younger et al., 2018).The risk of bias in estimation of the effects of the interventions in individual studies was therefore assessed given the context of the study design, using guidance drawn from the Cochrane Systematic Review version 5.1.0handbook (Higgins et al., 2011) and Cochrane Qualitative and Implementation Methods Group (Noyes et al., 2018) and CerQUAL approaches (Lewin et al., 2018).Bias was assessed across multiple domains, including selection bias in the sampling, performance bias in the study design and intervention delivery, detection bias in the outcomes, attrition of the sample and reporting bias in analysis plans and reported findings.

Summary measures and synthesis of results
The scope of this review encompassed peer-reviewed studies in any form, thus no single approach to summary measures was applied across the extracted data.Extracted data were grouped by study design: controlled experimental and mixed methods evaluations, uncontrolled quantitative and/or mixed methods studies and qualitative study designs.Only one paper reported effect sizes; as such, no meta-analysis of quantitative data was carried out.Results from each group of study design were assessed for quality of evidence and methodological limitations, then summarised and emergent themes around successful methodological approaches, and study outcomes were synthesised using thematic analysis (Noyes et al., 2018).Risk of bias and overall quality and confidence in the studies were rated on a three-point scale using terminology adapted from the Cochrane handbook for Systematic Reviews of evidence (McKenzie & Brennan, 2019); studies were rated as low risk of bias, some concerns, or high risk of bias.

Review of findings
Controlled experimental and mixed methods evaluations Evidence summary and quality assessment.Two studies were controlled; one was of a randomised experimental design (Siddiqui et al., 2019), and one of a non-randomised quasi-experimental mixed methods design (Hatt et al., 2009).Summary tables and assessment of evidence quality are available in supplementary Tables 3 and 4. Siddiqui et al., (2019) used a randomised-controlled trial to evaluate the multi-component, long-term Children's University intervention aimed at primary school pupils, intended to primarily target Free School Meals (FSM) pupils.Hatt et al. (2009) evaluated a short-term summer school for Year 11 students who were potential first-generation university entrants, employing a positive control in the form of alternative Aimhigher interventions to evaluate part of the intervention.Aimhigher was an umbrella strategy of multi-faceted widening participation programmes that ran nationally in England from 2004 to 2011 (Doyle & Griffin, 2012).
Teacher rated change in attitude to university was the only outcome in the Children's University study (Siddiqui et al., 2019) relevant to this review.These views were not a predetermined key outcome but elicited through discussion with teachers and staff postintervention only.This did not form part of a formal, methodologically driven piece of qualitative research, as it was not the prime focus of the evaluation.As such, though the quality of the study itself was very high, the relevance of the study to answering this review's questions was fairly low.
Outcomes of the Aimhigher summer school programme (Hatt et al., 2009) were highly relevant to the review, as they directly examined both attitudes towards higher education, and university applications.However, high levels of bias across multiple domains meant that there were some concerns about the quality of evidence in answering the review questions.The researchers attempted to mitigate these issues through taking a multistranded approach to their evaluation.However, lack of explanation of analysis plans or methodological approaches particularly regarding qualitative aspects result in a study that is largely descriptive.Even though a positive control comparison group was used in the tracking data, there was no discussion as to the demographics, sampling and recruitment, or any attempt at matching these groups.It is also unclear which exact interventions these groups received, and if these were comparable in content or temporality.

Thematic analysis
A progression of aspiration?Common themes across the two studies were minimal, which was unsurprising given their divergence in most aspects of study design.The key relevant outcome from the Siddiqui et al. ( 2019) study was a perception among students that the programme had encouraged them to think about higher education; similarly, teachers believed the programme had impacted upon students' aspirations.In contrast, students taking part in the Aimhigher summer school (Hatt et al., 2009) had already been assessed as having largely pre-existing aspirations to go to university, again perhaps unsurprising given the disparity in participant ages between the studies.Students were instead reporting an improved enthusiasm for and confidence towards going to university, consolidating existing aspirations.The different nature of the two studies is such that direct comparison or synthesis of findings would be inappropriate, yet it may be useful to consider the findings as some evidence of the utility of establishing a continuum of aspiration; students in the younger age group reported an increased consideration of university as an option (Siddiqui, 2019), the Year 11 students found the summer school to be an affirmation of pre-existing goals to attend university (Hatt et al., 2009).Such a chain of associations is missing several links of evidence, though a perceived positive effect on aspirations at both primary and secondary level might point to the importance of widening participation work with a range of pupil ages.

Uncontrolled quantitative and mixed methods studies
Evidence summary and quality assessment.Six studies employed an uncontrolled design using quantitative data, or mixed methods (Casey et al., 2011;Hatt et al., 2007;Hatt et al., 2008;La Velle, 2013;Lewis, 2014;Maras et al., 2007).A summary of findings is presented in Table 5 and a summary of quality assessment in Table 6 as online supplements.
This group of studies were largely mixed methods.It should be noted that all the mixed methods studies used descriptive quantitative approaches only, with no inferential analysis.The studies were assessed for bias outside of that inherent in an uncontrolled study design; bias across studies is discussed further on.All studies in this group were aimed at secondary school aged children, and four of the six evaluated, or included elements of, an Aimhigher intervention (Hatt et al., 2007;Hatt et al., 2008;Lewis, 2014;Maras et al., 2007).Hatt et al. (2007Hatt et al. ( , 2008) ) examined the impact of the Aimhigher initiative in the south west of England which was targeted towards low-SES pupils, firstly using a mixed methods evidence synthesis including tracking of 580 Year 10 pupils, and latterly from a teacher's perspective; Lewis et al. (2014) evaluated a specific Aimhigher programme targeted to disadvantaged and first-generation pupils, framed around an archaeological excavation programme, but with a wider remit to raise aspirations and improve skills; Maras et al. (2007) used a different approach to examine associations between takeup of a range of widening participation activities within a group of schools, some of which included Aimhigher activities, and attitudes towards university as well as covariates such as FSM status, among over 2700 London school children.Casey et al. (2011) evaluated a widening participation programme for 80 gifted and talented disadvantaged children, and La Velle (2014) evaluated a university-school partnership programme for 565 pupils in Plymouth targeted to schools in disadvantaged areas of Plymouth with high Free School Meal levels.The studies providing the highest quality of evidence in minimising bias in the context of their design were Casey et al., 2011;Hatt et al., 2007;Hatt et al., 2008;Lewis, 2014.Thematic analysis Combining information.Casey et al.'s evaluation (2011) established a methodology and analysis plan prior to data analysis.Triangulation of data strengthened their conclusions, for example patterns around enhanced aspirations and self-belief following attendance on campus were supported by questionnaire data indicating an increase in students intending to apply to university, though this was descriptive data analysis only.The authors also acknowledge the lack of comparison group rendering weak any claims about grade improvements but note anecdotally that some teachers observed an increase in motivation, and consequent target grade achievement among the group of students enrolled on the programme.The authors appear to have been creative in their approaches to dealing with challenges of selection bias.They acknowledge the complexities of selecting students for the programme, particularly in identifying gifted and talented students in the context of their backgrounds, taking a multi-pronged approach in close collaboration with teaching staff.Synthesis of multiple data sources and using an established qualitative methodology led to this evidence being judged of moderately high quality given the study design.Lewis et al. (2014) used a similar methodology to evaluate a programme with a slightly different focus; the Higher Education Field Academy (HEFA) at the University of Cambridge operated under the umbrella of Aimhigher.It had similar aims to raise aspirations towards higher education and enthusiasm for learning.This was delivered through the prism of an archaeological excavation programme.The short programme, a two-day excavation followed by a one-day workshop on campus, was repeated with many cohorts across eight years, with almost 3500 students.The authors used a similar approach to data collection to Casey et al. (2011), triangulating data from various sources including student pre-post questionnaires, interviews, field notes and written self-reports from the students to build a multi-layered picture of the impact of the programme.Longitudinal data of Year 11 students' studying intentions were also integrated as part of the follow-up evaluations.Combining layers of data supported the findings of an increased intention in students to apply to university, and more positive feelings about applying to University of Cambridge specifically.Hatt et al. (2007Hatt et al. ( , 2008) ) took different approaches to evaluating the impact of the Aimhigher programme in the South West of England.First, they conducted a multi-faceted mixed methods review of the programme's impact (2007).This synthesis of evidence gathered secondary data, intervention self-report data, and implemented a tracking study, to build a rich and layered view of the impact of the intervention on students' attitudes towards HE, intention to attend, and progression rates.There is inherent bias in the lack of clarity around individual participants for individual elements of the programme, though there is participant data available for the tracking study element of the evaluation.However, strong triangulation of the data sources into themes allows for a clear overview of the impact of the programme.The authors reported an increase in awareness and aspirations.Linking the programme to attainment, and then onto HE progression is not possible as the authors note, but Aimhigher participants were more likely to continue and remain in Level 3 study.Following this evidence synthesis, Hatt et al. (2008) used a mixed methods evaluation to add to understanding the impact of the South West Aimhigher programme, examining teacher perceptions of its perceived impact.Teachers from a self-selecting sample of the pool of Aimhigher participating schools in the South West of England commented on their perceptions of students' awareness of Higher Education, aspirations, motivation, and attainment following the programme.This encompassed a wide range of activities.Such data cannot provide a view of a particular component's efficacy, but an overview of the general effects of participating in the activities across the intervention.Teachers reported students as having increased awareness, aspirations, and motivation, believing that university was a possible option for them, though were more equivocal on the effects on attainment.Teacher comments noted the impact of Aimhigher on student knowledge of university, and that attendance on campus opened up university as a possibility for the students.
These four studies all examined programmes with clear aims to raise aspirations and had outcomes highly relevant to this review.All four indicated the programmes had an impact upon raising aspirations, though these were assessed in slightly different manners; the zoomed-out view of Hatt et al. (2008) on the Aimhigher programme taken alongside the detailed, more focussed pupil reported data of Lewis et al., (2014) on a specific element provides a differentiated view of its impact.The multiple data source approach within a study taken by Hatt et al., (2007), Casey et al., (2011) and Lewis et al., (2014) helped to build discussion around more descriptive quantitative and qualitative data, supporting interpretations where inferential analysis of data compared to control group data was lacking.
Combining sources of information to strengthen findings was a theme of the higherquality studies in this group.However, employment of mixed methods does not automatically produce a nuanced understanding of the impact of an intervention, where the data are not triangulated and synthesised together.The evaluation of the Plymouth Model programme (La Velle, 2014) used student and teacher self-report questionnaires pre-and post-intervention, alongside student focus group interviews to evaluate their month-long programme aimed at encouraging disadvantaged students to aim for university.Teacher views on student attitudes were discussed, but with little integration with student views.Discussion of the focus group interviews was limited, and again this was not synthesised with the student survey data.Further to this, the high level of reporting bias in the student quantitative survey data, already descriptive only, weakens the reliability of the reported positive findings.
Alternative methodologies.Hatt et al.'s (2008) different approach to evaluation through teacher perception helps to add an additional layer to understanding the impact of the Aimhigher programme.When taken in the context of the difficulty in running a randomised controlled trial to evaluate widening participation programmes, making creative use of alternative evaluation methods may offer opportunities to gain some insight into their efficacy, whilst acknowledging the limitations of non-experimental methods.
An alternative means of evaluation not used by the aforementioned studies is the quantitative, cross-sectional approach taken by Maras et al. (2007).Developed out of the first section of a longitudinal study, the authors examined a range of factors as predictors of take-up of widening participation activities and attitudes to higher education.As a directly relevant outcome to this review, the study assessed the association between participation in WP activities and attitudes towards university.Take-up of WP was selfreported by students ticking which activities they had been involved in from a list, which included some Aimhigher events.The study (n = 2731) reported findings indicated that take-up of WP activities was positively associated with attitude towards higher education, though there were other factors measured of greater predictive value.As there was no comparison group, it is difficult to ascertain whether such association is down to the activity, or a pre-existing positive view of university, or other confounding factors.Interactions between factors were not examined, though it is noted that the WP activities were targeted to higher ability students.Overall, the sample characteristics, the type of activities and their aims, and the level of engagement are unclear; multiple dimensions of bias were assessed as high or moderate, sometimes owing to lack of clarity, and as such the study cannot provide much evidence beyond the potential for positive associations with participating in WP activities.
A difficulty of taking such a broad-stroked approach to evaluation is the risk of bias within the study design, particularly performance bias, where participants may be exposed to factors other than the intervention which could influence outcomes (Higgins et al., 2011).Fidelity of experience across the programmes is unclear or not commented upon, and temporality of data collection in relation to the experience of the interventions is unclear at an individual level, particularly in Maras et al.'s (2007) study.Such issues require clarity and discussion to build a higher-quality base of evidence from alternative evaluation approaches.
Subject-focussed intervention.The subject focus of the HEFA archaeology programme (Lewis et al., 2014) was not the key aim of the programme; as the authors discuss, it was not intended as an intervention to recruit more archaeologists.Rather, the particular activities and skills learned during an excavation were identified as a crucial component in the success of the intervention.Findings from student questionnaires, reports, and interviews highlight how the element of discovery and a perception of the importance of the work they were doing helped to give the students a sense of real contribution to university research.This is of interest as an aspect that is perhaps not given as much attention in widening participation activities.Further to this, the purposeful activity appeared to give students a sense of responsibility and achievement, developing soft skills that students believed would be helpful in the future.
Framing an intervention within a particular subject area yet expanding the focus of the intervention beyond increasing study of the subject, may offer opportunities to develop students' aspirations and skills in a nuanced and engaging manner.Investigation of the reasons and mechanisms for such changes could also add to the evidence base for such interventions.Analysis which makes reference to theoretical constructs such as self-determination theory, whereby an intervention may be understood through its effects upon increasing autonomous motivation for example (Ryan & Deci, 2016), or through helping students to broaden and redefine their possible or probable "future selves" (Harrison & Waller, 2018) would underpin evaluations with an understanding of not just if they work, but why.

Qualitative studies
Evidence summary and quality assessment.Three studies employed a qualitative study design.A summary of findings is presented in Table 7 and a summary of quality assessment in Table 8 as online supplements.All three qualitative studies took different approaches to their evaluation methodology, though this amounted to only moderately reliable evidence in answering the review questions in each evaluation.Moogan et al. (2011) evaluated a long-term intervention programme using a longitudinal case-study approach with 38 Year 11 pupils from an Aimhigher target school with a very high proportion of disadvantaged pupils; Murphy (2002) used a post-intervention case study approach to evaluate a short campus-based programme for 12 pupils from two schools with low university progression rates; Raven (2015) assessed the use of photography as a novel evaluation methodology for a campus day-visit for 30 Year 10 pupils from schools/areas of low university participation and high deprivation.

Thematic analysis
Improving knowledge.Changes in knowledge of varying aspects of university form a thread between the outcomes of all three studies; however, the links between increased knowledge and changes in attitude towards university are demonstrated to differing degrees.The strongest evidence came from Murphy's (2002) case study.Changing attitudes towards university was a clear outcome of the study, and the authors used teacher and student interviews to examine the impact of a four-day campus programme.One of the head teachers believed that a change in student attitude came about through meeting students who were from their area, and the knowledge that not just one group or type of person goes to university.This change in perception is echoed by student comments, such as their surprise at the friendliness and normality of the staff and students.Students and one of the head teachers reported an increased confidence that they could attend university.The consideration of both teacher and student responses offers some strength to the interpretation of the study outcomes, particularly with the inclusion of both present and past participants in the student interviews.The authors acknowledge the bias in the sample, as students were selected for participation by school staff, and then a sub-set of these from one school selected by the head teacher for interview.They also point to the clear enthusiasm of the interviewed students for the programme.
Taking a longitudinal approach to case-study, Moogan et al. (2011) evaluated a longerterm, multi-faceted programme, following a group of Year 11 students from an inner-city school.The aims of the programme were similar to Murphy (2002) in providing information around Higher Education, with a specific focus on student choices.Data collection in the autumn term revealed a lack of knowledge about university course structure, and the authors discuss this as an important issue in terms of student retention once at university.Students discussed the risks of moving away, of non-completion, of money and debt.Students then attended a range of workshops, talks and campus visits.Further data collection in the spring term shifted in focus, onto subject choices.Thus, the comparison of pre-to post-intervention in this study is not clear and weakens the evidence for any change in attitude following the intervention.One theme identified as illustrating change in knowledge was the students' understanding of the structure of a university course, for example that they would not have to attend all day in the manner of a school week.Comments were highlighted as illustrating this change in knowledge and a commensurate shift in consideration of the possibilities of attending university, for example the newly imagined possibility of working to support themselves whilst studying.Such a consideration was evidently important to this group of students from the comments made.However, any connection between increased knowledge and shift in attitudes is tenuous, as it was not directly measured or discussed in a comparable manner across time-points.
Last, a novel approach to evaluation was piloted by Raven (2015); the use of photographic evaluation is novel to widening participation activities.Pupils on a one-day campus visit were placed in small groups with a university student helper who documented pupil attitudes and beliefs about university using photographs.The group then reviewed and commented on the photographs at the end of the day.This was supplemented by post-intervention questionnaires, and interviews with staff and helpers.This multi-source approach to data collection offered an opportunity to compare more established with newer methods and discern any added value in their use.However, while the photographic element appeared to offer possibilities for more detailed description of student knowledge about university, as the data were collected over the course of the day with no pre-intervention discussion, the method does not elucidate any changes in knowledge following the intervention programme.The student questionnaires indicated an increased desire to attend university, and the programme leaders felt that students increased their knowledge of what university was like following the event.This is acknowledged by the authors, and the results must be considered in the context of a pilot study.Use of such methodology in conjunction with more established survey methods, and pre-intervention data collection could offer an unusual and participantled insight into the impact of an intervention, but this study has limited scope to add to the evidence base on efficacy of such WP programmes.
A strength of all three studies was the use of multiple data sources.As with the mixed methods studies this added to the persuasiveness of the study findings; however, improvement in capturing changes in knowledge and attitudes would be necessary to consider the quality of the evidence as high.

Risk of bias across studies
This review was conducted to examine a gap already identified in the literature, namely the lack of experimental/causal evaluation of UK-based widening participation programmes (Robinson & Salvestrini, 2020;Younger et al., 2018).The review was thus deliberately narrowed to UK-based studies but broadened to include any study design.The purpose of this was to identify if the gap in literature was due to a lack of robust study design, or a lack of peer reviewed research in the area altogether.Only two controlled evaluations were included (Siddiqui et al., 2019;Hatt et al., 2009), and only one of these (Siddiqui et al., 2019) included any element of randomisation; most of the evidence in this review has the inherent issues of bias across multiple dimensions, which are discussed in the study limitations below.As such, the evidence synthesised in this review was assessed for bias in the context of the different methodologies.The main strength of the range of mixed methods and qualitative studies in reducing bias was the inclusion of multiple data sources, though data from these sources were synthesised to varying levels of quality.

Summary
The evidence synthesised in this review presents an overall positive picture of the effects of various widening participation programmes.A common theme is that students enjoy the programmes and teachers appear to feel broadly positive about their effects.Raised aspirations are a frequently reported outcome, however none of the studies explicitly addressed raising pupils' expectations as an important component of decision making about university (Harrison & Waller, 2018), though Casey et al. (2011) and Lewis (2014) reported increased intention to attend.This distinction is important, as disadvantaged pupils are likely to have lower expectations than aspirations compared to their more advantaged peers (Boxer et al., 2011), and it may be a contributing factor to lower university attendance.Lower expectations of higher education may be driven by perceived or actual low attainment, or by external factors such as financial stress (Kaye, 2021).
Incorporating assessment of university expectations as an outcome is an important area for improvement in future evaluations.
The body of evidence here only provides moderate support at best for the efficacy of these programmes.Despite a generally positive impression from the findings across the studies, most of them have at least moderate levels of bias, with limitations in the application of both qualitative and quantitative methodologies.Randomised controlled trials can be difficult and expensive to run, which makes the high-quality trial run by Siddiqui et al. (2011) impressive, and they may not even be appropriate in all cases given the methodological difficulties.However, there was only one other study that employed any form of control or comparison group (Hatt et al., 2009) and this had several limitations.Arguably, a poorly controlled study is not much better than an uncontrolled study, but the deficiencies in design, data analysis, and levels of bias may potentially be mitigated without the difficulty and expense of a randomised trial.
It is also important to acknowledge the changing context in which widening participation interventions are delivered.Interventions under the Aimhigher umbrella between 2004 and 2011 focussed on aspiration raising, which largely continued under the more dispersed model of widening participation in recent years.Moving forward under revised Access and Participation Plans for universities, a renewed emphasis on collaboration with schools earlier in pupils' educational timeline and support for attainment (Office for Students, 2022) will require a shift in focus from widening participation practitioners to help enhance pupils' expectations through greater confidence in their ability and their attainment results.
In consideration of this moderate level of reliability and the changing WP landscape, there are some specific aspects of the interventions that are highlighted as potentially useful for improving both aspirations and expectations, and may be worth further, more focussed, and rigorous evaluation.So called "black-box", multi-component interventions are common, but little work has been done to evaluate the impact of individual components (Robinson & Salvestrini, 2020;Younger et al., 2018).Moreover, these particular intervention approaches have the advantage of being broadly applicable across heterogenous education systems and may offer a useful focus for further exploration in an international context where knowledge gaps exist.

Skills development
Two of the higher-quality studies (Casey et al., 2011;Lewis et al., 2014) note the benefits and enjoyment of pupils felt through the development of specific skills.Lewis et al. (ibid) highlight how the use of an archaeological excavation as a vehicle for widening participation was particularly effective, as it gave students a sense of ownership and responsibility.Their contributions to the excavation were taken seriously, and as such students felt they were taking part in real research.Their commensurate skills development was given further weight and meaning through rigorous and standardised reporting and reflection processes.Casey et al. (2011) also found positive student responses to the problem solving and critical thinking aspects of the programme, as they could apply these skills to other areas of their schooling; parents felt that such skills helped to ground students' aspirations in reality and help them develop a future plan for their education.Interventions that focus on specific skills, with a tangible outcome may be a more rewarding way to develop aspirations and expectations of university attendance, and the focus of the skills in the intervention may be adapted to suit local and national skills shortages and developing work sectors within varying international educational and economic contexts.

Visitors and role models
The use of persons from within the realms of university and/or career areas as information sources and/or role models was a recurring theme.Students felt a new understanding that university was not just for people outside of their social group upon meeting university students from their own locale.Attending on campus and meeting staff changed perceptions in the friendliness of university students and staff (Murphy, 2002); feeling welcome, and seeing reflections of themselves in the student body appears to have been impactful for these disadvantaged students.Hatt et al. (2007) found elements of the Aimhigher programme in the south west of England to have been particularly successful in raising aspirations when involving HE students.Contact with outside speakers also appears to have had a positive effect; students have made use of contacts and sustained these in some cases to develop their career ambitions (Casey et al. 2011).Again, such an approach may be readily adapted to suit differing international contexts for underrepresented and marginalised groups of students.

Limitations of the data
Within the individual studies, bias was a barrier to the evidence being judged high quality in most cases.Targeting and recruitment of students is a key issue for selection bias.Response levels, attrition, prevalence, and treatment of missing data are unclear or unexplored in many of the studies reviewed.This results in samples that may often be unacknowledged sub-sets of the original sample, and the potential for additional bias in undisclosed methods of dealing with such.Attrition and incomplete data in such interventions may be unavoidable, but lack of transparency in the prevalence of this, and the manner of dealing with missing data are common weaknesses of social science research (Berchtold, 2019).
Another issue particularly with the mixed methods studies, was the potential for reporting bias through lack of establishment of analysis plans in the methods section of evaluations.Establishing a clear methodology for both qualitative and quantitative aspects of the studies, and analysis methods, would improve the quality of evidence from mixed methods designs and reduce such bias (Boutron et al., 2019).
Aside from the issue of bias, selection of students for interventions is a potential problem for implementation of programmes, for example students may feel singled out (Murphy, 2002), or there may be difficulty in identifying individuals for the intervention (Hatt et al., 2008;Casey et al., 2011).Moreover, schools may have to reconcile the aims of the widening participation initiatives, and their desire to provide exposure to these experiences for all students (Hatt et al., 2008).Resolving these issues in a manner that offers the opportunity to reduce selection bias seems to be an area for exploration for future studies; for example, integration of elements of randomisation at a class level with a wait-list control design in the planning of WP interventions.This would help to reduce some of the variability in the data, whilst providing a more practical and equitable sampling option from the perspective of participating schools.
Finally, a lack of strong theoretical underpinning to several of the interventions and the evaluations of those programmes limited explanations of how interventions were acting to effect change in university attitudes and behaviours, for example through individual difference in motivation drawn from psychological theories.Though some of the studies alluded to ideas drawn from sociology such as habitus (Hatt et al., 2009), social and family factors of influence (Maras et al., 2007;Moogan, 2011), or the need for selfknowledge and goal development (Casey et al., 2011), the studies in the main did not explicitly link the evaluations to any psychological or sociological theory as a causal mechanism.Although la Velle et al. ( 2013) make reference to theories upon which the intervention in their study was founded, including social cognition, this theoretical foundation for the programme was not incorporated into its evaluation.Thus, future WP programmes and evaluations aiming to establish a casual mechanism for change following the programme should examine and explicitly state the theoretical basis upon which the intervention is founded, and use said theory to examine programme effects and impact.This will help strengthen the evidence base for WP intervention efficacy.

Limitations of the review
The main limitation of this review was established through the discovery of a relevant study (Lewis et al., 2014), through indirect means, via another related paper that did not meet inclusion criteria.The paper did not appear in initial searches as it was published in the Journal of Public Archaeology.Searches in Web of Science were restricted to certain subject criteria, and so this subject specific journal was not included.This raises the possibility that other subject focussed widening participation interventions have been published in more specific disciplinary journals, and not brought within the scope of this review.Subject specific interventions are of interest as some groups of students are still underrepresented in certain areas, for example high-tariff subjects like medicine (Gore et al., 2018).Future reviews may wish to examine the prevalence of such programmes, and how well they are integrated into general widening participation literature.
This review also limited studies to those from peer reviewed journals.This was a deliberate choice to examine how much of widening participation research was subject to the same quality control processes as other research.As such, there may be a wealth of information that exists within grey literature, unpublished internal reviews, and books.Robinson & Salvestrini, (2020) obtained some unpublished data from a call for evidence to UK university WP practitioners, though this was by nature a self-selecting sample.Future reviewers may wish to examine the extent of unpublished literature and why so little evaluation from widening participation programmes is subject to peer review and formal publication.
Last, this review was limited to evaluations of information and/or activity-based interventions with secondary and primary school aged children.This was to restrict the focus of the studies to those working with students still in compulsory full-time education, where drivers and motivations for choices may be rather different to those of a student in post-compulsory education.Students who have chosen to remain in education may be more likely to want to attend university as they have already taken a step towards doing so in attending sixth-form or college post-16.As such, motivational interventions in this age group may be less effective, or act differently e.g.motivate students towards a higher tariff course.Alternative approaches to widening participation such as contextual admissions were not included.Though they form part of a range of tools to increase participation, they act in different ways to widen access, and thus are outside the scope of this review.

Conclusions
There has undoubtedly been a range of widening participation work being evaluated in the UK over the last two decades.However, a gap remains in rigorous, experimental evaluation.There is also limited underpinning by psychological theory to explain any mechanisms for change in attitudes.The range of study designs that have been used have largely been uncontrolled, mixed methods studies, or qualitative in nature.There is value in such information, but the bias in the studies reviewed here is typically moderate to high, and methodological issues with the mixed methods studies in particular mean the evidence base is not as strong as it could be.Causal inference is not possible in the majority of cases, and the lack of clarity in study design and methods render many evaluations unreliable.Implementing randomised controlled evaluations of widening participation interventions is not necessarily financially viable, or practical.Recruitment of schools to studies is difficult and incorporating an element of randomisation whilst implementing an evaluation that is perceived as fair and accessible to students, and minimising disruption to day-to-day schooling is extremely challenging.Where randomisation is not possible, use of non-randomised control groups, and a greater adherence to transparency and clarity in study design, data handling, and analysis plans could improve the quality of evaluations.