Evaluating youth-friendly health services: young people’s perspectives from a simulated client study in urban South Africa

Background Few youth-friendly health services worldwide have been scaled up or evaluated from young people’s perspectives. South Africa’s Youth Friendly Services (YFS) programme is one of the few to have been scaled up. This study investigated young people’s experiences of using sexual and reproductive health services at clinics providing the YFS programme, compared to those that did not, using the simulated client method. Design Fifteen primary healthcare clinics in Soweto were randomly sampled: seven provided the YFS programme. Simulated clients conducted 58 visits; young men requested information on condom reliability and young women on contraceptive methods. There were two outcome measures: a single measure of the overall clinic experience (clinic visit score) and whether or not simulated clients would recommend a clinic to their peers. The clinic visit score was based on variables relating to the simulated clients’ interactions with staff, details of their consultation, privacy, confidentiality, the healthcare workers’ characteristics, and the clinic environment. A larger score corresponds to a worse experience than a smaller one. Multilevel regression models and framework analysis were used to investigate young people’s experiences. Results Health facilities providing the YFS programme did not deliver a more positive experience to young people than those not providing the programme (mean difference in clinic visit score: −0.18, 95% CI: −0.95, 0.60, p=0.656). They were also no more likely to be recommended by simulated clients to their peers (odds ratio: 0.48, 95% CI: 0.11, 2.10, p=0.331). More positive experiences were characterised by young people as those where healthcare workers were friendly, respectful, knew how to talk to young people, and appeared to value them seeking health information. Less positive experiences were characterised by having to show soiled sanitary products to obtain contraceptives, healthcare workers expressing negative opinions about young people seeking information, lack of privacy, and inadequate information. Conclusions The provision and impact of the YFS programme are limited. Future research should explore implementation. Regular training and monitoring could enable healthcare workers to address young people’s needs.

healthcare facilities) aims to improve the sexual and reproductive health of young men and women (6). The YFS programme has previously been highlighted as a successful model for implementing youth-friendly services within a public health system (7). However, evaluations of this programme have focussed on the attainment of pre-defined standards relating to the services provided, policies supporting adolescents' rights and the clinic environment; just one study (conducted in 2005) investigated adolescents' experiences (8Á11). The Department of Health (DoH) took over the management of this programme [previously known as the National Adolescent Friendly Clinic Initiative (NAFCI)] from the non-governmental organisation (NGO) loveLife in 2006, and no evaluations have taken place since (7,12). Recent work has suggested that YFS provision is limited with just one of eight clinics in rural Mpumalanga reporting implementation of this programme (13).
This study had two objectives: the first was to investigate whether primary healthcare facilities providing the YFS programme delivered a more positive experience to young women requesting information on contraceptive methods and young men requesting information on condom usage than primary healthcare facilities not providing this programme. The second was to examine the characteristics of more and less positive experiences.

Methods
The simulated client method, where the healthcare provider is not aware that a given client is participating in research, was used to address these objectives. This method removes observation bias and issues of privacy and confidentiality that may occur with direct observation or interviews with patients or healthcare workers. It has been used to study healthcare providerÁclient interactions by a number of studies in low-and middle-income countries (11, 14Á20).
Simulated clients have previously been recruited from groups including medical or university students, research assistants, nurses or people with particular sociodemographic characteristics (14,18,21). In this study, seven simulated clients (young men and women aged 22 years) were recruited from a long-running birth cohort of Soweto youth [described elsewhere (22,23)] as it provided a practical sampling frame and these young people were within the target age group of the YFS programme. As members of the general population they were also likely to be more representative of the programme's target population than research assistants. Simulated clients were randomly selected from a group of cohort participants who had previously participated in cohort health services-related research as this study formed an extension of that research programme. Young people were eligible to participate in the cohort health services-related research study if they tested positive for a sexually transmitted infection (STI) or urinary leukocytes at age 13 years. The simulated clients underwent training to present at clinics with two gender-specific scenarios, which were developed in collaboration with a clinician and two research nurses, with feedback from the simulated clients (Table 1).
A thorough response to either of these scenarios was defined as one which included: a discussion of correct condom use, the offer of a condom demonstration (to both young women and young men), assessment of STI exposure, STI/HIV counselling and testing, or referral. In addition, it would include the discussion of injectable contraceptives, oral contraceptives, and intra-uterine devices or systems with young women (24,25).
Training involved conducting scenario role-plays with research nurses. Simulated clients were told that the aim of this study was to find out what young people's experiences were like at the clinics, whether or not staff had been trained to try and make services friendly to young people. Local research assistants were present to provide language or cultural translations although this proved unnecessary.
Fifteen primary healthcare clinics, from a total of 29 in Soweto, were randomly sampled to receive 3Á4 simulated client visits each. At least one male and one female simulated client visited each clinic between November 2011 and March 2012. Simulated clients were not asked to visit the clinics nearest their homes, or to visit a clinic The young man has heard that condoms can break and would like to know how reliable they are.
The young man would also like a demonstration of how to put a condom on correctly.
The young man is sexually active with a girlfriend. They use condoms (this information is only given if requested).
Advice on contraceptive methods The young woman would like to know about how to prevent pregnancy.
The young woman currently uses condoms (this information is only given if requested) but would like to learn about other methods.
The young woman is sexually active with a boyfriend. They use condoms (this information is only given if requested).

Statistical and qualitative analyses
Definition of the exposure and outcome The exposure (YFS provision) was defined based on DoH signs detailing the services provided at each clinic. It was not possible to blind simulated clients to which clinics provided the YFS programme given these signs. However, simulated clients were not told which clinics provided this programme, that these signs were present or that YFS provision was the focus of the study. No additional data were available or could be obtained on which facilities the DoH classified as implementing this programme. Other exposures of interest were clinic characteristics and simulated client socio-demographic characteristics. Clinic characteristics were collected using a short questionnaire administered to the nurse-in-charge at each clinic, who provided clinic-level informed consent. Socio-demographic characteristics of simulated clients were available having been collected previously as part of their participation in the Birth-to-Twenty cohort.
Because there were a large number of individual questions or potential outcomes, those which between them were felt to capture information that would define a good experience overall were combined into a single measure of the simulated clients' overall experience during each clinic visit using principal components analysis. This clinic visit score was the first outcome measure. All collected variables were tabulated against each other, and those where there was evidence for correlation (p50.1) were included in the principal components analysis. Twenty-nine correlated variables were included in this measure, including those relating to the simulated clients' interactions with staff, details of their consultation, privacy, confidentiality, the healthcare workers' characteristics, and the clinic environment. A larger clinic visit score corresponds to a worse experience than a smaller score. As an illustration, the clinic visit with the smallest visit score ((4.71) had universally positive responses to the questions inputted into the principal components analysis. By contrast, the clinic visit with the largest visit score (2.47) had only three positive responses, which related to: the consultation room being clean and affording privacy, and the healthcare worker being respectful. The secondary outcome was whether or not simulated clients would recommend a clinic to their peers. Data were entered into Microsoft Access, statistical analyses were conducted in STATA 11, and framework analysis was performed in Microsoft Excel (26).
Multilevel regression models accounting for clustering at both simulated client and clinic levels were used to examine the association between YFS provision and the outcomes. Linear regression was used for clinic visit score, logistic regression for clinic recommendation. Univariable models were used to examine crude associations between YFS provision and each outcome, and between other exposures of interest (clinic size, local, or provincial government managing authority, presence of a peer educator, type of healthcare worker seen, simulated client gender, healthcare worker gender and age, maternal age at birth of the simulated client, and household SES) and outcomes. Any other exposure of interest associated with an outcome at pB0.05 in univariable models was included in the multivariable model for that outcome. Although experiences of health services may differ by gender, the small number of simulated clients (three males and four females) precluded conducting a quantitative analysis stratified by gender.
For the framework analysis, categorisation of themes was guided by Bruce's Quality of Care for Family Planning framework (27). Additional themes could also be defined. Themes were compared between the consultations with the best clinic visit scores (n015) and those with the worst score (n015), and between male and female simulated clients. Data saturation, where no new or relevant information emerged, was reached.
Ethical approval was obtained from the London School of Hygiene and Tropical Medicine and The University of the Witwatersrand (Number: M110360). Permission to work with the clinics was granted by the relevant provincial and district health authorities and informed consent was obtained from all simulated clients and the nurse-in-charge at each clinic.

Results
Fifty-eight clinic visits were conducted, 30 by female simulated clients and 28 by males. Thirteen clinics received four visits, two from male simulated clients and two from females: two clinics received three visits. Data from two of the 58 visits were not included in the analyses because the simulated client revealed their participation in the study. Refresher training was provided to these simulated clients. Table 2 presents the clinic and simulated client characteristics. Of the 15 clinics sampled, eight were small ( B6 nurses; 53%) and run by the local health authority. Approximately half of clinics provided the YFS programme (n07) of which five were small, local authority clinics. Table 3 illustrates some of the characteristics of the clinic visits; !70% (n043) of consultations were with a nurse or sister and !85% were with a female healthcare worker (n049). Only one variable was statistically significantly associated with YFS provision: whether a simulated client felt that the healthcare worker seemed happy to talk about condoms or other contraceptive methods with them. However, this association was driven by 14% of consultations at non-YFS clinics having no discussion of condoms (males) or other contraceptive methods (females) and a similar proportion of consultations at YFS clinics involving a healthcare worker who did not seem happy to talk about condoms or other contraceptive methods with them. Since these two situations are quite similar, the observed association is unlikely to be meaningful.
There were also no statistically significant differences observed between clinics that provided the YFS programme and those that did not in univariable analyses in terms of clinic size, managing authority, presence of a peer educator, healthcare worker gender, healthcare worker age and level, cleanliness, waiting times, whether the clinic was perceived to be welcoming, and the provision of information and education materials (data not shown). Sexual health histories were not taken in any consultation, nor was HIV/STI testing offered. Counselling on HIV/STI prevention was rarely provided, and correct condom usage was demonstrated in only half (59%) of consultations with males and to no female simulated clients.
There was no evidence that clinics that provided the YFS programme delivered a more positive experience to simulated clients than clinics that did not provide this programme, adjusting for the effect of simulated client gender, healthcare worker age, and clustering (mean difference in clinic visit score: (0.18, 95% CI: (0.95, 0.60, p0 0.656) ( Table 4). There was strong evidence that male simulated clients had generally more positive experiences than female simulated clients, with a mean difference of 1.52 in clinic visit score between consultations conducted by males and females (p00.009). There was also some evidence that consultations with older (compared to younger) healthcare workers were more positive experiences, adjusting for the effect of the provision of the YFS programme, simulated client gender and clustering (p00.041). For each one-unit increase in healthcare worker age group, the mean difference in clinic visit score was (0.59. There was no evidence that clinics that provided the YFS programme were more likely to be recommended by simulated clients to their peers than those that did not, adjusting for the effect of maternal age at birth, healthcare worker age and clustering at the simulated client and clinic level (odds ratio: 0.48, 95% CI: 0.11, 2.10, p 00.331) ( Table 5).
Characteristics of more and less positive clinic experiences Less positive consultations were those where information was not given, privacy was lacking, and simulated clients experienced unnecessary barriers or negative opinions about seeking information. More simulated clients at more positive consultations reported that the healthcare worker was friendly, knew how to talk to young people, treated them with respect, and appeared to value them seeking information.
A common unnecessary barrier was that in the majority of consultations, female simulated clients were told that they would be required to return and show soiled sanitary products before they would be prescribed contraceptives. However, South African guidelines state that the initiation of hormonal contraceptives should not be restricted to menstruation (24). One female simulated client said: 'She said I must come immediately I have my period and that I will have to show them the pad as many girls are coming when they are not on their period and are lying' [Simulated Client 5 (female), Clinic 12 (not YFS)].
Simulated clients perceived that the healthcare worker valued them seeking information at the majority of the more positive consultations, but not at the majority of less positive consultations. One male simulated client reported that; 'When I said I wanted to ask about condoms she said that she doesn't have time because she has to see those  Both female and male simulated clients, particularly those who had children, reported that healthcare workers expressed surprise at them seeking information on contraceptives or condoms. One female simulated client described the healthcare worker's reaction: 'She said that I have a baby now, I should know better than getting information on prevention from here' [Simulated Client 5, Clinic 3 (YFS)].
Judgmental attitudes were often linked to healthcare workers not providing certain information. For example, one female simulated client said: 'When I came in she just stared at me. I said I wanted to ask about the different methods of prevention and before she answered she asked how old my baby is. When I was surprised and asked how she could know I had a child she said she could see it in my body. She said I should take the injection and that we shouldn't go into the other methods. When I asked why she recommends the injection she asked how old I am and said that they don't recommend pills for young people because they are careless. I said I am not a party person' [Simulated Client 5 (female), Clinic 3 (YFS)]. Despite this  simulated client's assurances the healthcare worker refused to give information on user-controlled methods.
Characteristics of more and less positive clinic experiences by gender More male than female simulated clients reported that the healthcare worker they consulted knew how to talk to young people and treated them respectfully. This may reflect different experiences by males and females, different expectations or both. However, judgmental attitudes were more commonly exhibited towards female than male simulated clients. Male and female simulated clients emphasised healthcare worker behaviour and attitudes as the most important factors in determining more and less positive experiences.

Discussion
There was no evidence that clinics providing the YFS programme provided a more positive experience to simulated clients, or were more likely to be recommended by simulated clients to their peers, than those not providing this programme. These results are consistent with those of an earlier study where clinics providing NAFCI were no more likely than facilities not providing NAFCI to provide a more positive experience to young simulated clients seeking HIV tests (11). Positive and negative experiences were predominantly determined by the healthcare worker's attitudes and behaviour. This is in line with the findings of other studies that the characteristics most valued by young people are staff attitudes and confidentiality, and that improving services for young people should focus on changing attitudes rather than addressing structural issues (28Á30). However, these findings offer important insight because the YFS programme is one of the few such interventions to have been scaled up to a national level. Although the YFS programme includes healthcare worker training, these results indicate a need for improvements in healthcare workers' capacity to deliver positive experiences to young people, to address young people's needs for information, contraceptive methods, and testing (for HIV/STIs and pregnancy), and to maintain confidentiality.
The limited discussion of some hormonal contraceptive methods and of condoms and the absence of condom demonstrations to young women or offers of HIV/STI tests are a cause for concern, particularly in a country with a high prevalence of HIV and unwanted pregnancy among young people (31,32). Positive engagement with effective sexual and reproductive health services in early adolescence could help reduce incidence of HIV and adolescent pregnancy (33Á35). However, findings from rural South Africa suggest that healthcare workers may not always agree to see young adolescents alone (although the legal age at which young people can access health services independently is 12 years), or may breach confidentiality to their parents, which should be addressed through training (13).

Strengths and limitations
A limitation of this study related to the definition of the exposure: YFS provision. At six of the seven sampled clinics identified as providing the YFS programme by the DoH, the nurse-in-charge reported that the clinic had not been involved with either NAFCI or YFS; lack of implementation of the YFS programme therefore seems to be a likely explanation for the lack of evidence for an association between the provision of the YFS programme and young people's experiences. However, this does not mean that this programme would prove effective, even with adequate implementation. None of the nurses in charge at the eight clinics that were not identified by the DoH as providing the YFS programme reported that the clinic provided this programme. In the absence of process evaluation data to define YFS implementation, a pragmatic decision was taken to use the available DoH information to define YFS provision. Further research, including process evaluations, could establish whether the lack of observed impact indicates that the YFS programme is an ineffective intervention, or an intervention that has not been well implemented (36). Another limitation was that simulated clients were all the same age, towards the older end of the YFS programme's target age group and no simulated clients lived in households in the poorest household SES quintile. However, ethical approval was not granted to recruit adolescents aged less than 18 years of age. Therefore, these results may not be generalisable to younger adolescents or those living in the poorest households. Other simulated client studies have reported that simulated clients from poorer households were more reluctant to initially visit, or return to health facilities than those from wealthier households due to expectations of and previous experiences of negative treatment [89]. Including the experiences of young men, who are often missed from research on young people's sexual and reproductive health, was a strength of this study, as was the use of the simulated client method to capture the realities of young women's and young men's experiences (37). As participants in a cohort study, these simulated clients have had repeated interactions with research nurses. They may therefore have been more confident when interacting with healthcare workers and have had more positive experiences than other young people. However, a wide range of experiences were described, the characteristics of more and less positive experiences are in line with other studies, and data saturation was reached, lending confidence in these results.
The South African DoH has acknowledged the limited success of the YFS programme (38). With improved implementation, this programme could increase the capacity of healthcare workers to provide positive experiences and high quality services to young people. These findings suggest five key changes that could enhance the implementation of youth-friendly health services and improve young people's experiences of requesting information on condoms and contraceptive methods. These changes could be implemented through a dedicated programme such as the YFS programme, or through broader health systems channels.
1. Training for all healthcare workers should emphasise the need to provide non-judgmental and confidential services for young people. 2. The information given to young people should be comprehensive; young women requesting information on contraceptive methods should be informed of all the available options. Discussion and provision of contraceptive methods should not be limited based on judgmental attitudes about young people's behaviour. 3. Sexual histories should be taken where time allows. 4. Condom demonstrations and HIV/STI tests should be offered to all young people requesting information or other services related to condoms or contraceptive methods. 5. Healthcare workers' performance should be monitored regularly where possible and feedback and additional training provided as required and on request.
These changes could increase young people's utilisation of health services and promote better health outcomes.

Implications and contribution
Few studies have evaluated youth-friendly services from young people's perspectives. YFS provision was not associated with more positive experiences; however, this may reflect limited implementation. Regular training and monitoring could enable healthcare workers to address young people's needs for information and services in a non-judgmental way, and maintaining confidentiality. This may facilitate increases in young people's utilisation of health services and promote better health outcomes.
assistance with the simulated client training, Thokozile Lephoto and Mary Mokhele for their assistance with clinic recruitment, Barbara Monyepote for her help recruiting young people, Dr Suarez-Fuller and Joanna Busza for their comments on the qualitative analysis, and Prof David Ross for his comments on the study design.