Key indicators for appraising adolescent sexual and reproductive health in South Asia: international expert consensus exercise using the Delphi technique

ABSTRACT Background There is a need for an accurate assessment of the patterns and determinants of sexual and reproductive health in South Asia owing to high fertility rates and high incidence of unplanned pregnancy among adolescents. Health indicator sets, with a wide range of health dimensions, also support in formulating evidence-based policies. For attaining this, indicators should be developed and prioritized based on consensus and relevance. Objective This study aimed to develop a comprehensive list of adolescent sexual and reproductive health (ASRH) key indicators for South Asia through systematic participatory expert consultation exercise using the Delphi technique. Methods Experts were invited to two rounds of an indicator rating exercise and a third round to discuss the results in a broader regional perspective. A list of nine indicator categories, including 41 adolescent health indicators, was rated by the expert panel. Prioritization was based on mean Likert scores while consensus was established using Kendall’s W. Results 24, 16 and six experts participated in the first, second and third round, respectively. Out of the nine indicator categories, demographics, reproductive health, violence, and nutrition were ranked high in relevance by the expert panel. Experts had a strong consensus on the relevance of parental control and connection, and behavioral indicators while there was moderate consensus on the relevance of nutrition, infectious disease, and mortality indicators. Conclusion As far as we know, this is the first study that employs the Delphi technique for prioritizing ASRH indicators for South Asia. Engaging a diverse group of experts, using an online platform, we developed a comprehensive list of key indicators for appraising ASRH relevant to South Asia based on expert panel consensus and recommendations. Our results also highlight that there is a need for developing a region-specific prioritized list of indicators which might assist in identifying regional health needs.


Background
During adolescence (10-19 years), rapid changes in social development, health, and wellbeing take place. Due to these reasons, a high priority has been given to the health, social development, and well-being of the adolescent population. The International Conference on Population and Development (1994) called on countries to educate and promote adolescent sexual and reproductive health (ASRH) [1]. Unfortunately, progress has been slow owing to misconceptions, organized community resistance regarding sexuality education, and implementation barriers in many regions of the world.
Globally, the foremost causes of death among adolescent girls are suicide, complications during childbirth and pregnancy [2]. Worldwide, almost 16 million adolescent girls aged between 15 and 19 years and 2.5 million under the age of 16 years give birth annually [3]. Unplanned and early pregnancy not only carries health risks for the young mother and child but also may be detrimental to the social, physiological, and psychological development of young girls [3]. According to the WHO, almost 1 million girls aged < 15 years give birth each year and 3 million girls aged 15-19 undergo unsafe abortions due to unplanned pregnancy [3]. Reports and literature reflect ignorance or destructive cultural norms in some countries: for example, two out of three girls in Low and Middle-Income Countries (LMICs) were unaware of what was happening to them when they first started menstruating [4][5][6][7]; the condom use in young people (15-24 years) at last high-risk sex in the previous year (non-marital, non-cohabiting sexual partner) was less than 50% [8]; and almost 50% of the girls worldwide believe that a husband or partner is justified in hitting or beating his wife or partner in certain circumstances [9]. South Asia consists of eight countries: Afghanistan, Bangladesh, Bhutan, India, Maldives, Nepal, Pakistan, and Sri Lanka. According to the World Bank, South Asia had a population of 1788.38 million in 2019, which is almost 20% of the total world population [2]. In LMICs, adolescents and young adults face many challenges due to poverty, inequality, and marginalization which adversely affect their mental and physical health and overall well-being [10]. In 2017, the annual population growth was 1.2% [2], the fertility rate was 2.4 births per woman while among adolescents (15)(16)(17)(18)(19) years) the fertility rate was 25.6 births per 1000 women in South Asia [2]. Males from 10 to 24 years of age are about 28.7% of the total male population and females of the same age are about 27.7% of the total female population in South Asia [2]. As almost one-third of the population in the region is an adolescent or late adolescent, there is a need to assess their health situation and circumstances affecting their health. Accurately assessing the situation, patterns, and determinants of reproductive health are critical due to the high fertility rates and incidence of unplanned pregnancies among adolescents in the region.
Reliable health indicators are essential for trustworthy and sound information on the health situation, patterns, and trends to help develop appropriate responses at the national, regional, or global level [11][12][13][14]. For assessing the health situation of a region, health indicators play a vital role in providing an overview of key gaps and health disparities [11][12][13][14]. Health indicators and data also guide in determining the priorities for investments in health, measure the health of a population, determine inequalities for different population segments, and to ascertain whether performance expectations are met or not in health [11][12][13][14]. Health indicator sets with a wide range of health dimensions also support in evidence-based policy synthesis [15]. For attaining this, indicators should be developed and prioritized based on consensus and relevance [14,16]. Understanding and priorities vary among stakeholders, experts, and policymakers regarding what connotes a perfect indicator [17,18]. This variation can also be seen among different regions of the world which can be attributed to regional disparities in health literacy, health priorities, health determinants, cultural norms, and demographics [19].
International experts and organizations (e.g. UNICEF and WHO) have developed numerous constructs, definitions, and prioritizations of adolescent health indicators [11][12][13]20]. These efforts have resulted in the development of a comprehensive adolescent health indicator list encompassing multiple health and social developmental aspects [11][12][13]20]. Most of the adolescent health indicator lists available are commonly used for LMICs but are not regionspecific [11, 12,21,22]. However, literature suggests that social determinants and contextual factors strongly affect reproductive health outcomes and service utilization, especially among the adolescent population [23][24][25]. Consequently, adolescent health determinants of social development, health and wellbeing differ widely across different geographical regions. Regional sets of health indicators might be able to provide a better insight into health needs, patterns, perspective, and data gaps regarding specific populations.
There are multiple techniques and methodologies to select indicators. Two of the methods for prioritizing indicators are: 1. Academics and researchers simply choosing indicators, which they give credence to as the most relevant indicators, and 2. Participatory methodologies for identifying and prioritizing indicators [26,27]. Using the second approach increases the chances that the prioritized indicators will be deemed more credible and relevant [28]. For this purpose, we developed a comprehensive list of ASRH key indicators for South Asia through a systematic participatory expert consultation exercise.

Modified Delphi technique
Delphi technique is a method that aids in structuring a group communication process and allows the participants to deal with an intricate problem as a group [29][30][31][32]. Delphi technique has numerous advantages including simplicity of implementation, collecting opinions of a vast array of participants with distinct expertise located in various geographical locations, while ensuring anonymity during the process [29][30][31][32]. For this purpose, a multidisciplinary panel of experts was identified and engaged for prioritizing and selecting ASRH indicators for South Asia. We relied on an online survey with a group of international experts, and for this, the Delphi technique was appropriate as the experts do not have to meet face-to-face during the Delphi process [30]. We conducted three Delphi rounds. The first two rounds provided the experts with the opportunity to rate indicators and were conducted in anonymity. Many modifications to the original Delphi have been used for conducting consensus exercise [31,32]. The third group discussion round was the main modification to the original Delphi technique where we invited the experts to discuss the results in a wider regional perspective.

Literature review and initial list of indicators
The initial list of indicators was developed by a thorough literature search for existing lists of key ASRH indicators. A WHO report of a technical consultation on indicators for adolescent health was determined as the main source of this list. From this report, 27 adolescent health indicators on health determinants, outcomes, and service delivery -grouped within nine categories -were used for rating in the first round [11,12].

Expert inclusion criteria, identification and recruitment
An online Delphi process was developed to engage experts and ascertain their opinions regarding indicators to be prioritized for appraising ASRH specifically for South Asia. Purposive sampling technique was used to identify and recruit experts for the survey. Experts were defined as: (1) Health researchers who published a peerreviewed paper (any author position) on adolescent health, or sexual and reproductive health focused on South Asia. (2) Public health professionals working in program areas (regional/national experts, representatives of technical organizations, NGOs, public sector and health department/health ministry representatives) of adolescent health, sexual and reproductive health in South Asia.
Internet searches were conducted to identify experts according to the inclusion criteria. Email addresses were extracted from lists of authors or organization websites. Snowballing was also used for recruiting experts. For each round, an initial invitation was sent, and in case of no response, two reminders were sent. Experts who were invited for the rounds and did not respond after two reminders were not contacted for subsequent rounds.

First round
The online survey was developed with Limesurvey (URL http://www.limesurvey.org). As explained above, 27 indicators on adolescent health from the WHO report were used for the first round. Experts were asked to rate the indicators on a Likert scale (5 = high relevance, 4 = relevant, 3 = moderate relevance, 2 = low relevance, and 1 = not relevant) based on relevance to ASRH in South Asia. Experts were also given the opportunity to recommend any other indicators that they deemed relevant during the first round. Indicators recommended by the experts were extracted through the online datasheet generated by the LimeSurvey. The list of indicators was compiled and duplicates were removed prior to rating in the second round.

Second round
A list of 14 indicators, recommended by the experts in the first round, were rated in the second round.
Results of the first round were also shared with the experts which included mean Likert scores along with the ranking of the 27 indicators.

Third round
Experts were invited to a group discussion session to discuss the results using the specificity, measurability, achievability, relevance, and targeted (SMART) approach regarding the prioritized indicators. A discussion guide was prepared which included probes on indicator categories, ranking of indicators, survey data availability in South Asia, barriers in data collection on adolescent health indicators, and a detailed discussion on the results of the first and second rounds (Appendix A). Notes were taken during the group discussion sessions and results were compiled post-session. Results were sent to the experts who participated in the group discussion session for any additional feedback. Qualitative content analysis was used for coding and interpreting the qualitative data from the third round and the additional feedback sent by the experts.

Delphi rounds and expert panel
Seventy-six experts were identified as eligible to participate according to the specified criteria. Fifteen email IDs extracted through Internet search were invalid. As a result, 61 invitations were sent for the first round. Twenty-one experts responded for the first round ( Figure 1). Ten additional experts were recommended through snowballing, out of which three experts responded. The participating experts were from Australia, Bangladesh, Brazil, India, Nepal, Pakistan, Switzerland, the UK, and the USA (Figure 2(a)). The experts had diverse professional backgrounds: academic institutes, research centers, United Nations agencies, public sector institutes, and non-governmental organizations (NGOs) (Figure 2(b)). Twenty experts agreed to participate in the second round out of which 16 responses were received. Six experts, of the 10 who agreed to participate in the third round, took part in the group discussion ( Figure 1) . The Delphi was carried out between March and August 2019.

Statistical analysis
The priority of indicators was determined using mean Likert scores whereas consensus among experts was assessed using Kendall's Coefficient of Concordance (Kendall's W) [33]. Kendall's W is a non-parametric statistic used to assess agreement among raters. Its value ranges from 0 to 1 and the values were interpreted using the following cutoffs; 0.9 = unusually strong agreement, 0.7 = strong agreement, 0.5 = moderate agreement, 0.3 = weak agreement, and 0.1 = very weak agreement [33]. Kendall's Tau (Tau-a and Tau-b) and Spearman's Rank Correlation Coefficient were used to assess the strength of the relationship between the first-round indicator ranking and WHO indicator ranking [33,34]. The statistical analysis was conducted using Stata 14 (Stata, College Station, TX).

Categorization and stratification of indicators
Categories developed by the WHO were used to group indicators for further statistical analysis [11,12]. The  categories were further stratified based on whether the indicator was in the WHO list (first round 27 indicators) or recommended by experts (second round 14 indicators) (Table 1). Table 1 shows indicator ranking, the WHO ranking (only for first-round indicators), mean Likert scores, standard error, and confidence intervals. Bootstrapping with 10,000 repetitions was used to generate the 95% confidence intervals. Twelve out of 14 indicators recommended by the experts in the second round were in the top 20 ranked indicators (Table 1). Indicator category ranking, mean scores, and confidence intervals are presented in Figure 3. Demographic and reproductive health indicator categories were rated highest and parental connection/regulation and mortality indicators were ranked lowest among the categories. One possible concern with ranking first and second round indicators is that the number of experts reduced from 24 to 16 in the second round. Therefore, we tested the correlation in the ranking of the first-round indicators between the two groups of experts; 24 experts who participated in the first round and the 16 experts in the second round. The two rankings were compared using Kendall's Tau (Tau-a and Tau-b) and Spearman's Rank Correlation Coefficients (Appendix C). Kendall's Tau-b suggests that the two groups of experts were 73.8% more likely to agree on the ranking of first-round indicators than to disagree (Appendix C).

Need for regional indicator priorities
Using Kendall's Tau-b, we compared the ranking of firstround indicators with the WHO's ranking of indicators and observed that these two rankings were only 27% likely to agree on the importance of the first-round indicators. Similar results were obtained from Kendall's Taua and Spearman's Rank Correlation: 27% and 29%, respectively. Secondly, 12 out of the 14 second round indicators newly proposed by the participating experts were ranked in the top 20 (Table 1).

Consensus among experts
The low estimated values of Kendall's W suggest a very weak agreement among the experts for individual indicator ranking (Table 2). However, the agreement for indicator categories is substantially stronger ( Table 2) Table 2). Additionally, some of these categories included indicators from the first and second rounds (Table 1) which were further explored. Within these subcategories (first and second round indicators), the level of agreement increased even further ( Table 2). Kendall's W could not be calculated for second round mental health, violence and behavioral categories as these only had one indicator.

Consensus versus relevance
Based on our indicator category Kendall's W and mean scores, we observed that demographics, reproductive health, violence, and mental health ranked highest in terms of relevance but there was weak agreement among the experts for these categories (W < 0.5) (Figure 4). For nutrition, behavioral, infectious disease, and mortality indicator categories, the relevance was high to moderate and the experts showed a moderate agreement for these indicator categories ( Figure 4). Experts had a strong agreement over the relatively low relevance of the parental connection/regulation category ( Figure 4). Possible explanations for this variation in agreement among experts might be due to the different number of indicators in each category, as Kendall's W is sensitive to the number of indicators.

Round three group discussion
Six experts took part in the group discussion. The experts agreed that there is a need to develop specific regional priorities for indicators based on relevance, concurrent with our results. The experts also identified a lack of resources and dedicated trained human resources, non-existent data registries, cultural taboos, legal barriers, and lack of political support around sexual and reproductive health as barriers contributing to lack of data availability and health interventions for adolescents. The adolescent population is also not well represented in the current national-level health surveys in the region and efforts should be made to improve their inclusion and participation. Specifically, the experts from Bangladesh, India, and Pakistan highlighted that the early adolescent group (10-14 years) is not covered in nationallevel health surveys, whereas the middle/late adolescent group (15-19 years) is included. Experts also identified child marriage, inadequate access to contraception, and violence as some of the major issues faced especially by female adolescents in South Asia. Coded qualitative data are included as Appendix B.

Discussion
The results of our study allow us to rank a wideranging list of indicators in terms of their relevance for assessing ASRH in South Asia. The list of indicators can be used to appraise the current situation and assess the determinants of sexual and reproductive health. It may also be useful for monitoring and evaluating specific ASRH programs and policies targeted at the adolescent population in South Asia. Delphi rounds and results indicate that the process was successful in developing and promoting consensus on a comprehensive set of prioritized ASRH indicators. For this purpose, a multidisciplinary team of experts constituted the panel, and, through the Delphi rounds, they were able to prioritize a set of 41 indicators for appraising ASRH relevant to South Asia. The Delphi technique aided in systematically developing consensus and prioritization of the indicators. The Delphi process also made it easy to conduct the consensus exercise with an international expert panel using an online survey. Implementing the rounds online increased the ease and efficiency of the process, especially data collection, analysis, and communication across rounds with the expert panel. The experts were able to reach a consensus for multiple indicator categories. The prioritized list of 41 indicators will be able to provide a holistic approach for measuring ASRH in South Asia. The prioritized list constitutes indicators for health outcomes, lifestyle and health behaviors, healthcare services, and demographics which affects the adolescent population's health in the region.
Based on our analysis, the demographics category was the highest-ranked which is in accordance with the literature, as demographics, in terms of age at first birth, literacy rate, and marital status among adolescents, have major implications for overall health outcomes of a population, in particular major adverse consequences on reproductive health outcomes and service utilization [23][24][25]. We established a strong to moderate expert consensus on most of the indicator categories. However, demographics and reproductive health indicator categories, although ranked the highest, had a relatively lower agreement among experts. There may be numerous reasons for the agreement among experts not being stronger; one possible explanation can be that both the categories had a wide range of indicators which could have contributed to the lower consensus. In contrast, the experts agreed on the lower relevance of parental connection/regulation and infectious disease categories in comparison to other categories. Eight indicators in the infectious disease category were related to HIV/AIDS, and experts seem to agree on its relatively lower ranking owing to the low prevalence of the disease in South Asia [35,36].
Understandably, owing to high fertility rates in South Asia [2], the adolescent fertility rate was ranked second among all indicators by the experts. Percentage of adolescents with improved knowledge and attitudes of menstrual hygiene management practices and adolescent girls using hygiene sanitary pads were ranked eighth and eleventh, respectively. There is a culture of silence around the menstrual health of girls due to  [4][5][6][7]37]. Due to this, many girls do not understand their periods and lack awareness regarding menstrual hygiene which further contributes to the non-attendance of girls in schools. In the region, it is estimated that girls are absent for 20% of the school year owing to menstruation. Access to safe sanitary pads is limited for girls and women in the region because of the cost, lack of awareness, and societal stigma [4][5][6][7]37]. The experts ranked sexual abuse and abortion rates among adolescents as first and third, respectively, among all indicators. Almost half of the world's children experience severe violence (physical, sexual, or emotional) and it is estimated that 64% of these children live in South Asia [38]. Almost half of unsafe abortions in the LMICs are occurring in Asia with sex-selective abortions and abortions due to unplanned pregnancy being quite common in South Asia [3,39,40]. Unfortunately, evidencebased information remains lacking on these issues, and, thus, obtaining reliable data remains a challenge. The WHO recommends that every child and adolescent have the right to comprehensive sexuality education (CSE), which includes age-appropriate information on various health topics including violence, sexual abuse, and planned parenthood [41,42]. Owing to community resistance, gaps in domestic legislation, non-existent national plans of action, inadequate law enforcement, and non-operative child protection systems in the region, adolescents have limited or no access to CSE and reproductive health services. This hampers the chances for substantial improvements in these aspects in the region [43][44][45][46].
As is frequently the case with the Delphi technique, the participation decreased for subsequent rounds which can be seen as one of the limitations of the study. Although eight to twelve participants are considered as an acceptable minimum for the Delphi technique, we tried to recruit as many experts who satisfied the inclusion criteria as possible to address the decrease in response rate for the subsequent rounds [29][30][31][32]. This also provided us with a wideranging panel of topic experts for the consensus  exercise. The Delphi technique also allowed the experts to iterate their evaluation of the relevance of indicators anonymously limiting bias and peer influence. The Delphi technique relies on group consensus for decision among a group of experts. Prioritization of indicators was based on mean Likert score, but these scores might be influenced by individual expertise of the participating expert and can introduce bias based on individual preferences. Hence, consensus formation among the group of experts is an integral part of the Delphi process. For individual indicators, we found low consensus among the experts, this was per our expectations since the indicators deal with a wide variety of health issues and the experts were not expected to agree on the importance and relevance of all these indicators. For indicator categories, experts had strong to moderate consensus for most of the categories. Incorporating the group discussion session also allowed us to reflect the results from a wider regional perspective. As it was a qualitative round, the somewhat lower participation was not seen as a major limitation, and the session had no implications on the indicator ranking or consensus development among the expert panel.
However, the discussion session was highly informative. It emphasized that lack of resources, shortage of dedicated skilled human resources, non-existent data registries, and cultural restrictions affecting sexual and reproductive health significantly can contribute to a lack of data on adolescent health indicators. Noninclusion of the early adolescent group and unmarried females adds to the unavailability of indicator data in national representative surveys. These barriers and limitations should be further investigated to explore possible potential enablers and explanations to improve data availability on adolescent health indicators.
When comparing the WHO adolescent health indicator ranking to the first-round indicator ranking, we found a low correlation between the two rankings. Twelve out of the 14 second round indicators, recommended by the expert panel, were also ranked in the top 20. Even though the WHO's ranking was developed for adolescent health and we focus specifically on ASRH, low correlation of the rankings and the additional second-round indicators with high mean Likert scores support the need for regional prioritization of indicators concurrent with the WHO and expert recommendations [11,12]. DHS and MICS are currently in use a universal survey tool for collecting health indicator data across the LMICs [21,22]. These tools allow monitoring, appraisal, and comparison of health indicators across countries and regions. Although the regional indicator priorities might provide rich information on the health patterns and trends, they pose a challenge for comparison across different countries and regions at the same time. This is one of the first studies to employ the Delphi technique for prioritizing ASRH indicators for South Asia. Using the online platform, engaging a diverse group of experts, and developing a comprehensive list of key indicators for appraising ASRH, are some of the major strengths of the study. Also, the Delphi rounds were systematic and well-structured which complements the validity of the findings of this study. Further studies are required to explore whether tailored region-specific surveys can provide better insights on population health, for example, regarding their potential policy impact.

Conclusion
This study explored the opinions of topic experts from diverse backgrounds on the relevance of key indicators for appraising ASRH in South Asia. Using the Delphi method, we determined a prioritized list of ASRH indicators, based on expert panel consensus and relevance, for South Asia. Our statistical analysis of the first and second round highlighted the need for developing region-specific indicator prioritization. Additionally, the discussion round highlighted the barriers and limitations to improved data availability on adolescent health indicators in South Asia.

Author contributions
HZ and TB supervised FA in developing and performing the Delphi study process. FA developed the questionnaires, collected the data, FA and GA analyzed the data. FA wrote the first draft. GA, TB and HZ contributed in reviewing and preparing the final manuscript.

Disclosure statement
Authors have no conflicts of interest to disclose, whether perceived or real.

Ethics and consent
The content and implementation of the survey complied with the overall ethical guidelines of the Leibniz Institute for Prevention Research and Epidemiology (BIPS). All participants were informed that the survey was voluntary, anonymous, and that they could stop at any stage/round of the survey. Participants were also required to electronically confirm, before participating in the study, that their participation was voluntary and their consent was recorded.
Information sheets were sent to participants with the invitation email which included information regarding the survey and number of rounds. Participants were also asked if they would like to be contacted for the consecutive round of the survey or not. No personal data was collected other than email IDs to contact participants for subsequent rounds. However, email addresses were not linked to participants' responses instead they were allotted a participant ID.

Funding information
This work was supported by the Department of Prevention and Evaluation at the Leibniz Institute for Prevention Research and Epidemiology-BIPS and German Academic Exchange Service (DAAD) as part of their Research grant Doctoral program. The research team at BIPS provided support in study design, data collection, analysis, and in the decision to submit the article for publication. DAAD had no involvement in; study design, in the collection of data, analysis and interpretation of data, and in the decision to submit the article for publication.

Paper context
Comprehensive sets of health indicators support in formulating evidence-based policies. Indicators should be developed and prioritized based on consensus and regional relevance. This is one of the first studies to employ the Delphi technique for prioritizing adolescent sexual and reproductive health (ASRH) indicators for South Asia. Using the online platform, engaging a diverse group of experts, we developed a list of key indicators for appraising ASRH for South Asia. • Recommendations on developing indicators and regional priorities.
• Final remarks and concluding the session (5 minutes) The discussion will be carried out on 9 categories using the SMART approach.