Developing the Global Health Cost Consortium Unit Cost Study Repository for HIV and TB: methodology and lessons learned

Consistently defined, accurate, and easily accessible cost data are a valuable resource to inform efficiency analyses, budget preparation, and sustainability planning in global health. The Global Health Cost Consortium (GHCC) designed the Unit Cost Study Repository (UCSR) to be a resource for standardised HIV and TB intervention cost data displayed by key characteristics such as intervention type, country, and target population. To develop the UCSR, the GHCC defined a typology of interventions for each disease; aligned interventions according to the standardised principles, methods, and cost and activity categories from the GHCC Reference Case for Estimating the Costs of Global Health Services and Interventions; completed a systematic literature review; conducted extensive data extraction; performed quality assurance; grappled with complex methodological issues such as the proper approach to the inflation and conversion of costs; developed and implemented a study quality rating system; and designed a web-based user interface that flexibly displays large amounts of data in a user-friendly way. Key lessons learned from the extraction process include the importance of assessing the multiple uses of extracted data; the critical role of standardising definitions (particularly units of measurement); using appropriate classifications of interventions and components of costs; the efficiency derived from programming data checks; and the necessity of extraction quality monitoring by senior analysts. For the web interface, lessons were: understanding the target audiences, including consulting them regarding critical characteristics; designing the display of data in “levels”; and incorporating alert and unique trait descriptions to further clarify differences in the data.


Introduction
A critical gap in the arsenal needed for planning HIV and tuberculosis (TB) programmes is a centralised source of standardised intervention cost data that is easily accessible to policy analysts, country officials, and implementing organisations planning programmes in low-and middleincome countries. These data will support accurate planning, efficient implementation, and allocative efficiency. At the international level there are significant funding shortfalls to meet projected needs in both HIV and TB (Global Fund, 2019;UNAIDS, 2016), while at the national level countries are contending with low levels of domestic resources, epidemics that are re-emerging (UNAIDS, 2018;WHO, 2018), striving to meet expectations regarding universal health coverage, and transitioning away from multilateral aid (Chemouni, 2018;El-Sayed, Vail, & Kruk, 2018;Kharas & Noe, 2018;Prizzon, Rogerson, & Jalles d'Orey, 2017). However, there is a scarcity of standardised and easily accessible cost data needed to support the costing of national strategies and investment cases (Anderson, Maliqi, Studies, in both the published and grey literature, report information inconsistently, making comparison and utilisation of the data difficult. Such inconsistencies include a lack of intervention typologies, as well as differences in defining outcome variables (i.e. the denominator in the unit cost), time horizons, and measurement of costs. The Global Health Cost Consortium (GHCC) was formed in 2016 to support achieving greater value for money of investments, through improving the availability, quality, timeliness and policy-relevance of HIV and TB cost information in low-and middle-income countries. As part of this support, in 2017 the GHCC launched the Reference Case for Estimating the Cost of Global Health Services and Interventions (Vassall et al., 2017;hereafter "GHCC Reference Case") to fill gaps in the availability and standardisation of costing guidelines applicable to multiple health areas, and to assist both producers and users of cost data to better conduct and interpret cost studies.
In addition, the GHCC sought to build on earlier endeavours to address gaps in the availability and standardisation of published cost data. Between 2010-2013, these efforts for HIV resulted in the Avenir Health Unit Cost Repository (UCR) and the London School of Hygiene and Tropical Medicine (LSHTM) database described in the paper entitled Costs to Health Services and the Patient of Treating Tuberculosis: A Systematic Literature Review (Avenir Health, 2013;Laurence, Griffiths & Vassall, 2015). The GHCC performed an update of the included literature, standardised the categorisation of intervention and cost data, expanded the repository to include all HIV and TB interventions with available cost data, and redesigned the web-interface for additional functionality and adaptability to mobile platforms. Note that the development of the UCSR both informed and was informed by the GHCC Reference Case, which was being developed concurrently. 1 Given the paucity of information on best practices for developing such repositories from published literature, particularly regarding methods for standardisation, extraction, data transformation, and web-interface design (Lund et al., 2011;Neumann et al., 2016;Visscher et al., 2017), the development of the GHCC Unit Cost Study Repository (UCSR) is documented here so that users understand its strengths and limitations, and to assist those undertaking similar repository efforts that may be more localised or may be targeting other health areas.

Systematic search and review
HIV systematic search and review GHCC collaborated with partners to identify 54 HIV interventions covering prevention, treatment and care, testing, enablers, and health systems. Using economic (e.g. "cost", "care cost") and disease-specific (e.g. "HIV", "human immunodeficiency virus") search terms, we conducted a systematic review of literature published without language restrictions between January 2006 and October 2017 in six databases: PubMed, Embase, Web of Science, The Cochrane Library, The NHS Economic Evaluations Database, and Literatura Latinoamericana en Ciencias de la Salud (LILACS). There were no restrictions on the types of treatment or interventions, as the goal was to capture as many relevant studies as possible. An initial search focused on recent HIV intervention implementation was done for articles published from January 1, 2006 through October 2, 2016. A follow-up search with the same databases was completed on October 20, 2017. We also used existing cost study databases from systematic reviews previously conducted by Avenir Health and LSHTM (Avenir Health, 2013;Laurence et al., 2015), which expanded the set of identified HIV studies to include those published since 1993. A follow-up search of grey literature and a focused Google search was completed on July 31, 2018.
Results from the systematic search were stored in an EndNote Library and merged with article lists previously obtained by Avenir Health and LSHTM, along with literature obtained from snowball searches. A team of four researchers screened the results, based on title and abstract, and senior researchers completed random checks of the excluded studies to identify potentially missed studies. The team excluded unique records if they were not from a low-or middle-income country, had no empirically collected data, were a commentary or letter to the editor, if the full-text of the record could not be found, or if the currency or perspective of the costing reported (from the article or after contacting authors) could not be ascertained in the full-text. All articles and reports were screened a second time during the extraction process to ensure that they contained the required empirical cost data. Ultimately, 175 peer-reviewed articles were selected and extracted from the initial 11 717 unique records in the search (Figure 1).
The grey literature search focused on relevant websites and literature referenced in published studies screened by the GHCC, while the Google searches focused on identifying specific interventions by region. After excluding duplicates and studies not meeting inclusion criteria, the grey literature search resulted in 38 reports. Focused Google searches were completed between November 2017 and July 2018 with the following search string format: "'[intervention name]' costs [one of: Africa, Asia, East Europe]-US". 2 For example, a search string for female condom provision would be: "'female condom provision' costs Africa-US". The first 20 results for each region were reviewed by title and abstract, resulting in another four unique reports for inclusion. Of 2 399 unique records screened, 42 grey literature reports were selected. In all, 1 344 unique unit cost estimates were obtained from 217 selected HIV studies. 3

TB systematic search and review
For TB, six health and economic databases were searched for studies published between January 1990 and July 28, 2016 without language restriction: PubMed, EMBASE, Econlit, The National Health Service Economic Evaluation Database, The Cost-effectiveness Analysis Registry, and The Cochrane Library. Two additional databases, Web of Science and Literatura Latinoamericana en Ciencias de la Salud (LILACS), were searched in February and March 2017. Broad searches were designed with search terms including economic (e.g. "cost", "economic", or "finance"), disease (e.g. "TB", "tuberculosis", "MDR", "XDR") and intervention-specific keywords (e.g. "treatment", "DOTS", "isoniazid preventive therapy", "patient cost").
The search results from each database were downloaded to Endnote into a single library. The screening of these results, using the same exclusion criteria as for HIV, was then undertaken in three stages: by title, by abstract, and by full text. First, one person reviewed all records by title, while a second person checked the records that were excluded based on title screening. Second, the resulting abstracts were screened by two analysts independently and were excluded if no costs or cost sourcing were reported. The remaining records were then subject to a full-text review by two analysts working independently and assessed for eligibility. Records were also cross-checked against a recent systematic literature review of tuberculosis costs for health services and patients (Laurence et al., 2015). Of 15 161 unique records, 170 were included ( Figure 2).
A search of the grey literature on TB was also conducted, aligned with the grey literature searches by the HIV team described above, through four principal databases: The European Association for Grey Literature Exploitation (EAGLE), The System for Information on Grey Literature in Europe (SIGLE), and the World Bank and WHO websites (documents and meeting reports). Google searches were also conducted, in which the first 50 documents that resulted from the algorithm used in Google for key websites, such as msf.org, who.int, unaids.org, and pepfar.gov, were reviewed. Although the OneHealth Tool, Resource Needs Model, Global Price Reporting Mechanism, Unit Cost Repository, and Unit Cost Estimations WHO-CHOICE tool were also searched, they did not contain unique cost information. The same screening process was followed as for the peer-reviewed literature, resulting in the inclusion of 31 TB grey literature studies from an original field of 398 unique records.

Development of the extraction form
To develop the early version of the extraction form, an extraction working group was created within the GHCC team. This subgroup looked at both the extraction form that underpinned the former Avenir Health Unit Cost Repository, and the extensive Principles and Methods Reporting Checklist from the GHCC Reference Case to identify the fields needed for the UCSR. The extraction form was designed to serve multiple purposes: (a) designate and populate key fields to be displayed in the UCSR, (b) support a quality rating index and summary ratings to be displayed in the UCSR, and (c) facilitate analysis through including analysis-relevant fields not necessary for the UCSR display. It was recognised at the outset that standardisation of key intervention and cost characteristics was critical to presenting a reasonable number of filters (and filter drop-down options) in the UCSR, comparing data accurately, and avoiding errors that would impact programming (e.g. using both "Republic of Tanzania" and "Tanzania" for a country name). Achieving consensus across the GHCC team, advisors, stakeholders, and partners for filters and standardised options was a time-consuming process and continued even after extraction began. For example, drawing upon previous experience with the Avenir Health Unit Cost Repository and discussions with GHCC advisors and stakeholders, it was planned early in the process that the UCSR would be structured around disease-specific interventions, and that this list of interventions would need to align with how global partner organisations categorised interventions in their reporting and budgeting structures. However, each partner organisation did so differently, and therefore extensive research into these categorisations and  discussions with partner organisations were necessary to develop an "Intervention Typology" 4 to frame the UCSR. The categories ultimately adopted were: Prevention; Case Detection, Testing, and Diagnosis; Treatment and Care; Enablers and Support; and Health System. A similar process was followed for the identification of other key extraction fields to be used as filters in the UCSR (e.g. target populations, service delivery platforms, ownership, technology, input cost categories, activity cost categories) as well as standardised drop-down options. Voluntary medical male circumcision (VMMC) and antiretroviral treatment (ART) interventions were used to pilot the extraction form and identify areas in need of standardisation.
Several fields were standardised to better support the UCSR display and analysis. For example, author-reported country names were standardised into ISO-3 codes to ensure correct spelling. Additionally, the open-text field 'population' was further divided and standardised into 'target group -demographic' and 'target group -clinical' to more clearly differentiate population groups across studies. The list of options for these fields was identified in a post-hoc analysis of author-reported information. Finally, the 'intervention details' and 'technology' fields were separated from a free-text field to allow for searching capabilities on intervention characteristics in the UCSR. Some fields were dropped entirely. After lengthy discussion within the team, the 'integrated services' and 'full/incremental cost' fields were dropped due to the impossibility of specifying consistent operational definitions. Similarly, the field "traded vs non-traded" was dropped because the field was often noted as "mixed," which stripped it of analytical value.

Data extraction and cleaning process
Extracted cost data were recorded on two tabs of the extraction template Excel workbook: a study attributes tab, and a cost data tab. The study attributes tab records all study-level variables across columns (e.g. authorship, type of intervention, implementation of the intervention, costing methods, etc.), with each row representing a unique unit cost reported within a given study. Unit costs were determined to be unique and recorded in an individual row in the extraction form if reported in a study for different countries, target groups (demographic or clinical), service delivery platforms, ownership (e.g. public, private), urbanicity (e.g. rural, urban), sites, or technologies. The cost data tab in the extraction template records each cost type (total unit cost, input or cost category, or input cost) associated with a given row of the study attributes tab. If a row in the study attributes tab describes the unit cost of a male circumcision in the public sector, at a secondary-level hospital, using surgical technology, this row could then be associated with more than one cost data row if disaggregated cost data are available. Thus, in addition to the total unit cost per person served for VMMC, there might also be separate rows for personnel cost, cost for recurrent goods, etc.
All of the HIV articles and reports were divided among three extractors. The lead extractor managed version control across all interventions. For HIV, a senior team member verified or corrected all fields displayed in the UCSR. For TB, double extraction was performed by two teams of two extractors each. If there were discrepancies, these were first discussed among the two data extractors and verified against the study. If the data extractors could not agree, then a senior member of the team was consulted for resolution.

Standardising the data to a common currency and year (2017 USD) and format
Costs were standardised into 2017 United States Dollars (USD). All costs reported in currencies other than USD were converted to USD based on the year of reporting using market exchange rates published by the World Bank. 5 Once all costs were converted to USD, costs were inflated using the US GDP price deflator, as needed, also from the World Bank. For those studies that did not report the year of the cost data represented in the study, the "publication year minus-one year" was used before exchanging to USD (if applicable) and then inflating the reported costs to 2017 USD. Alternative methods of these transformations were explored including using local inflation rates for some cost inputs before converting to USD (Kumaranayake, 2000) and are explained further in Annex A. Although, the method utilised by the GHCC (converting first to USD then inflating to 2017 USD) risks masking important within-country variation in inflation over time for different categories of cost inputs in different contexts (Whiteside & Zebryk, 2017), we found that the reporting of cost data from original published articles was too heterogeneous in nature and that too many assumptions were necessary to justify using different inflation procedures for different cost inputs when not all inputs were uniformly available across articles.
Data were combined into a single "wide file" 6 for subsequent data analysis and exported to the UCSR. Data from both the study attributes and cost data tabs were imported into Stata in two separate Stata .dta files. Data from the cost data file were then merged field by field into the study attributes file so that each row represented a unit cost. Each row included the total unit cost, as well as the disaggregated input costs by a broad cost category (e.g. capital, personnel, traded goods, etc.) and a narrow cost category (e.g. personnel: direct service delivery; personnel: support staff) (Tables 1 and 2).

Quality assurance
Validation of data extraction, transformations and programming Quality assurance checks were performed during the workflow in five principal stages. In Stage 1, the completed extractions were reviewed for formatting errors, missing values, and other issues. A summary report for each intervention was then generated highlighting fields needing review by the team for standardisation (e.g. population and technology fields, and preferred output unit costs). These summaries were reviewed to achieve consensus on standardised options, after which the lead extractor updated the corresponding fields in the extraction.
In Stage 2, the vetted interventions were sent out for data validation checks in Stata 15 (described further below), which were developed jointly between Avenir Health and the data team after it was recognised that the files being sent for programming to the UCSR website contained numerous problems. These checks took about one day to run and flagged inconsistencies, misalignment, and errors in cost totals. Among numerous data checks, several that should be highlighted are: (a) checking to ensure that costs summed correctly; (b) reviewing any studies that omitted "critical costs"; (c) aligning the costed activities and intervention details fields to identify discrepancies (e.g. intervention elements that were included in the intervention but not costed, or vice versa); and (d) checking variable labels and names to ensure that they were standard across interventions, and not duplicative (e.g. HIV+ and HIV positive). Once this process was complete, flagged errors and a PDF summary showing the range of data/labels for each variable were sent back to extractors.
Two additional fields for programming into the UCSR emerged from the Stage 2 review in (a) and (b) above. Although slight deviations between the total unit cost and the sum of the inputs were expected due to rounding errors and variations in how authors reported costs, or modifications from exchange rate changes, the consensus in the GHCC team was that a discrepancy of more than $1 USD (remaining after review) should be flagged. It was also recognised that incomplete unit costs could be misleading to UCSR users. Thus, the GHCC determined, through analysis of the literature and of primary data sets, that key personnel, commodities, or services were "critical" cost inputs, and that where the author had reported omitting these from their reported total unit cost, those total unit costs should be flagged. Therefore, columns called "disaggregated costs flag" and "omitted costs" were created in the wide file, which feed into the alerts field of the UCSR display.
In Stage 3, the extractors split up checking the extraction flags based on their respective familiarity with a given intervention. Depending on the number of flags, the clean-up process then took approximately one day. The flags were then reviewed and documented in a separate file. The cleaned extraction form was sent back for a final check prior to processing for the UCSR. In a parallel process, the cleaned interventions from cleaned intervention extractions underwent quality assurance (QA) by a senior researcher. This was an iterative process that took approximately 1.25 hours per study, in parallel to the other stages in the process. When completed, the data in the wide file was then updated.
In Stage 4, the data validation checks were then re-run, and any remaining flags were addressed by the extractors. Once cleaning was complete, the extraction file was transformed into four wide files (provider and patient cost files separately for HIV and TB) for use in the UCSR. In addition, intervention-specific wide files were sent to the GHCC analytics team for use in various analyses.
Finally, in Stage 5, the wide files were sent to Avenir Health for review and programmed into the UCSR using JavaScript and Highcharts. Once the system of Stata checks was in place, Avenir Health's data check became more focused on standardising between the TB and HIV wide files and probing where new fields should be added to support better user understanding of the cost. For example, fields on alerts (e.g. omission of key costs), number of sites included in the unit cost, and unique trait (i.e. why it is a unique cost within a study) were all added. Another review was done by Avenir Health after each iteration of the wide file was programmed into the UCSR to identify remaining issues and work with the programming and data teams to track the error source to either the UCSR programming, STATA coding for the wide file, or to the extraction itself, thereby circling back to stages 3, 4 or 5 again.

Development of the study quality rating index
The multi-component quality rating (QR) system was developed to provide the GHCC user with a nuanced quality assessment for HIV and TB intervention costing studies. 7 This will help the user detect potential inaccuracies and assess his/her confidence in the displayed unit costs. Sixteen QR items were identified across four summary quality indicators -1: Bias low (L; i.e. underestimation); 2: Bias high (H; omitting or failing to appropriately account for key cost components resulting in overestimation); 3: Precision (P; the precision of the displayed unit cost); and 4: Reporting standards (R; the extent to which the displayed unit cost conforms to the GHCC Reference Case reporting standards). For each identified QR item, 1-3 points are deducted depending on the field and the perceived impact of the magnitude of the problem, based on expert opinion within GHCC (Table 3).
The final quality rating for each item is presented as a letter score A-D with A representing the highest quartile of scores and B-D representing the three successively lower quartiles. All items used were aligned with the definitions and topics outlined in the GHCC Reference Case. The system was validated against 14 published and commonly cited quality assessment tools, of which two are scoring systems for costing studies (Beck, Harling, Gerbase, & DeLay, 2010;DeCormier Plosky & Bollinger, 2012), and 12 are cost-effectiveness analysis (CEA) checklists (    • Maximum deductions of 3. • Deductions = (number of cells where geographical area, site, patient = "NR") * 3/(3-[number of cells where geographical area, site, patient = "N/A"]) A deduction of 1 point is taken if the number of sites is 3-9. A deduction of 1-3 points is taken if sampling method for geographical area sampling, site sampling, and/or patient sampling is listed as "convenience" or "not reported". Please note that only the types of sampling applicable to the study design are included. For example, if geographical area sampling, site sampling, and patient sampling are listed as "N/A", "NR", "NR", 3 points are deducted. If "N/A", "N/A", "NR", 3 points are deducted. If "N/A", "NR", "reported -not convenience", 1.5 points are deducted. If "N/A","N/A", "reported -not convenience", then no points are deducted.  Sanders et al., 2016;Ungar & Santos, 2003;Weinstein, Siegel, Gold, Kamlet, & Russell, 1996). The two scoring systems for costing studies compare with the GHCC QR system as follows: a) many items are similar, b) scoring rules for individual items differ in minor ways, and c) the Beck and Avenir Health systems provide single summary scores by summing numerical responses, while the GHCC system provides four ordinal ratings representing low and high biases, precision, and reporting.

Designing the UCSR web interface
The conceptual design of the UCSR web interface began soon following the GHCC project initiation, as it needed to align with principles and methods in the GHCC Reference Case, and because development of the extraction form needed to reflect the structure of the UCSR. The large and complex dataset underpinning the UCSR had to be easy to navigate by both costing experts and novices, provide the most important information needed to interpret varying cost estimates for the same intervention, follow standardised categorisations consistent with the monitoring and budgeting structures of GHCC partners to be useful for different costing purposes, and function smoothly on both desktop and mobile devices that may be utilising low-speed connections.
Each element of the current UCSR design expressly reflects the lessons learned from: the previous Avenir Health Unit Cost Repository; the consensus-driven alignment and standisation processes to operationalise the GHCC Reference Case, intervention typology, and data extraction forms; comparisons of visualisation software platforms for cost and user training time; input from the GHCC team, GHCC stakeholders and advisors; numerous rounds of beta-testing (at the international level, country level, and within the Avenir Health programming team); a survey posted to the UCSR website and GHCC Newsletter; oral and written feedback after presentations at international/regional conferences and workshops; and a website design firm and a data visualisation specialist (Table 4).

UCSR structure
The UCSR 8 collates all published and grey literature cost estimates for HIV and TB interventions into one easily accessible online platform. The cost estimates are standardised in terms of output units (e.g. cost per person served, cost per visit), intervention implementation (e.g. service delivery platforms, ownership, target populations, technologies), disaggregated cost categories (e.g. personnel, capital costs), currency and year and costing perspectives. The final design requires the user to either utilise the Export Data feature to download the results for conducting their own external analysis and/or visualisations, or to get started on the Filter (Select) page by selecting one intervention that is defined further by disease and intervention class. This page allows for further filtering by geographic location (region/country, urbanicity) and target populations (demographic and clinical). The Filter (Refine) page provides additional filters for intervention implementation (platform, ownership, technology) and costing methodology (cost perspective, whether the cost is economic or financial, year of cost data collection, whether scale is discussed in the study, whether sensitivity analysis is performed in the study).
The order of the filters reflects stakeholder preferences on the most important fields for the interpretation and utilisation of cost data and the order of principles in the GHCC Reference Case. Stakeholder feedback, through beta-testing multiple versions of the UCSR, provided critical input into the number of filters in the Filters pages to enhance and simplify the user experience. Also, limiting the choice on the Filter (Select) page to one intervention was intentional, to prevent overwhelming the user with an unmanageable number of filters, search results, and visualisation possibilities.
Once the intervention is selected on the Filter (Select) page, the user clicks "apply" to see results on the results page. To simplify the presentation of information for the user, a two-level structure for display of results was developed. The first level is a simple table display listing the study (lead author/year of publication), unit cost in 2017 USD, unit, alerts, unique trait, number of sites, perspective, country, technology detail, treatment phase (for treatment interventions only), demographic target group, clinical target group, platform detail, ownership, urbanicity, and quality rating. At this first level the user can also scroll to the right to view table columns not immediately visible on smaller screens, and filter further by a keyword or phrase in the search field (e.g. for "Xpert" within TB interventions).
To access the second level of results, the user clicks on any unique unit cost row in the first level to see further information about that cost. This second level includes three main sections: (1) study attributes (citation, geography, intervention description, timing and coverage, population detail, study design, and costing methods); (2) cost disaggregation (input categories including personnel, recurrent, etc., activity categories, and patient cost categories); and (3) alerts (omitted costs, GHCC calculations, and whether disaggregated costs sum to total unit cost).
On the data visualizations page, UCSR users may create data visualisations by comparing results either by country or by a specific characteristic (e.g. platform, ownership). Note that the UCSR does not calculate an average cost or state which cost is preferable; rather the two-level display of results (e.g. unique trait, technology detail, cost disaggregation) and the visualisations respectively show the heterogeneities in, and the range of, costs for each country (including the median), using maps, bar charts, and box plots. Disaggregated costs can be displayed using either bar or pie charts by clicking on a unique cost estimate in the initial bar chart.
Averages are not calculated in the UCSR for various reasons: a) with the exception of standardising nomenclature and inflation/currency conversion to a common year and currency, the GHCC is a repository, representing the study data as reported; b) the average could be overly influenced by one study that has many estimates (such as for each facility studied); and c) a second tool, to be developed, was expected to calculate an "average" point estimate for the cost of each intervention in each country, on the basis of To understand what level of detail will be necessary for data extraction (e.g. input cost categories, activity cost categories, specific input costs, costing methods) and what features will need to be built into a web repository (if put online). Discuss what the data will be oriented around (i.e. the key or selection characteristic) To align with budgeting, planning, grant application, and reporting frameworks, and/or inter-agency typologies; Design of web repository begins with key selection characteristic/filter. Agree on a typology for that selection characteristic (e.g. if intervention, how to categorize interventions); Define what typically comprises each type of that characteristic (e.g. if HIV intervention, what is standard for a VMMC or ART intervention) To align with budgeting, planning, grant application, and reporting frameworks, and/or typologies used across organizations; Engender comparability of cost data.
Agree to additional key characteristics to extract/ display; Determine what data users would like to see "first" in a spreadsheet or web-interface (i.e. how to "layer" the data) To align with budgeting, planning, grant application, and reporting frameworks, and/or typologies used across organizations; Align with stakeholder needs for analysis and presentation of data; Display a limited number of characteristics in a user-friendly way. Identify the software, hardware, platforms, and types of internet connection used by stakeholders To determine the formats needed for download options (e.g. Excel, CSV, PowerPoint, JPG), the level of intricacy of data visualizations, the "user journey" (i.e. how a user scrolls and clicks through the website) on both desktop and mobile devices, and how large files (including photos, data) can be while meeting processing needs on low-bandwidth connections. Create processes for communication within/among extraction and design teams and for quality control

Regular communication meetings within and among teams
The extraction can be subjective, and subject to more heterogeneous interpretation with more extractors. Therefore, regular communication is necessary for consistency. Create documents to track the stage of review (included and excluded studies), extraction, quality control for each study (and for decision points that were made; Post these documents in a place (Dropbox, Smartsheet, Google Drive) accessible to all teams This allows the team/s to be on the same page and prevents different extractors from doing [unplanned] duplicate extractions; Users of the study repository and study authors will want to know which studies are included, and the reasoning behind those excluded.
Double-extract or engage a senior staff member to review a sub-set of extractions; Automate data checks where possible To assist with standardizing the extraction, to catch any extraction errors, and assess if extraction makes sense (e.g. average age entered in one column matches target population in another). The person engaging with the web design team should be familiar with costing, and be able to both inform the design of the extraction template and review the collation of extracted data before uploading to the web To align with needs of stakeholders and with costing guidance such as the GHCC Reference Case; To interpret the extraction and act as a check that the data is correct. *Note: The design team will need a collated extraction format that is easy to upload (e.g. Excel); going through Stata requires staff and programming. Define and standardise extraction fields Define each characteristic in the extraction To facilitate extractor interpretation; Provide clarity for users. Use drop-down lists where possible To facilitate extraction; Avoid misspelling and proliferation of terms (e.g. Taznania/Tanzania/Rep. of Tanzania) for analysis. Clarify differences in the data Use key characteristics; Identify what is unique about costs if not a key characteristic Users will want to be able to quickly understand why two or more cost estimates for the same intervention are different. Note aberrations in the data (e.g. reported medians vs means; omitted unit cost inputs) To facilitate comparability; Relevant data (e.g. personnel costs reported as $0) may not be in the first few columns of the extraction or shown on the first page of web results. data drawn from extensive analysis of literature and primary data. While there is a filter for scale in the UCSR, so the user can identify the rare cases where an article did discuss or analyse scale, there are no calculations by the GHCC for scale in the UCSR. However, the GHCC data team has been looking into the issue of scale and presented results in a session entitled Moving Away from the Unit Cost: Estimating Country-Specific VMMC Average Cost per Service Curves Accounting for Variations in Implementing Platforms at the International Health Economics Association World Congress (Bautista-Arredondo et al., 2019).

Conclusion
The UCSR provides the only standardised and thoroughly validated repository of unit cost information from published HIV and TB literature. Documentation of the process and lessons learned from this endeavour may assist not only with the further refinement of the UCSR, but also with other cost data collection, collation, analysis, presentation, and utilisation efforts. One key lesson is that the extraction form and process have several advantages and disadvantages. The extraction template is comprehensive and flexible, while offering opportunities for standardising author-reported content. However, there are several trade-offs when using such a comprehensive extraction template. First, the time required for each extraction is substantial, and there are more fields that need to be monitored for accuracy. Second, with multiple extractors, consistency across extractors required constant communication and frequent revisions of the template before reaching a stable form that minimised possible extraction differences. Since many of the fields in the UCSR are subjective, there were often differences among extractors in how data were extracted, even with extensive training of extractors and data quality checks. Table 4 provides a circumscribed set of recommendations for creating cost repositories, and thus directs practitioners toward focusing on: a) knowing their audience (e.g. what typologies/indicators they use for planning or budgeting, if they will want to conduct analysis with the data and what level of detail they will need, what types of platforms/ devices they access data on, what formats they will need data to be downloadable in for analysis and presentation), b) developing a process at the outset to ensure clear communication both within and across data extraction and UCSR design teams, c) clearly defining and standardising each field used in the extraction, if possible in alignment with the GHCC Reference Case, and with particular attention given to the output units (unit cost denominators); in addition to utilising drop-down lists wherever possible in the extraction form and d) incorporating extraction fields and web-design features to clarify differences in the data.
The process of standardising fields did often require ad-hoc review of author-reported content by intervention, and ongoing review of each intervention for themes and commonalities. As a result, version control of the extraction form became a priority as older interventions that were extracted prior to a given change needed to be updated. Accordingly, the GHCC created a system to contact the lead author of each study in the UCSR, ask them to review and validate the information presented, and then make edits to the extraction according to the author feedback. Note that the availability of "as reported" fields, in addition to GHCC standardised fields, was advantageous during the validation and data checking processes so that extracted costs in the original currency and cost categories could be checked back against the study.
Several next steps can be recommended for future developments. First, due to the complexity of the existing extraction and validation process, the GHCC has explored systems for direct web-based data entry consisting of two activities: first, uploading a form to be completed by study author, and second, a quality review of entries by a senior GHCC staff member. This may result in a more sustainable approach for the UCSR and other cost repositories. Second, the GHCC has provided technical assistance, the GHCC Reference Case (inclusive of the checklist for how to transparently report cost estimates and key indicators critical to interpretation of cost estimates), and extraction and web-design templates to partners working to create cost repositories in immunisation, malaria, social and behavioural change (SBC), water, sanitation and hygiene (WASH), and country-level databases that incorporate further sources (e.g. National Strategic Plans, expenditure reporting). These databases could be integrated into the UCSR in the future, and lessons learned from how partners in immunisation, malaria, and SBC adapted the GHCC templates and conducted expedited cost repository processes were presented at the 2019 International Health Economics Association World Congress (DeCormier Plosky, Bollinger, Vaughan, Patouillard, & Vassall, 2019). In addition, due to the intervention-specific nuances to data extraction that make recording in a standard template challenging, the GHCC Reference Case delineation of interventions, activities, and outputs could be reviewed to better understand how broadly that standardisation can serve as a broader template.
Finally, the UCSR displays unit costs as recorded; however, it does not provide guidance regarding which specific cost should be used. Although the UCSR has received excellent reviews for the quality of the product, greater discussion and testing with users, particularly at the country level, remain vitally important. User-centred design can better inform which tools and types of cost data have been or will be employed in country planning, reporting, and grant applications, how exactly these tools are utilised, and what kind of assistance will be most helpful in the years to come. The wide file is a single worksheet that has been exported to Excel from STATA. It contains column headings that align with each of the headings (fields) in the UCSR (filter headings, main result table headings, and headings found on each page in the second level display, and data in each row (associated with a unique unit cost) that provides the information for each heading. 7 These do not apply to studies using a patient perspective. 8 The UCSR is available at https://ghcosting.org/pages/data/ucsr/ app/

Annex A: GHCC inflation procedure
There are different potential methods of inflating extracted and primary data to current USD for both HIV and TB data. The resulting prices are then used in the UCSR for reporting unit costs and inputs as well as for analysis. Given substantial variation in costs with the inflation methods described, a unified approach is recommended. Following Kumaranayake, 1 the general convention for inflating currency over time is to use either the US consumer price index (CPI) or the US gross domestic product (GDP) implicit price deflator. Figure A1 compares the use of both indices using the example of $100 USD in the year 2000 inflated to 2016 USD (the most current year available at the time of the UCSR launch using World Bank data), resulting in 2016 USD values of $139.38 for the CPI (a 39.4% increase over 16 years) and $136.06 for the GDP implicit price deflator (a 36.1% increase). 2 Year-to-year inflation in the United States over this period was steady, ranging from -0.36% to 3.8% for the CPI, and 0.76% to 3.22% for the GDP deflator. Most costs in the extracted and primary data collected by the GHCC are reported during this timeframe, thus the difference in values generated by either inflation method is slight. Nonetheless, a preference has been expressed by GHCC members for the GDP Implicit Price Deflator. The procedure for inflating costs using either index is to verify that all extracted and primary costs are first converted to USD in the reporting year. Then, multiply these costs by the ratio of the index in the reporting year to the index in the current year: e.g. the resulting values are then reported in 2016 USD given US inflation rates from the reporting year to present.
Another method is to account for inflation within countries before converting to current USD. Kumaranayake discusses this briefly, noting that "…unless there is a situation of rapid inflation (e.g. 15-20% or higher), it is generally better to work in local currencies and then convert at the end (p. 234)." This procedure (referred to by the GHCC as the 'Mozambique method') requires first ensuring that data are in the countrycurrency of data collection for the year of reporting, then inflating costs to 2016 in local currency (in our case, using the country-specific GDP implicit price deflator), then finally converting back to 2016 USD using the market exchange rate. As an example, Figure A2 uses the GDP implicit price deflator for Tanzania to illustrate the difference in inflation indices. First, the $100 figure is converted to Tanzanian shillings (TSh) in 2000 using the World Bank's official exchange rates (LCU per USD), then the equivalent TSh value is inflated each year using the country-specific GDP deflator index from the World Bank, with 2000 as the base year for each inflation year from 2001-2016. Then each year is converted back to USD using the same official exchange rate conversion. 3 While overall inflation from 2000 to 2016 is around twice the rate of inflation as the US (64.15%), resulting in $165.14 in 2016 USD, year-to-year inflation varies substantially from a low of -0.0004% between 2010 and 2011, to a high of 20.66% between 2007 and 2008. Furthermore, there is both inflation and deflation during different year-to-year periods for Tanzania. Thus, costs recorded in any given year during this period in either extracted or primary data could experience drastically different rates or directions of inflation.
This pattern of inconsistent year-to-year inflation is not unique to Tanzania. Figure A3 uses the example of 11 of the 12 sub-Saharan African countries for which the GHCC team extracted published cost data for voluntary medical male circumcision, and compares country inflation rates across all of these countries for the same $100 USD in the year 2000 (Zimbabwe is not included because it abandoned its currency in 2009). 4 Overall inflation is relatively similar to the US CPI and GDP inflation indices (39.4 and 36.1% respectively) in a number of countries (Swaziland, 27.9%; Namibia, 32.0%; Botswana, 36.8%; South Africa, 37.9%; Lesotho, 42.5%; Rwanda, 44.5%; and Uganda, 53.5%), while inflation was somewhat higher in Tanzania (65.2%), and substantially different in Mozambique (-32.0%), Zambia (114.4%) and Kenya (162.5%) by 2016. Further complicating matters, year-to-year inflation varies widely by country in the region.
Given the substantially different year-to-year rates of inflation (many of which exceed the recommended 15-20% inflation rates mentioned by Kumaranayake), this method may not be advisable for all costs. An amalgamation of methods for different input costs has been suggested in which input costs such as personnel may be inflated using country-level inflation, and tradable goods could be inflated using US GDP or CPI inflation indices. In order to assess the feasibility of this approach, the GHCC tested generating inflated cost data collected by GHCC using variations of each exchange and inflation method and shared the resulting cost structures among the team. Ultimately, the GHCC decided that input cost reporting was too heterogenous across studies (with studies using various levels of specificity in reporting the source or type of input costs, if any) and that too many assumptions about the nature of these cost inputs was necessary to arrive at a thorough and replicable set of rules for the broad array of collected data. 1 Kumaranayake, Lilani. (2000). How to do (or not to do) … The real and the nominal? Making inflationary adjustments to cost and other economic data. Health Policy and Planning, 15 (2): 230-234. 2 Figure A1 uses US CPI and US GDP deflator data from the World Bank, available online. US CPI data are available at <https://data.worldbank.org/indicator/FP.CPI.TOTL>, and data series displayed is "Consumer Price Index (2010 = 100)". US GDP Deflator data are available at: <https://data.worldbank.org/ indicator/NY.GDP.DEFL.ZS>, data series displayed is "GDP deflator (base year varies by country)", and all country-level data exported by clicking on either CSV or EXCEL. For both series, only data for the United States are used to generate the figure. 3 Exchange rate data comes from the World Bank Official Exchange Rate (LCU per US$), available online at <https://data. worldbank.org/indicator/PA.NUS.FCRF>. Data are then inflated using local inflation using the GDP deflator. Data are available at <https://data.worldbank.org/indicator/NY.GDP.DEFL.ZS>, and the data series "GDP deflator (base year varies by country)" is used. Data are then converted back to USD using the same World Bank Official Exchange Rate data from above. Note that only in cases where the exchange rate is not reported do we use the WB exchange rates. These rates are the annual averages published on the World Bank website <https://data.worldbank. org/indicator/PA.NUS.FCRF>. If the date of the currency is not mentioned in the paper, we use the publication year minus one. 4 Figure A3 uses the same procedure as Figure A2, with countryspecific inflation data pulled from the same World Bank data source here: <https://data.