Measuring causes of adult mortality in rural northern Malawi over a decade of change

Background Verbal autopsy could be more widely used if interpretation by computer algorithm could be relied on. We assessed how InterVA-4 results compared with clinician review in diagnosing HIV/AIDS-related deaths over the period of antiretroviral (ART) roll-out. Design In the Karonga Prevention Study demographic surveillance site in northern Malawi, all deaths are followed by verbal autopsy using a semi-structured questionnaire. Cause of death is assigned by two clinicians with a third as a tie-breaker. The clinician review diagnosis was compared with the InterVA diagnosis using the same questionnaire data, including all adult deaths from late 2002 to 2012. For both methods data on HIV status were used. ART was first available in the district from 2005, and within the demographic surveillance area from 2006. Results There were 1,637 adult deaths, with verbal autopsy data for 1,615. Adult mortality and the proportion of deaths attributable to HIV/AIDS fell dramatically following ART introduction, but for each year the proportion attributed to HIV/AIDS by InterVA was lower than that attributed by clinician review. This was partly explained by the handling of TB cases. Using clinician review as the best available ‘gold standard’, for those aged 15–59, the sensitivity of InterVA for HIV/AIDS deaths was 59% and specificity 88%. Grouping HIV/AIDS/TB sensitivity was 78% and specificity 83%. Sensitivity was lower after widespread ART use. Conclusions InterVA underestimates the proportion of deaths due to HIV/AIDS. Accepting that it is unrealistic to try and differentiate TB and AIDS deaths would improve the estimates. Caution is needed in interpreting trends in causes of death as ART use may affect the performance of the algorithm.

O ver the past 20 years, there have been dramatic changes in both the levels and causes of mortality in sub-Saharan Africa. Initial catastrophic rises in mortality due to HIV have been partially curtailed by antiretroviral therapy (ART) (1), and deaths due to diseases conventionally associated with more affluent societies are becoming more prominent (2).
Few countries in sub-Saharan Africa have vital registration or death certification, so estimation of mortality rates and distribution of causes of death relies on indirect methods or on detailed studies in demographic surveillance sites. In these sites, regular population updates through repeated censuses and/or continuous registration systems ensure almost complete registration of births and deaths. Accurate determination of the cause of death is much more complex. Many deaths will occur without clinical care, and diagnostic facilities for those who do reach care are often limited.
Demographic surveillance sites rely on verbal autopsy to classify major causes of death (3,4). An informant, preferably the caregiver during the final illness, is interviewed using a set of standard questions. Information may be added from health passports if available. A diagnosis is made based on clinician review or through computer algorithms such as InterVA (5). In another paper in this series, the performance of InterVA-4 based on responses only (not HIV status unless reported during the verbal autopsy) was assessed against known HIV status across six demographic surveillance sites (6). This showed a high specificity (90%) but could not estimate sensitivity as HIV-positive individuals can die from other causes.
The Karonga Prevention Study, one of those six sites, has unusually detailed data collection and clinician review (7). We have previously documented adult mortality rates and causes of death up to 2009, showing a decline in allcause mortality following antiretroviral roll-out and replacement of HIV/AIDS by non-communicable disease as the leading cause of death (2). We now update this analysis to 2012 and assess the performance of InterVA against the conclusions reached by the clinicians and investigate how this varies by age group, sex, time period (pre-and post-antiretroviral roll-out), and HIV status.

Methods
The demographic surveillance site covers a population of about 35,000 in the southern part of Karonga District, northern Malawi. It is predominantly rural, although half of the population lives within 1 km of a tarmac road. The main occupation is subsistence farming. HIV was already present in the area in the early 1980s. The prevalence peaked in the mid-1990s and is now around 9% in women and 7% in men (8). The HIV is mainly subtype C (9).
Demographic surveillance started in 2002, with a baseline house-to-house census which covered the whole area by 2004 (10). This records detailed data on personal identifiers, sociodemographic and economic status for all individuals, and the physical location of all households. Community-based key informants are responsible for updating information within defined geographical areas that cover between 30 and 40 households. They report monthly on births and deaths and collect migration data for annual reports in their area. Births and deaths are followed-up immediately by the field staff, and all events are checked during annual re-censuses.
In 2005Á06, a HIV sero-survey was conducted in selected clusters of the demographic surveillance area. Between 2007 and 2011, four annual house-to-house HIV sero-surveys were completed in the whole demographic surveillance area. HIV rapid tests were used, with results available to the participants during the visit (11). The serosurveys followed about 6 weeks after re-census in each area (8). All adults (]15 years old) who were resident in the demographic surveillance area at the time of the re-census were visited at home, with up to three repeat visits if necessary. Consent was sought separately for interviewing and HIV testing. Participants were encouraged to know their status, but could accept testing and choose not to know the result, in which case HIV testing was performed at project headquarters. Interviewers asked about previous HIV testing, including the timing and result of the most recent test, and about ART use if the participant reported that they were HIV positive. HIVpositive individuals who elected to know their status were referred for screening for eligibility for ART. HIV results are also available from linked epidemiological studies within the area.
Free ART was first available at the Karonga district hospital in July 2005, in the rural hospital within the demographic surveillance area in September 2006, and at further clinics within the area from October 2010. By mid-2008, ART uptake in the demographic surveillance area was estimated to be 58% of those eligible (12). Initial criteria for eligibility were WHO stage 3 or 4 disease or CD4 of B250 cells/mm 3 (after a brief period of B200). CD4 counts were rarely available so eligibility was usually assessed clinically. The criteria for eligibility widened to include those with CD4 up to 350 cells/mm 3 in July 2011.

Cause of death
After a death has been reported by a key informant, a verbal autopsy is conducted as soon as possible, after allowing a 2-week mourning period. An interviewer with clinical training visits the deceased's household, and if consent is given, fills in a semi-structured verbal autopsy form (2). This is similar to the 2003 INDEPTH verbal autopsy tool, which was an adaptation of the contemporaneous WHO questionnaire (13). Close relatives of the deceased, preferably those who were present most of the time or who nursed the deceased before they died, are asked an open-ended narrative question about the death, and then a series of closed questions about specific symptoms. The completed questionnaire is given to two independent clinicians (physicians or clinical officers) who independently assign a cause of death. If the two assigned causes of death are discordant, a third physician reviewer assesses all the evidence, including the two previous reviews, and assigns a cause of death. Information on HIV status is available to the reviewers if known. Data on ART use was asked specifically from 2009 and may have been mentioned by relatives before that. Deaths of individuals with tuberculosis were assigned as 'TB', if they had good evidence of tuberculosis (TB) and no other symptoms of AIDS. Otherwise, they were classified as 'AIDS' or 'TB/ AIDS unspecifiable'. Different clinicians have assigned the cause of death over the long period of study. The two coding clinicians agreed on cause of death (comparing the categories 'not TB/AIDS', 'AIDS', 'TB', 'TB or AIDS', 'unspecifiable') for 82% of deaths (Kappa 0.68). Grouping the TB and AIDS categories, the overall agreement increased to 87%, Kappa 0.76.
For the InterVA analysis, the original data from the verbal autopsy forms were entered according to the InterVA-4 required format. Data were directly extracted from the questionnaire for 112 of the 190 InterVA-4 input items which are relevant for adult deaths. The remaining input items were left as missing. Data were available for all the key symptoms associated with HIV-related deaths (e.g. oral candidiasis and wasting), and where possible, available data on HIV status, including self-report, were also included in the InterVA input.

Ethical approval
Ethical approval for the study was obtained from the National Health Sciences Research Committee of Malawi and the Ethics Committee of the London School of Hygiene and Tropical Medicine.

Data analysis
Data from August 2002 to the end of 2012 were included. The study period was divided into three: pre ART (before 7 July 2005); during ART-rollout up to 2 years after ART availability within the demographic surveillance area (7 July 2005Á6 September 2008); and more than 2 years after the availability of ART locally (after 6 September 2008). Person-time at risk (in years) was used as the denominator in the calculation of all mortality rates. Analysis was done using Stata version 13 and the InterVA-4 Model was run through R Studio (version 1.0) (14). InterVA-4 requires estimates of the prevalence of HIV and malaria among deaths in the population; these were set to be high ( 1% of all deaths).
Analyses compared the overall mortality rates and the proportion of deaths attributed to HIV/AIDS as estimated by InterVA and by clinician review. We used the InterVA estimations [as in Ref. (6)] but included any additional available HIV data not already reported during the verbal autopsy. InterVA calculates the likelihood for each cause of death for each individual, and these are summed across the whole population or subgroups. We explored the effect of including or excluding TB cases with AIDS cases in both the InterVA and clinician estimates as the symptoms and signs of TB and AIDS are difficult to distinguish. Finally, we compared the most likely cause of death for each individual as assigned by InterVA with that assigned by clinician review. While clinician review is far from perfect, it is the best available diagnosis, and was used as a 'gold standard' against which to estimate the sensitivity and specificity of InterVA. The data used by the two methods was almost the same, except that the clinicians see the narrative which is not available to InterVA, and InterVA was given additional data on HIV status (from self-report and ART use data) that was not necessarily known to the clinicians unless mentioned in the narrative.

Results
From August 2002 to December 2012, a total of 1,637 deaths of individuals aged 15 years and above were registered in 155,875 person-years at risk giving a crude mortality rate of 10.5/1,000 person-years [95% confidence interval (CI): 10.0Á11.0]. Verbal autopsy questionnaire data were available for 1,618 (99%). Three had no clinician review available, leaving 1,615 with information from both methods. Table 1 shows age-specific all-cause mortality rates stratified by ART availability. All cause adult mortality rates declined from 13.8/1,000 person-years (95% CI: 12.6Á15.1) before ART introduction in the district to 8.5 (95% CI: 7.9Á9.2) once ART provision was widespread, with the reduction seen in all age groups except the over 60s.
The proportion of deaths attributable to HIV/AIDS in different time periods according to the different methods of ascribing cause is shown in Table 2 for those aged 15Á59 and in Table 3 for those aged 60 and over. HIV status was known (by test result or report) for 606 deaths, of which For those aged 15Á59, the proportion of deaths due to HIV/AIDS decreased over time whichever method was used, but the proportion attributed to HIV/AIDS was higher when based on clinician review than when using InterVA. This was partially explained by the way TB was handled by the two methods. If TB deaths were grouped with AIDS deaths, there was less discrepancy between the methods. In those aged 60 and over, the proportion of deaths attributable to HIV/AIDS was much lower, and similar in InterVA and clinician review. In this age group, including TB deaths greatly increased the discrepancy, with InterVA over-diagnosing TB deaths compared to the clinician review.
To allow a more detailed examination of the diagnoses reached by InterVA and clinician review, the most likely cause determined by InterVA for each individual was compared with that from the clinician review (Table 4). This shows the results by age group, and for those aged 15Á59, by sex, time period, and HIV status. Of the 320 deaths in age group 15Á59 attributed to HIV/AIDS by InterVA, clinician review attributed 259 to AIDS or TB/ AIDS, 2 to TB only, 6 to other communicable disease, 40 to non-communicable disease, 1 to external causes, and 12 were indeterminate. Of the 124 attributed to TB by InterVA, only 18 were attributed to TB alone by clinician review with 11 attributed to TB/AIDS, 75 to AIDS, 2 to    107  5  0  3  19  0  8  142  TB  37  5  13  1  8  0  1  65  Other  49  1  4  57  63  52  16  242  Indeterminate  4  0  0  1  11  8  11  35  Total  197  11  17  62  101  60  36  484  Women  HIV/AIDS  144  3  2  3  21  1  4  178  TB  38  6  5  1  7  1  1  59  Other  47  1  1  27  66  53  10  205  Indeterminate  2  0  0  2  3  2  1  10  Total  231  10  8  other communicable disease, and 15 to non-communicable disease. By contrast there was poor correlation between InterVA and clinician diagnoses for the over 60s. Of 30 deaths attributed to HIV/AIDS by InterVA, 9 were attributed to AIDS or TB/AIDS by clinician review, with 6 other communicable diseases and 14 non-communicable diseases. And of 56 attributed to TB by InterVA, clinician review attributed 12 to TB or TB/AIDS, 7 to AIDS, 3 to other communicable disease, and 32 to non-communicable disease. Diagnoses reached by InterVA and clinician review were also compared by HIV status (Table 4). Neither method assigned HIV/AIDS as the cause if there was a recent negative HIV test (with one exception) and neither method automatically assigned HIV/AIDS as the cause if there was a positive HIV test. The relative underreporting of HIV/AIDS deaths by InterVA was more marked in those known to be HIV positive than HIV unknown: among those known to be HIV positive, HIV/ AIDS deaths were diagnosed for 47% by InterVA and 80% by clinician review (53 and 84%, respectively, if on ART). For those with unknown status, HIV/AIDS deaths were diagnosed for 35% by InterVA and 47% by clinician review.
The sensitivity and specificity of InterVA compared to clinician review is shown in Table 5 (excluding those with undetermined cause of death). Among those aged 15Á59, the sensitivity of InterVA for identifying HIV/AIDS deaths was 59% and the specificity was 88%. Sensitivity decreased with age and was only 32% in those aged over 60. Among those aged 15Á59, the sensitivity was lower in the last period, after ART use was widely established. Specificity varied less and was lowest during ART introduction. Grouping HIV/AIDS and TB together greatly improved the sensitivity of InterVA in all groups, with little loss in specificity. In the 15Á59 age group, the sensitivity was 78% and the specificity 83% for the combined definition.

Discussion
In this setting, adult mortality rates have fallen dramatically since ART became widely available. This is mirrored by the decreasing proportion of deaths attributable to HIV/AIDS. This decrease is seen with both methods of classifying the verbal autopsy data Á InterVA or clinician review Á but there were differences between them, with the proportion attributed to HIV/AIDS by InterVA being lower than that attributed by clinician review in all year groups for those aged 15Á59. The difference between the two methods was particularly marked in those who were known to be HIV positive. Part of the difference in performance in the 15Á59 age group is explained by the classification of TB and TB/ AIDS deaths. If all TB deaths were grouped with the AIDS deaths for both methods the results were much more similar, although clinician review continued to attribute a slightly higher proportion to HIV/AIDS/TB in this age group.
In the over 60s, the proportion of deaths attributed to HIV/AIDS without TB was low and similar in the two methods. In this age group, the inclusion of TB deaths greatly increased the proportion attributed to HIV/AIDS/ TB by InterVA, with much less increase in the clinician review diagnoses. As shown in Table 4, of the 56 TB cases diagnosed by InterVA in the over 60s, only 34% (19 cases) were classified as TB or AIDS by clinician review, and 57% (32 cases) were thought to be due to non-communicable disease. By contrast, among those aged 15Á59 diagnosed as TB deaths by InterVA, 84% (104/124) were classified as TB or AIDS, and only 12% (15/124) as non-communicable disease by clinician review. In Karonga, TB case finding and diagnosis is good, as TB has been a major focus of studies for 30 years, so it is likely that it is the InterVA diagnoses that are more often incorrect. In the over 60s, although the overall numbers attributed to HIV/AIDS by the two methods was similar, at an individual level there was poor correlation so the similarity is probably due to chance.
The estimates of sensitivity and specificity show that the sensitivity of InterVA can be greatly improved, with little loss of specificity, by grouping HIV/AIDS/TB as one diagnostic category. An analysis in Nairobi comparing InterVA with AIDS mortality estimates from spectrum also found under-estimation by InterVA which was improved by including deaths classified as TB (15). A study in Ethiopia using hospital diagnoses and HIV testing as the comparison group found improved sensitivity without loss of specificity for InterVA when combining TB and AIDS (16), and a study in Kenya found that InterVA tended to over-diagnose TB when compared to hospital diagnoses (17). A comparison in a demographic surveillance site in Ethiopia found poor agreement between InterVA and physician review for both HIV/ AIDS and TB diagnoses (18), and worse agreement in older adults (19). A large multi-country study also found relatively poor performance by InterVA compared to physician review where tertiary hospital cause of death assignments, with stringent diagnostic criteria, were used as the gold standard (20,21), though methodological questions have been raised about some of the comparisons in this study (22).
Obviously, clinician review of verbal autopsy data is not a true gold standard. Even pre-mortem clinical diagnoses in settings with good clinical facilities are often contradicted by conventional autopsy (3,23,24), and reaching a diagnosis based on a limited amount of reported information is much more difficult. What the comparison does test, however, is how well the computer algorithm performs compared to two (or in cases of discrepancy, three) clinicians working in the local environment, with essentially the same information. The results suggest that using the InterVA definition of HIV/AIDS alone will underestimate AIDS death, but that it performs better for the combined diagnosis of HIV/AIDS/TB, and this amendment could be recommended in the 15Á59 age group. Results in the over 60s appear less reliable. The apparently lower sensitivity of InterVA for diagnosing HIV/AIDS/TB after established ART use in the population suggests that considerable caution will be needed in interpreting trends in HIV-related deaths.