A structured review evaluating content validity of the Asthma Control Test, and its consistency with U.S. guidelines and patient expectations for asthma control

Abstract Objective To assess whether the content of the Asthma Control Test (ACT) served as a valid measure of asthma control (i.e., content validity) by mapping ACT items to the National Heart, Lung and Blood Institute (NHLBI) guideline asthma control definitions, and to language used by patients to describe their asthma. Data sources PubMed and EMBASE databases were used for a structured literature analysis. Study selections Full-text, English-language articles that reported findings from qualitative studies conducted in adults, focusing on patient descriptors of asthma symptoms, impacts, or severity, were included. Pediatric studies, studies conducted in patients without asthma, and studies that did not contain qualitative data were excluded. Results ACT items reflected all domains of asthma impairment described in the NHLBI guidelines, except pulmonary function. Following the literature review, 28 full-text publications were identified that included patient descriptors that could be mapped to ACT items. For example, per ACT Item 1, patients described having trouble at work, school, and completing household chores; and, per ACT Item 2, patients used the phrase “short of breath” to describe asthma-associated symptoms. Conclusion ACT item content corresponded well with the NHLBI guideline definitions of the impairment domain of asthma control (focused on asthma symptoms and impact), and we identified numerous examples in the literature indicating that ACT concepts and item content mirror the language patients use when discussing asthma symptoms and impact, and their degree of asthma control. This provides further evidence to support content validity of the ACT as a measure of asthma control.


Introduction
Long-term maintenance of asthma control is a key goal of asthma management (1). Asthma control involves reducing asthma-related impairment, including managing symptoms, and maintaining near-normal pulmonary function and activity levels, and reducing risk, with respect to the frequency and intensity of asthma exacerbations, and loss of lung function (2).
Physicians build their assessment of asthma control on the goals of asthma therapy that are outlined in the National Heart, Lung and Blood Institute (NHLBI) asthma guidelines (2). These are based on querying a patient's report of relevant clinical status (3), and thus may rate a patient's level of asthma control differently than the patient would themselves. Without sufficient information, patients may assume that asthma symptoms are inevitable and come to accept, or fail to recognize, poorly controlled asthma (4). The use of patient-reported outcome measures (PROMs) can improve physicians' understanding of patients' health, and support clinical decision-making (5), and also change how patients feel about their condition (6). The U.S. Food and Drug Administration released guidance on PROMs for use in clinical trials that explicitly states PROMs must be constructed with input from patients (7). Many PROMs have been developed without patient input and have been evaluated psychometrically, but without additional patient input in constructing or validating the PROM, it could become invalidated as a study endpoint in clinical trials. It is important, therefore, that any PROM is validated and can produce meaningful outcomes to patients, caregivers, and clinicians (8). PROMs that are appropriate, simple, relevant, and accessible are desirable for reliable assessment and measurement of given health concepts (9).
The Asthma Control Test (ACT) (3) is a brief, patient-reported assessment of asthma symptoms and impact that evaluates the impairment domain of asthma control in patients with asthma. The ACT is typically used in clinical practice to monitor the effectiveness of asthma management and to support treatment decisions.
The ACT represents a questionnaire that uses patient-specific language to measure the concept of interest (i.e., asthma control). The ACT was developed from a 22-item draft questionnaire completed by 471 patients with asthma. From this, a stepwise regression was performed to support a five-item questionnaire that assesses patient-reported asthma control (3). The ACT has previously been validated for use in clinical practice (10) and for administration over the telephone (11), and the reliability and validity (including criterion, construct, and predictive validity) of the ACT have been demonstrated previously in observational studies (12,13). Additionally, the psychometric properties of the ACT in two randomized clinical trials of differing design (14,15) have been evaluated (16). However, establishing whether the content of ACT items provides a valid assessment of asthma controlas defined by the NHLBI asthma guidelines (2) and whether it is relevant in reflecting patient experiences of asthma (i.e., content validity), is needed in addition to psychometric evaluation to further support the ACT as an appropriate PROM for use in assessing asthma control.
This study had two aims: 1) to map ACT item content to the 2007 NHLBI asthma guidelines descriptions of the impairment domain of asthma control; 2) to map the language used by patients to describe asthma control (from a literature search of qualitative studies) with the content found in ACT items, in order to evaluate content validity.

ACT versus NHLBI guidelines
When comparing the ACT with current U.S. guidance, we mapped ACT items (Table 1) to the asthma control definitions and guidelines established by the NHLBI (2).

Literature review to establish content validity of ACT items
To provide evidence of content validity, we reviewed the literature to identify published patient-reported language related to asthma control, symptoms, and impact. We then compared these findings against the ACT as a measure of the impairment domain of asthma control. A structured literature review was conducted using the PubMed (the National Center for Biotechnology Information, Bethesda, MD, USA) and EMBASE (Elsevier BV, 2017) databases, in order to capture publications that focused on studies reporting patients' descriptions of asthma (i.e., asthma control, symptoms, and impact). The database search terms used are detailed in Figure 1. Truncation of search terms was used to identify all words or phrases that began with a given text string. Articles unavailable in English were excluded. Studies were also excluded if they enrolled pediatric patients [i.e., patients <12 years of age, based on NHLBI's clinical treatment algorithms (2)], were conducted in patients without asthma, and did not contain qualitative data.
All abstracts were reviewed to select articles for full-text review. Full-text articles that reported patient descriptors of asthma symptoms, impacts, or severity were included. Articles with abstract content deemed irrelevant to the study objectives were excluded from full-text review. During the full-text review process, the references cited in each relevant publication were reviewed for any additional articles.

NHLBI guidelines and ACT comparison
The language used in the ACT items to describe attributes of the impairment domain of asthma control was largely similar to that used in the NHLBI definitions (Table 2). Guideline content reflected in ACT questions includes effects on normal activity, work, and school (ACT Item 1); the presence of troublesome symptoms, specifically dyspnea (ACT Item 2); nighttime awakening due to asthma (ACT Item 3); and frequency of rescue therapy use (ACT Item 4). The ACT items reflect all subjective domains of asthma impairment described in the NHLBI asthma guidelines, with the exception of pulmonary function (which is an objective endpoint, not a patient-reported attribute). It is important to note that the ACT does not provide an assessment of the asthma risk domain of the NHLBI, which relates to the likelihood of developing exacerbations, a progressive decline in lung function (including worsening of symptoms), or adverse effects due to treatment.

Search results
When combined, both literature searches yielded a total of 732 publications (PubMed, n ¼ 379; EMBASE, n ¼ 353) and, after the identification and removal of duplicate articles, 441 unique articles underwent abstract screening for relevance. Of these 441 articles, 82 were eligible for full-text review. Three additional articles were added from a review of the references cited in each publication. Of these 85 articles, 28 (which covered a time period from 1992-2017) were found to include patient descriptors of experience of asthma symptoms and impacts that could be mapped to the ACT ( Figure 1).

Patient language comparison
Details of the language used by patients that could be mapped to ACT items (including both direct quotes and information provided by researchers) are provided in Table 3  . Our literature review identified numerous examples indicating that the topics assessed by the ACT, and the specific language used in each item, closely mirrors the language that patients use when discussing the impact of asthma and its symptoms.
Patients frequently discussed the impact that asthma had on their ability to perform everyday tasks and chores, as assessed by ACT Item 1. For example, asthma led to problems at both work and school (25,27), and affected the ability to complete household chores (26). Thus, ACT Item 1 was seen to evaluate an area of impact that is recognized by patients with asthma.
ACT Item 2 assesses shortness of breath ("During the past 4 weeks, how often have you had shortness of breath?"). "Shortness of breath" and being "short" of breath were terms used by patients describing asthmaassociated breathing problems (34,36,37). Furthermore, a study by Nelsen et al. (22) reported that 72% of patients with severe asthma who participated in a concept elicitation interview spontaneously described "shortness of breath" as a symptom, while an additional 17% of patients endorsed the symptom after probing. These results were in line with the wording used in ACT Item 2, and indicate that "shortness of breath" is a common and relevant term used by patients with asthma when discussing symptoms.   (21) I'm not able to get up and clean up the house Nelsen, 2017 (22) I remember once when I had been off work for a couple of days with an asthma attack. When I came back to work I was still very tired Snadden, 1992 (23) I mean I've had jobs where I've worked in canneries in the packing room an' the fumes from the, the glue pot for putting labels on is impacted us an' that, that I did suffer at work then. Eh, an' I've been on other jobs where eh, where I would say the, the environment impacted my breathing an' brought on asthma Steven, 2004 (24) It even impacts my work. Gater, 2016 (29) … the evening is when it's really hard on me. It bothers me more at night than it does during the day and I'm trying to figure out why is that? Lawson, 2014 (30) Well, I usually wheeze just about every day I be wheezing, and at night it's more serious. It -I mean, you know, I cough a lot at night, and, uh, you know, I'mbasically I'm on medication so it helps. But it's usually pretty serious Nelsen, 2017 (22) It's been at night and I've been lying there and all of a sudden I haven't been able to breathe at all and I've been terrified Snadden, 1992 (23) I didn't know what it was like to have a full night's sleep up until, oh I think it was, was it maybe the end of last year? Steven, 2004 (24) I just was getting a slight wheeze at times, at night really Steven, 2002 (33) … certainly the lack of sleep is a definite drawback, especially as I get older, I can't really cope as well with work over extended periods of lack of sleep Steven, 2002 (33) Wake-up at night Trochtenberg, 2007 Gabe, 2002 (27) I don't get it every night but Mum says I usually cough a lot during the night but I don't usually remember Gabe, 2002 (27) (Continued) ACT Item 3 assesses how often patients have been awakened by asthma symptoms during the night or early in the morning. The literature review identified examples where patients specifically described being woken by nighttime wheezing (28), shortness of breath (29), or coughing (25), while other patients described the experience of chest pain upon waking in the morning (17). Some patients also discussed the impact of asthma on their ability to sleep well without describing specific symptoms (24,30,34), and wondered why their asthma seemed to get worse at night (30). These descriptors indicate that ACT Item 3 captures a relevant area of impact, and that the specific wording and examples provided by the item appropriately reflect patient experiences with asthma-related awakenings during the night.
ACT Item 4 assesses the frequency of rescue medication use. Evidence to support the relevance of ACT Item 4 was found across multiple publications, wherein patients' descriptors of asthma medication use focused heavily on describing their use of rescue inhalers. Patients demonstrated an awareness of rescue medication use frequency (40). Some studies reported descriptors that indicated patients felt they were taking rescue medications too frequently (22,24,39), and also that some patients felt uncomfortable if they did not have a rescue inhaler close by Refs. (41,42).
ACT Item 5 asks patients to rate their asthma control. In published reports, patients generally understood the meaning of "asthma control," and used this phrase when discussing asthma symptoms and impacts. Patients discussed a lack of control (e.g., asthma that "can't be controlled") when their symptoms worsened (30,40), while other patients described their asthma as "well-controlled" when discussing the relationship between asthma and daily activities (24). In a study where patients were asked to evaluate content areas assessed by asthma-related quality of life surveys, some indicated that an assessment of asthma control and management would be relevant to them, and highlighted that this was currently missing from the surveys they evaluated (the ACT was not included in this study) (43). Thus, the reviewed literature supports that ACT Item 5 represents a topic of importance for patients when considering the state of their asthma, and uses terminology that would be readily understood by patients. Yet another type of preventive overuse was described that only involved the reliever medication. It was regarded as a required preventive measure to make sure that the asthma would not play in the foreground, for example when performing strenuous physical activities (r) Axelsson, 2011 (17) The blue one takes a battering, it really does and that's probably not the best way of doing things but it's the only way I  Steven, 2004 (24) "Control" was expressed in terms of the management of symptoms and the management of inhalers (r) Steven, 2002 (33) ACT, Asthma Control Test; ED, emergency department. a Items marked with (r) indicate the text was a summary provided by researchers, and not a direct patient quote.

Discussion
The ACT was created specifically to assess patientreported symptoms and perceptions of asthma control. Our study demonstrated close alignment of ACT items with both NHLBI descriptors of asthma control (2) (i.e., descriptors of asthma symptoms and impact included in the "impairment domain" of asthma control), and with patient descriptors of asthma control, symptoms, and impact reported in the literature, thereby supporting the content of the ACT as a valid measure of asthma control. Evidence for content validity is often derived directly from qualitative research with patients (7,(45)(46)(47). However, as the ACT measures a physician-derived concept of asthma control, it is important to confirm both that the ACT measures the clinical concept of asthma control, and to confirm indirectly that the wording presented to patients is consistent with the patient experience of these symptoms (as taken from published literature).
The NHLBI recognizes the ACT as a validated instrument for assessing asthma control (2), and identifies it as providing an appropriate composite score for asthma control when assessed by an expert group assembled by the National Institutes of Health (48). Our study has demonstrated that the content of ACT items corresponds well with how the NHLBI asthma guidelines define the impairment domain of asthma control in terms of symptoms and impacts, thus supporting the overall content validity of the ACT as a measure of asthma control.
Taken together with ACT items, the descriptors that patients provide related to asthma symptoms and impacts provide evidence of the content validity of the ACT from a patient perspective. Specifically, each of the items included in the survey assesses content areas that can be directly linked to patient descriptors of asthma control, matching both the general experiences and specific language used by patients with asthma.
A strength of this study is that it surveyed a wide spectrum of literature spanning a 25-year period, including a heterogeneous group of patients in terms of demographic and asthma clinical characteristics. Our findings indicated that the descriptors used to define asthma impact, symptoms, and control had not changed over time, and that they did not differ across patient groups. A possible study limitation is that information was based on a literature review of patient descriptors of asthma, rather than a qualitative study designed specifically for patients to discuss the relevance of ACT items, and their understanding of each aspect of the ACT. Additionally, there may be ways in which patients describe or characterize their asthma that are not represented among the items comprising the ACT, and patient choice of vocabulary may be influenced by prompts that healthcare providers are encouraged to use. From our results, it is also not clear to what extent variations in patient subjective experiences predict levels of asthma severity. Further studies considering such factors could be beneficial in helping to improve treatment outcomes.
Although the ACT items and NHLBI asthma guidelines generally focus on the same patient descriptors of asthma, the ACT is entirely patient-reported, and as such, does not reflect any assessments of spirometry, a key component of clinical asthma management. Consistent with other measures of asthma symptoms and impacts, ACT scores have been shown to be significantly related to spirometric measures, but correlation coefficients are often relatively low (10,49), and factor analysis has shown that the ACT is mostly independent of pulmonary function (50). Thus, assessment of pulmonary function alongside the ACT as a standardized assessment of the impairment attributes of asthma control would complete the definition of guideline-based impairment. Additionally, assessing the risk of asthma exacerbations, e.g., by considering the level of asthma control, previous exacerbations, and nonadherence to inhaled corticosteroids, alongside the ACT and pulmonary function, would more completely define guideline-based asthma control by assessing the risk domain of asthma control. Despite these potential limitations, the results of this study nevertheless support the content validity of the ACT as a measure of asthma control.

Conclusions
This structured literature review supports the hypothesis that the ACT measures concepts consistent with the NHLBI definitions of asthma control, which adds to previous quantitative psychometric validation studies in both observational and randomized, controlled settings. The reviewed studies identified numerous examples to support that the ACT concepts and item content correlates with the language patients use when discussing asthma symptoms and asthma impact, further supporting the content validity of the ACT as a measure of asthma control.