Multicenter evaluation of two chemiluminescence and three lateral flow immunoassays for the diagnosis of COVID-19 and assessment of antibody dynamic responses to SARS-CoV-2 in Taiwan

ABSTRACT This multicenter, retrospective study included 346 serum samples from 74 patients with coronavirus disease 2019 (COVID-19) and 194 serum samples from non-COVID-19 patients to evaluate the performance of five anti-severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody tests, i.e. two chemiluminescence immunoassays (CLIAs): Roche Elecsys® Anti-SARS-CoV-2 Test (Roche Test) and Abbott SARS-CoV-2 IgG (Abbott Test), and three lateral flow immunoassays (LFIAs): Wondfo SARS-CoV-2 Antibody Test (Wondfo Test), ASK COVID-19 IgG/IgM Rapid Test (ASK Test), and Dynamiker 2019-nCoV IgG/IgM Rapid Test (Dynamiker Test). We found high diagnostic sensitivities (%, 95% confidence interval [CI]) for the Roche Test (97.4%, 93.4–99.0%), Abbott Test (94.0%, 89.1–96.8%), Wondfo Test (91.4%, 85.8–94.9%), ASK Test (97.4%, 93.4–99.0%), and Dynamiker Test (90.1%, 84.3–94.0%) after >21 days of symptom onset. Meanwhile, the diagnostic specificity was 99.0% (95% CI, 96.3–99.7%) for the Roche Test, 97.9% (95% CI, 94.8–99.2%) for the Abbott Test, and 100.0% (95% CI, 98.1–100.0%) for the three LFIAs. Cross-reactivity was observed in sera containing anti-cytomegalovirus (CMV) IgG/IgM antibodies and autoantibodies. No difference was observed in the time to seroconversion detection of the five serological tests. Specimens from patients with COVID-19 pneumonia demonstrated a shorter seroconversion time and higher chemiluminescent signal than those without pneumonia. Our data suggested that understanding the dynamic antibody response after COVID-19 infection and performance characteristics of different serological test are crucial for the appropriate interpretation of serological test result for the diagnosis and risk assessment of patient with COVID-19 infection.


Introduction
Coronavirus disease (COVID-19), a potentially lifethreatening respiratory tract disease, emerged at the end of 2019 in China and rapidly evolved to become a pandemic, resulting in severe repercussions to human health and life [1][2][3]. As of July 19, 2020, there have been 14,043,176 confirmed cases of COVID-19 and 597,583 (4.26%) deaths reported to the World Health Organization [4]. The early identification and isolation of patients with COVID-19 to prevent the transmission of the causative agent, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), in the community are vital for curbing the pandemic. However, the identification of patients with COVID-19 is challenging because of its broad spectrum of clinical manifestations, ranging from asymptomatic infection to critical illness requiring intensive care [3,5]. The real-time reverse transcriptase-polymerase chain reaction (qRT-PCR) assay directly amplifies and detects SARS-CoV-2-specific viral nucleic acid sequences and has been an important and irreplaceable diagnostic tool for COVID-19 detection. However, the diagnostic sensitivity of qRT-PCR assay is influenced significantly by the timing of specimen collection during the disease course, site of specimen collection, and skill required during specimen collection [6,7]. It is therefore plausible that a significant proportion of patients with COVID-19 failed to be diagnosed using qRT-PCR [8,9].
Serological testing, i.e. detection of anti-SARS-CoV-2 antibodies simultaneously or separately, can be used as an adjuvant to qRT-PCR in the clinical diagnosis of acute COVID-19 [10][11][12]. Furthermore, serological testing can detect a previously undiagnosed SARS-CoV-2 infection among individuals for whom qRT-PCR was either falsely negative or not performed. Serological testing is therefore useful for studying the epidemiological seroprevalence of COVID-19 to obtain a more accurate estimate of the circulating dynamics and virulence of SARS-CoV-2 [13]. To date, many point-of-care or fully automated immunoassays for COVID-19 diagnosis have been developed and launched [11,12,[14][15][16][17]. However, the performance and usefulness of different serological tests should be fully evaluated in clinical laboratories before its large-scale application into routine diagnostic protocols for patient management and pandemic control. The primary goal of this study was to evaluate the performance of five serological tests: Elecsys® Anti-SARS-CoV-2 (Roche Test), SARS-CoV-2 IgG (Abbott Test), Wondfo SARS-CoV-2 Antibody Test (Wondfo Test), ASK COVID-19 IgG/IgM Rapid Test (ASK Test), and Dynamiker 2019-nCoV IgG/IgM Rapid Test (Dynamiker Test) for the diagnosis of COVID-19. Further, previous studies have shown a correlation between the high-level upsurge of the anti-SARS-CoV-2 antibody response and tissue injury among patients with COVID-19 [18][19][20]. The secondary goal of this study was to validate the dynamic immune responses among patients with COVID-19 of different clinical severity with the individual or collective testing results of the five serological tests.

Policy and status of the COVID-19 epidemic in Taiwan
In Taiwan, the respiratory tract specimens from  patients who meet the reporting criteria for COVID-19 have to be submitted to virology laboratories validated and associated with the Centers for Diseases  Control of Taiwan (Taiwan CDC) for SARS-CoV-2 qRT-PCR assay [21]. Three sets of primers and probes targeting the SARS-CoV-2 envelope (E), nucleocapsid (N), and RNA-dependent RNA polymerase (RdRp) genes were used. If the result of the first sample was negative for SARS-CoV-2, an additional SARS-CoV-2 qRT-PCR assay for another respiratory tract sample from the patient suggested of having COVID-19 was performed to minimize the risk of false-negative results using the qRT-PCR assay [22]. All qRT-PCR confirmed patients with COVID-19 have to be reported to the National Health Command Center and are mandatorily hospitalized in a negativepressure isolation room to prevent the transmission of SARS-CoV-2 in the community. As of 19 July 2020, Taiwan has maintained a record of limited community transmission of COVID-19 and has 455 confirmed cases [23].

Study design and patient enrollment
Six hospitals participated in this retrospective, observational study: National Taiwan University Hospital, National Cheng Kung University Hospital, Tao Yuan General Hospital, Ministry of Health and Welfare, Changhua Christian Hospital, Nantou Hospital, Ministry of Health and Welfare, and China Medical University Hospital. A total of 346 serum samples collected from 74 qRT-PCR confirmed patients with COVID-19 from the six participating hospitals were included for analysis. This study was approved by the institutional review board of all the participating hospitals, and the requirement for informed consent from each patient was waived (202003004RIND).

Collection of serum samples and clinical data
We used the residual blood samples of patients with COVID-19 collected by the attending physicians providing regular medical care. All blood samples were collected on the date of venipuncture. If multiple blood samples were collected from a patient with COVID-19 on the same day, only the first sample collected that day was used for anti-SARS-CoV-2 antibody testing. The serum of the collected blood samples was stored at −20°C before testing.
We further included 194 control serum samples to evaluate the cross-reactivity and diagnostic specificity of the five antibody tests in this study: 70 from hospitalized patients with an acute respiratory infection (ARI) who tested negative ≥2 times using SARS-CoV-2 qRT-PCR and without any other confirmed aetiology for ARI collected in 2020, 50 from patients with ARI who tested negative ≥2 times using SARS-CoV-2 qRT-PCR and presence of specific microbiological aetiologies collected in 2020, 36 from patients who tested positive for any specific auto-antibody collected in 2020, and 38 from patients with pre-COVID-19 sera with presence of specific microbiological antigens or antibodies collected in 2019. Finally, control serum sample of positive result from any of the five serological tests was further examined if the presence of rheumatoid factor (RF) with Siemens N Latex RF Kit (Siemens BN™ II System) to examine the possibility of cross reaction between RF and anti-SARS-CoV-2 antibody [24,25].
Patient data including sex, age, comorbid medical condition, date of symptom onset, initial presentation, date of hospitalization, length of hospital stay, presence of pneumonia on chest roentgenogram, requirement of intensive care unit (ICU) admission, and survival status on hospital discharge were retrieved from electronic medical records of the participating hospitals.

Chemiluminescence immunoassays (CLIA) for the detection of anti-SARS-CoV-2 antibodies
Elecsys® Anti-SARS-CoV-2 is an electrochemiluminescence immunoassay using the recombinant N protein for the detection of antibodies (including IgG) against SARS-CoV-2 with Cobas e immunoassay analyzers (Roche Diagnostics Basel, Switzerland) [15,26]. SARS-CoV-2 IgG, a chemiluminescent microparticle immunoassay, qualitatively detects IgG antibodies to the SARS-CoV-2 nucleocapsid protein (N protein) in human serum and plasma using the ARCHITECT i System (Abbott Laboratories, IL, USA) [14,27]. Test results were interpreted as positive if the electrochemiluminescent signal value of the Roche Test (cutoff index, COI) ≧1.0, or the chemiluminescent signal value of the Abbott Test (index [sample/calibrator], S/C) ≧1.4, as manufacturers' instructions. The detailed information of the two assays is summarized in Table 1. Co., Ltd., China). All the three rapid tests detected either anti-SARS-CoV-2 IgG and IgM antibodies separately (ASK Test and Dynamiker Test) or total antibody (Wondfo Test) within 5-15 min. The viral protein labelled was the SARS-CoV-2 Spike protein (S protein) in the Wondfo and ASK Tests and N protein in the Dynamiker Test. These tests used either the whole blood, serum, or plasma as the testing specimen and required only 10-20 µL of sample volume. Positive results were interpreted as the presence of control line and either IgG or IgM test line for ASK Test and Dynamiker Test, or control line and total antibody test line in Wondfo Test. A weakly positive result (any shade of colour in the test lines) of an antibody rapid testing was considered positive according to the manufacturers' instructions [18,[28][29][30].

Definitions
Test-specific seroconversion in a patient with COVID-19 was defined as the earliest date on which a positive anti-SARS-CoV-2 antibody response was detected in a serum sample using a specific antibody test. The presumptive seroconversion was an ideal approximation of the true biological seroconversion. It was defined as the earliest date on which a positive anti-SARS-CoV-2 antibody response was detected in a serum sample using any one of antibody tests in this study. COVID-19compatible presentation included fever (≥38°C), respiratory tract, constitutional, anosmic, dysgeusic, and diarrheal symptom. Date of symptom onset was defined as the onset date of the first aforementioned COVID-19-compatible symptom reported by a symptomatic COVID-19 patient. Time to seroconversion detection was defined as the duration from the date of symptom onset to the date of seroconversion. The diagnostic sensitivity of an antibody test was defined as the percentage of serum samples from confirmed patients with COVID-19 that is positive for antibodies against SARS-CoV-2, whereas the diagnostic specificity of an antibody test was defined as the percentage of serum samples from control patients that is negative for antibodies against SARS-CoV-2.

Statistical analysis
We calculated means and standard deviations (SDs) for age variables and percentages for the categorical variables. The measurement agreements between different antibody tests were evaluated with Cohen's kappa (к) statistics. The cumulative probabilities of detection of seroconversion for a specific antibody test were obtained using the Kaplan-Meier method. The difference in the cumulative probability of detection of seroconversion between the five antibody tests was evaluated using the log rank test. Further, the difference in cumulative probability of detection of seroconversion between patients with COVID-19 with and without pneumonia was also investigated using the log rank test. Data were analysed using SPSS for Windows (IBM SPSS Statistics v26). All p-values are two-sided, and a p-value less than 0.05 was considered statistically significant.

Clinical characteristics of patients with COVID-19
A total of 346 serum samples were collected from 74 consecutively qRT-PCR-confirmed COVID-19 patients who were treated at six participating hospitals between 23 January 2020 and 31 May 2020, were enrolled in this study. The mean patient age was 38.5 years (SD, 16.2 years). Forty-one (55.4%) patients were men and 67 (90.5%) of them had no significant comorbid or surgical condition. All the 74 enrolled COVID-19 patients reported at least one COVID-19-compatible symptom. Lower respiratory tract symptoms were the predominant symptom at the time of diagnosis (66.2%), followed by upper airway symptoms (62.2%), and fever (45.9%). Twenty-eight (37.8%) patients developed pneumonia during hospitalization, among whom five (6.8%) required ventilator support and intensive care ( Table 2).
Of the 74 patients with COVID-19, 48 patients had sequential serum samples adequate for estimating the date of seroconversion. Clinical and serological data of these 48 patients were used to evaluate the utility of different antibody tests in the early detection of seroconversion after COVID-19 infection. The clinical characteristics of these 48 patients and the other 26 patients who were excluded from seroconversion analysis are compared in Table 2. The date of first serum sample collection from symptom onset was significantly different between these two patient groups (5.7 ± 4.1 vs. 22.0 ± 20.8, p < 0.001).

Anti-SARS-CoV-2 antibody response among patients with COVID-19
A total of 346 serum samples were obtained from the 74 patients with COVID-19 (the number of samples The antibody responses at different time intervals after symptom onset were further evaluated using the five antibody tests (Figure 1). The electrochemiluminescent signal value of the Roche Test (cutoff index, COI) and the chemiluminescent signal value of the Abbott Test (index [sample/calibrator], S/C) at different time intervals after symptom onset were shown in Figure 2(A and B), respectively. Percentage of positive testing result of all the five serological tests and chemiluminescent signal values of the two CLIA serological tests for anti-SARS-CoV-2 antibodies increased as the duration after symptom onset. One 29-year-old woman had negative test results at days 8,11,14,19, and 74 of symptom onset, with all five serological tests. Including serological testing data from this woman, all tests had high diagnostic sensitivity of more than 90% after 21 days of symptom onset. Between-test measurement agreements among the different anti-SARS-CoV-2 antibody tests were further evaluated and are detailed in Table 3. The Wondfo Test had a higher measurement agreement with the ASK Test (к value = 0.830) than with the other three antibody tests in this study. In contrast, the Roche, Abbott, and Dynamiker Tests had a higher measurement agreement (к value ranged from 0.745 to 0.789) than with the Wondfo or ASK Test (к value < 0.683). For the two antibody tests that detected IgM and IgG antibodies separately, there was high measurement agreement for IgM and IgG detection within the Dynamiker Test (к value = 0.919) but not within the ASK Test (к value = 0.334).

Cross-reactivity and diagnostic specificity of the anti-SARS-CoV-2 antibody tests
The results of cross-reactivity analysis of the 194 control serum samples from the patients without COVID-19 with the five antibody tests are detailed in Table 4. Two samples showed reactivity with the Roche Test; both were anti-cytomegalovirus (CMV) antibodypositive serum sample. Four samples showed reactivity with the Abbott Test. Among them 3 serum samples showed anti-CMV IgM/IgG antibodies, and 1 serum sample showed autoantibodies. None of the above six serum samples showing reactivity with     Table S1).

Assessment of antibody dynamic responses to SARS-CoV-2 infection severity
Of clinical and antibody test data from all enrolled patients with COVID-19, data from 48 patients who had sequential serum samples adequate for estimating the date of seroconversion were used to evaluate the ability of early detection of seroconversion among the different anti-SARS-CoV-2 antibody tests and to assess the antibody dynamic response after SARS-CoV-2 infection. Analysis of the ability of early diagnosis of COVID-19 among the five antibody tests was carried out using the Kaplan-Meier test for detection of the cumulative probability of seroconversion ( Figure 3). There was no statistical difference in the results after 7 (p = 0.416), 14 (p = 0.297), 21 (p = 0.554), and 28 days (p = 0.528) of symptom onset on the basis of the log rank test. Twenty of the 48 patients with COVID-19 developed radiographic evidence of pneumonia. Comparison of the time to presumptive seroconversion between patients with COVID-19 with and without pneumonia is shown in Figure 4(A). COVID-19 infected patients with pneumonia showed a shorter seroconversion time than did those without pneumonia (log rank test p = 0.003). Similar results were observed with test-specific seroconversion in the two CLIA anti-SARS-CoV-2 antibody tests (Figure 4(B and C)) and the three LFIA anti-SARS-CoV-2 antibody tests (Supplement Figure S1). Finally, patients with COVID-19 with and without pneumonia showed significant differences in the electrochemiluminescent signal of the Roche Test (COI) (17.76 ± 17.66 vs. 7.28 ± 16.28, p = 0.039) and the chemiluminescent signal of the Abbott Test (S/C) (5.18 ± 2.95 vs. 2.50 ± 2.83, p = 0.003) ( Figure 5).

Discussion
This study parallelly evaluated the performance of five anti-SARS-CoV-2 antibody tests in the diagnosis and severity assessment of COVID-19. There are four major findings of this study. First, the performance of serological tests in the diagnosis of COVID-19 highly depends on the time during the disease course and it stabilized 21 days after symptom onset. Second, although there was no significant difference among the five serological tests in the early detection of seroconversion after COVID-19 development, variations in the diagnostic sensitivity and between-test measurement agreements existed among the five studies serological tests. Third, cross-reactivity with anti-CMV and autoimmune antibodies was observed in anti-SARS-CoV-2 antibody tests using the chemiluminescence method but not in serological tests using LFIA method. Forth, patients with COVID-19 with pneumonia exhibited an earlier anti-SARS-CoV-2 antibody response and a higher post-symptom onset week-2 antibody chemiluminescent signal than did patients with COVID-19 without pneumonia. These findings are important for the appropriate application and interpretation of the results of serological tests by first-line physicians for diagnostic and therapeutic decision making during the COVID-19 pandemic.
Many automated or point-of-care serological tests have been rapidly manufactured to meet the urgent clinical and epidemiological needs to cope with the unprecedented spread and tremendous impact of the COVID-19 pandemic [14,15,17]. An ideal serological test for SARS-CoV-2 should have a high diagnostic sensitivity, low or no cross-reactivity with other existing antibodies, and a high sample throughput to prevent the delay of timely therapeutic decisions due to false-negative results, erroneous assumption of immunity to SARS-CoV-2 due to false-positive result, and facility collapse due to staff burn-out [31]. Previous Table 3. Agreement among findings from the five anti-SARS-CoV-2 antibody tests. IgM studies have reported a high performance of various serological tests after the plateau phase of antibody formation, mostly 3 weeks after symptom onset [11,14,15,18]. Our study, however, found diverse diagnostic sensitivities for the different serological test. The failure of a serological test to detect anti-SARS-CoV-2 antibodies might be the result of the high detection limit of the test, low antibody titre, or a delay in or absence of immune response to SARS-CoV-2 after infection. For example, in this study, one 29-year-old woman with qRT-PCR-confirmed COVID-19 showed multiple negative results for sequential serum samples with all five serological tests till 74 days after symptom onset. Because this woman had no known immunosuppressive disease   and was not under immunosuppressive medication, transient SARS-CoV-2 existence without induction of a measurable antibody response is a plausible explanation for the negative test results. Similar observations have been reported in the recent literature [32]. Therefore, our study demonstrates the effect of individual variations in immunological responses to infection on the performance of serological tests.
Another interesting observation was the measurement agreement grouping phenomenon of the five studied serological tests. In this study, the Roche, Abbott, and Dynamiker Tests had a high betweenmeasurement agreement with each other but not with the Wondfo and ASK Tests. All of these three tests detect nucleocapsid protein-specific antibodies. In contrast, both the Wondfo and ASK Tests detect antibodies targeting the SARS-CoV-2 spike protein and showed high between-measurement agreements among each other but not with the other three tests in this study. Although there was no significant difference in the timing of seroconversion detection, positive percentages of the 346 serum samples were relatively higher in the Wondfo and ASK Tests (77.2% ∼ 79.5%) than in the other three tests (70.5% ∼ 78.0%). This suggests a plausible methodologyspecific effect on the diagnostic sensitivity of a serological test.
Cross-reaction with autoantibodies or antibodies responding to other microorganisms, resulting a false-positive result in a serological test is important and should be comprehensively evaluated for accurate interpretation of test results. Previous studies have reported cross-reactivity with sera containing antibodies positive for CMV, Epstein-Barr virus (EBV), and systemic lupus erythematous in the Roche Test [15,16]. Our study further identified that cross-reactivity occurred not only in the Roche Test but also in the Abbott Test. Of the 6 serum samples from patients without COVID-19 that yielded positive results on the Roche Test or Abbott Test, 3 were anti-CMV IgM/IgG antibody-positive and 1 was autoantibody-positive. The other 2 serum samples, although containing different antibodies, also were positive for anti-CMV antibodies. Cross-reactivity, however, was not observed in serological tests using LFIAs. This is an interesting observation deserved further exploration. Difference in labelling viral protein between serological tests might explain the absence of cross reaction with anti-CMV antibody in Wondfo Test and ASK Test, which use S protein as labelling viral protein. The Dynamiker Test, similar to the Roche Test and Abbott Test, uses N protein as labelling viral protein but also showed no cross reaction with anti-CMV antibody. Because for anti-CMV containing sera with false-positive result, the COI (Roche Test) or S/C (Abbott Test) values were generally low. We therefore suppose that chemiluminescence immunoassay method combining analyzer might be more sensitive than visual lateral flow immunoassay method in the detection of weak cross-reactivity between anti-SARS-CoV-2 and anti-CMV antibody.
On the basis of these observations, positive results of serological tests should be carefully interpreted, especially in communities with a low prevalence and low local transmission rates of COVID-19, like Taiwan [23], and with a high prevalence of CMV infection, as well as in patients with known or possible autoimmune disease [33][34][35]. From the above findings, it can be concluded that although different tests show comparable results in detecting anti-SARS-CoV-2 antibodies, factors such as test-specific methodology, cutoff value determination, biological variation among individual patients, and the presence of cross-reactive antibodies remain important and effect the overall performance of serological tests. As of 19 July 2020, a total of 79,645 people in Taiwan had been tested for SARS-CoV-2 and 455 (0.57%) were confirmed as having COVID-19, with an incidence rate of 19.3 per 1,000,000 people. Among the 455 COVID-19 patients, only 55 (12.1%) had contracted the disease from local transmission. To date, Taiwan has gone >100 days without reporting a single case of local transmission of COVID-19 [23]. Therefore, if a CLIA method (either Roche Test or Abbott Test) was applied for seroprevalence study for mass surveillance or among different risk populations in countries with a low prevalence of COVID-19 like Taiwan, further tests, including LFIAs, evaluation of the presence of anti-CMV antibodies or autoantibodies, the western blotting method, and even qRT-PCR assays for respiratory secretions of enrolled participants with positive results on CLIAs, are needed to confirm or exclude the presence of SARS-CoV-2 RNA or anti-SARS-CoV-2 antibodies [32,36,37].
In our previous study using LFIAs for detecting anti-SARS-CoV-2 antibodies, we demonstrated that patients with COVID-19 with pneumonia exhibited earlier seroconversion than did patients with COVID-19 without pneumonia [18]. In this study, we consolidated this observation from serological tests of LFIA method to tests of CLIA method with a large case number. In addition, we further demonstrated a significant antibody chemiluminescent signal difference between patients with COVID-19 with and without pneumonia. Although designed for qualitative detection of antibody to SARS-CoV-2, our studies found that information on signal number from CLIAs during the second week of symptom onset might be useful for clinicians to alter the risk of subsequent progression at an early phase of COVID-19.
This study has some limitations. First, it was a multicenter study. Though we used a standardized patient reporting form for clinical data collection, investigators from different participating hospitals were involved during data collection, which could have contributed to information bias. Second, as this was a non-protocolized retrospective study, serum samples tested at different post-symptom stages were not standardized; therefore, this study is also subject to information bias due to laboratory data. Third, owing to a small number of cases, significant differences in diagnostic performance between different serological tests may not have been observed. Fourth, concomitant infection with other respiratory tract viral pathogens, especially other coronaviruses, was inadequately evaluated among patients with COVID-19 in this retrospective study. Finally, because plain chest roentgenography is less sensitive than computed tomography in detecting parenchymal changes of viral pneumonia, we could not exclude the possibility that COVID-19 pneumonia patients with subclinical pulmonary infiltration were misclassified as non-pneumonia patients. Due to the above limitations, the validity of our study findings needs to be further confirmed in a prospectively designed diagnostic accuracy study.
In conclusion, in addition to qRT-PCR, serological testing may be a useful tool for the diagnosis of patients with active or past COVID-19. However, the diagnostic sensitivity of serological tests for COVID-19 is highly dependent on antibody dynamics, which reach a stable state after 3 weeks of symptom onset. Furthermore, cross-reaction against anti-CMV antibodies and autoantibody-containing sera was observed when using chemiluminescence immunoassay-based tests, although this was not observed with LFIA. Patients with COVID-19 complicated by pneumonia exhibited earlier anti-SARS-CoV-2 antibody response and a higher antibody chemiluminescent signal than those without pneumonia. Our study findings are informative and provide supportive evidence for the appropriate application and interpretation of serological tests in the diagnosis and management of patients with COVID-19.

Disclosure statement
No potential conflict of interest was reported by the author (s).