Article Text
PDF
Abstract
Objective This multicentre study was performed to evaluate the diagnostic accuracy of a wide spectrum of novel technologies nowadays available for detection of myeloperoxidase (MPO) and proteinase 3 (PR3)-antineutrophil cytoplasmic antibodies (ANCAs).
Methods Sera (obtained at the time of diagnosis) from 251 patients with ANCA-associated vasculitis (AAV), including granulomatosis with polyangiitis and microscopic polyangiitis, and from 924 disease controls were tested for the presence of cytoplasmic pattern/perinuclear pattern and atypical ANCA (A-ANCA) by indirect immunofluorescence (IIF) (at two sites) and for the presence of PR3-ANCA and MPO-ANCA by eight different immunoassays.
Results The area under the curve (AUC) of the receiver operating characteristic curve to discriminate AAV from controls was 0.923 (95% CI 0.902 to 0.944) and 0.843 (95% CI 0.814 to 0.871) for the two IIF methods. For the antigen-specific immunoassays, the AUC varied between 0.936 (95% CI 0.912 to 0.960) and 0.959 (95% CI 0.941 to 0.976), except for one immunoassay for which the AUC was 0.919 (95% CI 0.892 to 0.945).
Conclusions Our comparison of various ANCA detection methods showed (i) large variability between the two IIF methods tested and (ii) a high diagnostic performance of PR3-ANCA and MPO-ANCA by immunoassay to discriminate AAV from disease controls. Consequently, dual IIF/antigen-specific immunoassay testing of each sample is not necessary for maximal diagnostic accuracy. These results indicate that the current international consensus on ANCA testing for AAV needs revision.
- Systemic vasculitis
- Autoantibodies
- Autoimmune Diseases
Statistics from Altmetric.com
Introduction
For several decades, antineutrophil cytoplasmic antibodies (ANCAs) are recognised as an important laboratory tool in the diagnosis of the small vessel vasculitides, that is, granulomatosis with polyangiitis (GPA), microscopic polyangiitis (MPA) and eosinophilic granulomatosis with polyangiitis.1–3 The multicentre study of Hagen et al4 has paved the way for an international consensus on the appropriate detection of ANCA in patients suspected of ANCA-associated vasculitis (AAV). According to this international consensus, current guidelines dictate that screening for ANCA should be performed by indirect immunofluorescence (IIF) on ethanol-fixed neutrophils and positive results are to be confirmed by ELISA specific for proteinase 3 (PR3) and myeloperoxidase (MPO).5 Indeed, a subsequent meta-analysis revealed that the combination of either the IIF cytoplasmic pattern (C-ANCA) with PR3-ANCA or the IIF perinuclear pattern (P-ANCA) with MPO-ANCA gives a high diagnostic precision for these relatively rare diseases.6
Since the publication of the international consensus on ANCA detection, many new developments in the detection of PR3-ANCA and MPO-ANCA have come to light. Next to the original ELISA, novel solid phase technologies, like addressable laser bead immunoassays,7 ,8 chemiluminescent immunoassays (CLIA),9 fluorescent-enzyme immunoassays (FEIA),10 ,11 line or dot immunoassays 12 and even IIF,13 ,14 have become available. In addition, antigen binding to the solid phase has evolved from direct binding towards binding via a capturing monoclonal antibody15–17 or via a peptide linker,18–20 that is, first-generation, second-generation and third-generation ANCA assays, respectively. Most often these novel assays have been clinically evaluated as an isolated entity, and this obviously hampers comparability due to distinct sample selection, study design and data analysis.
Due to the consecutive technical innovation in ANCA diagnostics, it is questioned if screening by IIF still is the most optimal approach.21 This discussion continued in 2009 with a debate during the 14th Vasculitis and ANCA workshop in Copenhagen.22 Thus far, the only study that challenged the position of IIF in the ANCA testing algorithm revealed that screening by antigen-specific ANCA is as effective as the algorithm proposed in the international consensus.23 Nevertheless, ANCA IIF is increasingly discarded completely in many clinical laboratories (Damoiseaux, manuscript in preparation).
In the current multicentre study, we evaluated the diagnostic accuracy of a wide spectrum of novel technologies nowadays available for detection of MPO-ANCA and PR3-ANCA. For this study, diagnostic samples were included from patients with AAV, that is, GPA and MPA, as well as from patients suspected of, but eventually not having AAV. The results are compared with two state-of-the-art ANCA IIF analyses, one based only on ethanol-fixed neutrophils and the other based on the combination of ethanol-fixed neutrophils, formalin-fixed neutrophils and HEp-2 cells.3 The data set obtained should enable to draw a firm conclusion about the role of ANCA IIF in the diagnostic work-up of AAV and is intended to be the basis of a novel international consensus on ANCA testing.
Patients and methods
This international study was performed in a large cohort of patients with AAV and controls with inflammatory disease. Serum samples were recruited at four different sites: Klinikum Bad Bramstedt (Germany), Statens Serum Institute Copenhagen (Denmark), University Hospitals Leuven (Belgium) and Maastricht University Medical Center (The Netherlands). Online supplementary data gives an overview of the number of patients with AAV and controls recruited at each site (see online supplementary table S1).
Supplemental material
The study group included patients with GPA (n=186) and MPA (n=65) and disease controls (n=924). Patients with eosinophilic GPA were not included because this constitutes a heterogeneous group.24 All samples included were obtained from patients visiting a university hospital clinic and had a diagnosis ascribed as a consequence of that visit. All samples included were diagnostic samples and, therefore, most patients did not receive any immunosuppressive therapy at the time of sampling. In a minority of patients, however, immunosuppressive therapy was already started before the samples were obtained. GPA patients fulfilled the American College of Rheumatology classification criteria25 and the Chapel Hill Consensus Definitions.26 ,27 The diagnosis of MPA was based on Chapel Hill Consensus Definitions.26 ,27 The distribution of organ involvement in patients with AAV is summarised in online supplementary table S2. Renal involvement was present in 45% of patients with GPA and in 60% of patients with MPA.
The disease controls recruited in Copenhagen, Leuven and Maastricht (n=735) were consecutive patients who visited the respective hospital and for whom the medical doctor considered it important to request ANCA. Afterwards, a diagnosis of ANCA-associated vasculitis was excluded. Patients in whom inflammatory bowel disease (IBD) and/or autoimmune liver disease was considered were excluded. In patients clinically suspected of IBD, ANCA by IIF is helpful to differentiate IBD from non-IBD and ulcerative colitis from Crohn's disease when combined with other serological markers such as anti-saccharomyces cerevisiae antibodies (ASCA).28 The disease controls in Bad Bramstedt included cohorts of patients with systemic lupus erythematosus (SLE) (n=59), rheumatoid arthritis (n=89), systemic sclerosis (n=11) and Sjögren's syndrome (n=30). Online supplementary table S3 summarises the medical conditions in the control population.
This study was carried out according to the 1997 Declaration of Helsinki of the World Medical Association and has been approved by the ethics committee of each participating centre.
ANCA detection methods
Two different IIF approaches were used for ANCA detection. The standard IIF using an ethanol-fixed mixture of neutrophils and lymphocytes as previously described29 was performed in the Staten Serum Institut (Copenhagen, Denmark). In Bad Bramstedt (Germany), IIF testing was performed on ethanol-fixed neutrophils in combination with additional tests on formalin-fixed neutrophils and HEp-2 cells to better discriminate between P-ANCA (or atypical-ANCA) and antinuclear antibody (ANA), as previously described.3
Seven commercial manufacturers of antigen-specific ANCA immunoassays agreed to participate in this study. ANCA assays were performed by each participating company. Bio-Rad Laboratories (Hercules, California, USA) used an automated multiplex flow immunoassay (BioPlex 2200 Vasculitis kit) for semiquantitative detection of IgG antibodies to MPO, PR3 and Glomerular Basement Membrane. Euro-Diagnostica AB (Malmö, Sweden) used a second-generation capture PR3-ANCA and MPO-ANCA ELISA.16 Euroimmun AG (Lübeck, Germany) detected PR3-ANCA by a third-generation anti-PR3-hn-hr-ELISA,19 and MPO-ANCA by a first-generation anti-MPO ELISA. Medipan/Generic Assays GmbH (Berlin, Germany) used CytoBead ANCA assays.14 Inova Diagnostics (San Diego, California, USA) used QuantaLite and QuantaFlash assays, being ELISA and CLIA, respectively.9 Thermo-Fisher Scientific (Waltham, Massachusetts, USA) used third-generation EliA PR3S and EliA MPOS fully automated FEIA. Orgentec (Mainz, Germany) used third-generation anti-PR3 hs ELISA and first-generation anti-MPO ELISA.18 The tests were performed according to the manufacturer's instructions and results were expressed in the individual kit units.
All serum samples were encoded and distributed to the participating manufacturers. After receiving all the data, the code was de-blinded at the study centre (Leuven) to reveal the clinical information on the diagnoses.
Statistical analysis
Receiver operating characteristic (ROC) curve analysis was performed using Analyse-it for Microsoft Excel V.3.90. A comparison of the area under the curve (AUC) was done using the method of De Long (Analyse-it).
Results
An overview of ANCA by IIF performed at two different laboratories and of PR3-ANCA and MPO-ANCA by eight different commercially available immunoassays (from seven different companies) is summarised in table 1.
Overview of the results for proteinase 3 (PR3)-antineutrophil cytoplasmic antibodies (ANCA) and myeloperoxidase (MPO)-ANCA obtained by immunoassays from different commercial sources1–8 and for cytoplasmic pattern ANCA (C-ANCA) and perinuclear pattern ANCA (P-ANCA) by indirect immunofluorescence (IIF) performed at two sites (Copenhagen (9) and Bad Bramstedt (10))
Of the 924 controls, 725 (78%) and 866 (94%) tested negative by IIF in Copenhagen (C) and Bad Bramstedt (BB), respectively, whereas 872–904 (94–98%) tested negative by the distinct antigen-specific immunoassays, reflecting a higher specificity of the antigen-specific immunoassays compared with IIF. For all immunoassays tested, the specificity of PR3-ANCA (98–99%) was higher than the specificity of MPO-ANCA (96–99%). For IIF, the specificity of C-ANCA (97–98%) was higher than the specificity of P-ANCA (81–96%).
Of the 186 patients with GPA, IIF revealed C-ANCA in 119 (64%; C) and 144 (78%; BB) patients and P-ANCA in 27 (14%; C) and 20 (11%; BB) patients, whereas the distinct antigen-specific immunoassays revealed PR3-ANCA in 140–148 patients (75–80%) and MPO-ANCA in 15–22 patients (8–12%). Double positivity (PR3-ANCA and MPO-ANCA) was found in 0–4 patients (0–2%). In total, 20–28 patients with GPA (11–15%) tested negative for both PR3-ANCA and MPO-ANCA.
Of the 65 patients with MPA, IIF revealed C-ANCA in 3 (5%; C) and 4 (6%; BB) patients and P-ANCA in 58 (89%; C) and 55 (85%; BB) patients, whereas the distinct antigen-specific immunoassays revealed PR3-ANCA in 2–4 patients (3–6%) and MPO-ANCA in 44–56 patients (68–86%). Double positivity (PR3-ANCA and MPO-ANCA) was found in 0–2 patients (0–3%). In total, 5–16 patients with MPA (8–24%) tested negative for both PR3-ANCA and MPO-ANCA.
In summary, for IIF, clear differences were found between the two laboratories, with one of the two laboratories performing at a higher specificity as well as at a higher sensitivity for GPA (table 1). For the antigen-specific immunoassays, the differences were less pronounced. In order to compare the different antigen-specific immunoassays, we calculated the accuracy, defined as the overall fraction of controls and patients that tested, respectively, negative and positive. For all antigen-specific immunoassays, except for the CytoBead assay (Medipan), the accuracy ranged between 0.944 and 0.954 (table 1). The accuracy for the CytoBead assay was 0.921. This assay displayed the lowest specificity and lowest sensitivity for MPO-ANCA. For comparison, the accuracy of the two IIF analyses was 0.794 (C) and 0.9270 (BB). Thus, the observed differences between the different antigen-specific immunoassays (with the exception of the CytoBead assay) are likely to be related to differences in the company-specific cut-offs. For example, when compared with the BioPlex 2200 assay from BioRad the Euro-Diagnostica capture ELISA had a lower sensitivity for GPA but a higher specificity (table 1).
Further investigation of the diagnostic performance characteristics of IIF and antigen-specific immunoassays was done by ROC curve analysis. In order to perform such analysis, patients with GPA and MPA were clustered as AAV (n=251). Thus, ROC curve analysis was used to evaluate the performance of IIF and antigen-specific immunoassays to distinguish AAV from disease controls (n=924). For IIF, the highest level of reactivity, independent of ANCA pattern, was selected for analysis. Similarly, for antigen-specific immunoassays the highest level of reactivity from the PR3-ANCA and MPO-ANCA determinations was selected for analysis. This was feasible as all manufacturers, except for two, apply similar cut-off values for PR3-ANCA and MPO-ANCA. For the two manufacturers that do not apply the same cut-off for PR3-ANCA and MPO-ANCA, the differences were minor, that is, 3 versus 5 IU/mL for FEIA and 5 versus 10 U/mL for the Orgentec ELISA. The results are shown in figure 1 and table 2. Marked differences between the AUCs were found for the two IIF analyses performed in different laboratories (AUC 0.923 (95% CI 0.902 to 0.944; BB) vs 0.843 (95% CI 0.815 to 0.871; C) (p<0.0001)). The AUC for the CytoBead assay was 0.919 (95% CI 0.892 to 0.945), whereas the AUCs for the seven other antigen-specific immunoassays varied between 0.936 (95% CI 0.912 to 0.960) and 0.959 (95% CI 0.941 to 0.976), which was statistically significantly higher than the AUC for IIF-C (p<0.0001 for all assays) and the AUC for IIF-BB, except for EliA (p=0.19). A breakdown of the data for C-ANCA and PR3-ANCA in GPA and P-ANCA and MPO-ANCA in MPA revealed that the CytoBead MPO-ANCA, but not PR3-ANCA, had a lower AUC than the AUC of the other immunoassays (see online supplementary table S4, S5 and figure S1, S2).
Area under the curve (AUC) for eight antigen-specific immunoassays1–8 and two indirect immunofluorescence (IIF) methods performed at two sites (Copenhagen (9) and Bad Bramstedt (10)) for antineutrophil cytoplasmic antibodies (ANCA) detection
Receiver operating characteristics curve for eight antigen-specific immunoassays and two indirect immunofluorescence (IIF) methods (Copenhagen and Bad Bramstedt) for antineutrophil cytoplasmic antibodies (ANCA) detection. For antigen-specific immunoassays, the highest level of reactivity from the proteinase 3-ANCA and myeloperoxidase-ANCA determinations was selected for analysis. For IIF, the highest level of reactivity, independent of pattern, was selected for analysis. The immunoassays included were from Inova (QuantaLite (1) and QuantaFlash (2)), Thermo-Fisher (EliA) (3), Bio-Rad (BioPlex 2200) (4), Euro-Diagnostica (5), Orgentec (6), Euroimmun (7) and Medipan (CytoBead assay) (8). IIF was performed at Copenhagen (9) and Bad Bramstedt (10).
In summary, ROC curve analysis revealed marked differences between IIF results obtained in different laboratories and showed that most of the antigen-specific immunoassays outperformed IIF for ANCA determination.
Next, we investigated whether IIF detected antibodies that were missed by antigen-specific immunoassays and vice versa. The results for comparison of the best-performing IIF (BB) analysis with the Euroimmun assays (as an example) are summarised in table 3. IIF detected antibodies in three patients with GPA (two with limited disease and one with low-disease activity) that were negative by antigen-specific immunoassays, whereas antigen-specific immunoassays detected antibodies in four patients with GPA that were negative by IIF. It should be mentioned that the respective antibody levels were low (three weak positive (dubious) IIF results; PR3-ANCA 23 and 76 U/mL (cut-off 20 U/mL); MPO-ANCA 27 and 32 U/mL (cut-off 20 U/mL)). Similarly, for MPA one patient was negative by IIF but positive by antigen-specific immunoassays, whereas another patient was negative by antigen-specific immunoassays and positive by IIF.
Concordance between indirect immunofluorescence (IIF) (Bad Bramstedt) and enzyme immunoassay
There were 12 patients with GPA and 4 patients with MPA who were negative for PR3-ANCA and MPO-ANCA by all eight enzyme immunoassays. All patients with GPA had limited disease: ear, nose and throat (ENT) or locoregional involvement (n=9), ENT and lung involvement (n=1), ENT and polyneuropathy (n=1), and ENT and arthritis (n=1). The organ involvement for the four seronegative patients with MPA was lung (n=1), nervous system (n=2), and lung and kidney (n=1). A summary of the controls in whom PR3-ANCA and/or MPO-ANCA was found with at least seven of eight assays is given beneath. A patient with ANCA-associated drug-induced (propylthiouracyl) vasculitis had PR3-ANCA (positive with all eight assays) and MPO-ANCA (positive with 7/8 assays). A control patient with polyarteritis nodosa and a control patient with Behcet's disease had PR3-ANCA (detected by 7/8 assays). In the patient with Behcet's disease, MPO-ANCA was also positive with 4/8 assays. Five control patients had MPO-ANCA detected by 8/8 assays. The clinical conditions of these patients were rheumatoid arthritis (n=3), unclassified systemic small vessel vasculitis (n=1) and SLE (n=1). The patient with SLE also had PR3-ANCA detected by 6/8 assays. Three control patients had MPO-ANCA detected by 7/8 assays. The clinical conditions of these patients were infection, arthralgia and SLE.
Table 4 shows the concordance between all antigen-specific immunoassays for PR3-ANCA and MPO-ANCA in controls, GPA and MPA. The concordance for PR3-ANCA between all assays was 94%, 89% and 91% in controls, GPA and MPA, respectively. For MPO-ANCA, the concordance between all assays was 94%, 93% and 77% in controls, GPA and MPA, respectively. Because the CytoBead MPO-ANCA assay (Medipan) had the lowest accuracy and AUC, we recalculated the concordance after exclusion of the CytoBead assays. After exclusion, the concordance for PR3-ANCA was almost similar as before exclusion, that is, 96%, 89% and 91% in controls, GPA and MPA, respectively, while the concordance for MPO-ANCA indeed increased to 96%, 96% and 85% in controls, GPA and MPA, respectively. Although the concordance between the different antigen-specific immunoassays is high, it is not absolute and differences between antigen-specific immunoassays do exist (further exemplified in online supplementary table S6).
Concordance between immunoassays
Discussion
We evaluated IIF and various antigen-specific immunoassays for detection of ANCA as a diagnostic marker for AAV, that is, GPA and MPA. Overall, the diagnostic performance, as evaluated by ROC curve analysis, of the antigen-specific immunoassays equalled or surpassed the diagnostic performance of IIF.
For IIF, two approaches were used. The first approach was the standard IIF method based on an ethanol-fixed mixture of granulocytes and lymphocytes as described by Wiik et al29 and performed at the Staten Serum Institute (where the method was developed). This standard IIF method was also the basis for the multicentre study of Hagen et al4 that formed the basis for the current international consensus on detection of ANCA.5 In our study, the specificity of this method was 97% for C-ANCA and 81% for P-ANCA, which was comparable to the specificity in disease controls reported by Hagen et al4 (95% for C-ANCA and 81% for P-ANCA). The sensitivity of C-ANCA for GPA in our study (63%) was comparable to the sensitivity reported by Hagen et al (64%),4 whereas the sensitivity of P-ANCA for MPA in our study (89%) was higher than the sensitivity reported by Hagen et al (58%).4 The second approach, performed at Bad Bramstedt, combined ethanol-fixed neutrophils with formalin-fixed neutrophils and antinuclear antibody testing on HEp-2 cells in order to increase the specificity of P-ANCA.3 Our results confirm the higher specificity of the Bad Bramstedt approach compared with the Copenhagen approach. Unexpectedly, we also found a higher sensitivity of the Bad Bramstedt approach compared with the Copenhagen approach, especially in GPA. The reason for this discrepancy is unclear and can be related to differences in source of cells, substrate, conjugates and fixation methods used and expertise of the observer. Taken together, our results show significant differences between the performance of IIF determinations executed at two different expert laboratories, thereby underscoring the high variability in results obtained by IIF in different laboratories. In a parallel study, we evaluated the performance of automated multiparameter IIF devices (Aklides and EuroPattern).14 ,30–32 In that study, we also observed variability in IIF results between automated devices.33
The antigen-specific immunoassays evaluated included first-generation, second-generation (capture-based) (Euro-Diagnostica) and third-generation (anchor-based) (Thermo-Fisher, Euroimmun, Orgentec for PR3-ANCA and Thermo-Fisher for MPO-ANCA) assays. The technology included classical ELISA (Inova QuantaLite, Euro-Diagnostica, Euroimmun, Orgentec), automated FEIA (Thermo-Fisher), chemiluminescence assay (Inova QuantaFlash), multiplexed flow immunoassay (BioRad) and multiplexed microbead IIF assay (Medipan). Our results did not reveal consistent differences between the different assay formats or assay generations, except for the CytoBead MPO-ANCA (Medipan) that performed less well compared with all other tested MPO-ANCA assays. This is in contrast with the literature that assumes a better sensitivity of second-generation and third-generation immunoassays and a better specificity for second-generation immunoassays (refs 34–36, reviewed in ref. 3). It should be mentioned that our study was performed mainly with diagnostic samples obtained from patients that did not receive any immunosuppressive treatment. Epitope spreading and affinity maturation can occur during the course of disease and could differentially affect reactivity with the different assay formats or assay generations in follow-up samples.
Today, mounting evidence suggests that AAV should be classified based on the ANCA serotype as PR3-ANCA and MPO-ANCA disease. Recent studies have shown that PR3-ANCA and MPO-ANCA diseases are strongly associated with distinguishable genetic risk alleles, phenotypic differences and differences in risk of relapse.37–40 Therefore, these data suggest that perhaps a new classification is required centred around the antigen specificity of ANCA and strongly argue for the ANCA screening only by antigen-specific assays.
ANCAs are helpful in the diagnosis of AAV, but their use as a diagnostic biomarker is insufficient. The diagnosis is based on clinicopathological features. Our study confirms that a fraction of patients with AAV is ANCA negative; depending on the assay, 11–17% of patients with AAV were negative by IIF and 9–16% by immunoassay. Therefore, a diagnosis of AAV cannot be excluded for ANCA-negative patients. Renal and/or other biopsies should be performed in these patients. When the patient tests negative for ANCA, however, the result could be a false negative due to reactivity to epitopes that are not seen in most ANCA assays.41 Patients with ANCA-negative GPA are likely to have localised (limited) disease.42
The strength of this study is that a large multicentre consecutively recruited patient cohort with diagnostic samples was included, as well as a large number of relevant disease controls. The same cohort was analysed in a blinded fashion with a multitude of assays, which allowed direct comparison of the different assays. A weakness of the study is that not all companies could be included due to the limited sample volume that was available.
In conclusion, screening for ANCA with IIF is not of added value when using high-quality antigen-specific immunoassays. This only holds for AAV and not for IBD or autoimmune liver diseases. Our findings warrant revision of the international consensus on screening for ANCA in the diagnostic work-up of GPA and MPA.
Acknowledgments
We thank BioRad Laboratories, Euro-Diagnostica AB, Euroimmun, Medipan/Generic assays GmbH, Inova Diagnostics, Orgentec and Thermo-Fisher for supporting the study.
References
Footnotes
Handling editor Tore K Kvien
Contributors JD and EC share first authorship. XB, JD, EC, NR and J-WCT designed the study. EC and NR performed indirect immunofluorescence analyses. NR, FM, PvP, BB, J-WCT, PV and DB gathered the clinical data. XB, JD, NR and EC gathered the laboratory data and XB, JD and EC analysed the data. JD, EC, J-WCT and XB wrote the manuscript. NR and DB helped editing. All authors agreed to publish the paper.
Competing interests XB has been a member of the ‘Scientific Advisory Committee’ for INOVA Diagnostics. XB has received a lecture fee from Thermo-Fisher.
Ethics approval Ethics committee of the University Hospitals Leuven, Maastricht University Medical Center, Klinikum Bad Bramstedt, Statens Serum Institute Copenhagen.
Provenance and peer review Not commissioned; externally peer reviewed.