点击显示 收起
Fred Hutchinson Cancer Research Center, Seattle, Washington 98109
University of Washington, Seattle, Washington
Duke University Medical Center, Durham, North Carolina 27710
St. Jude Children's Research Hospital, Memphis, Tennessee 38105
ABSTRACT
The Platelia galactomannan enzyme immunoassay is a commercially available nonculture method for diagnosing invasive aspergillosis. Recently, steps have been taken to improve sensitivity; specifically, a low (0.5 to 0.7) galactomannan index (GMI) value to determine positivity has been validated by multiple groups. We evaluated the intra-assay and interassay reproducibility at low index levels using three different kit lots on three different days in three different microbiology laboratories. Clinical and spiked sera were blinded and sent with galactomannan enzyme immunoassay (EIA) kits to the participating laboratories. We also prospectively collected data on all galactomannan EIAs performed between 1 September 2003 and 21 July 2004 at the University of Washington Medical Center microbiology laboratory to assess reproducibility of clinical samples analyzed in "real time." From the multilaboratory study, a total of 836 results were available for evaluation. Reproducibility was excellent between laboratories and on different days. Significant variability was seen between runs/lots, which may in part be associated with different threshold control values in different kits. Among the 1,410 clinical samples that were prospectively analyzed, 168 (90%) were confirmed to be positive on repeat testing (GMI, 0.5). Among the 19 (10.2%) initially positive samples not confirmed on repeat testing, the majority had a GMI at the threshold of the assay (between 0.5 and 0.7). Our findings suggest that the Platelia galactomannan immunoassay has good reproducibility. However, changes in GMI levels when different kit lots are used, and single samples with low-positive (GMI of 0.5 to 0.7) indices, should be interpreted with caution.
INTRODUCTION
Invasive fungal infections are a leading cause of death in immunocompromised patients (13). Aspergillus species are responsible for many of these infection-related deaths, with a mortality rate of approximately 60 to 90% depending on the immunity and underlying disease of the host (4, 10, 14). Poor outcomes are, in part, associated with delayed diagnosis and inadequate or inappropriate early antimicrobial therapies (18).
In the past, the diagnosis of invasive aspergillosis (IA) relied solely on clinical, radiological, and histopathologic examination. Unfortunately, these methods lack both sensitivity and specificity, particularly in the early stages of infection, when the diagnosis is often missed. The Platelia Aspergillus enzyme immunoassay (EIA) (Bio-Rad) detects circulating galactomannan (GM), a major component of the Aspergillus cell wall. It is a double-sandwich EIA that uses a rat monoclonal antibody (EB-A2) to capture the -1,5-galactofuranoside side chain of the GM molecule. In multiple studies, the GM EIA has been shown to be a useful diagnostic test for IA in neutropenic patients with cancer and recipients of hematopoietic stem cell transplants (5, 7, 8, 16); however, estimates of sensitivity and specificity vary, ranging from 50 to 92.6% and 82 to 99.6%, respectively (5, 7, 8, 15, 16).
The GM EIA results are interpreted as an optical density (OD) index, which is the ratio of the sample OD divided by the mean OD of two threshold controls. When the GM EIA was originally marketed in Europe, the recommended cutoff to define a positive test was an index of 1.5. Since then, multiple studies have shown that sensitivity can be increased with minimal loss of specificity by lowering the cutoff to the 0.5 to 0.7 range (3, 6, 9). The U.S. Food and Drug Administration (FDA) cleared the test using an index cutoff of 0.5; another methodological change with FDA approval is the recommendation that the test be repeated on the same specimen to confirm positivity with an index cutoff of 0.5.
The goal of this study was to examine the reproducibility of the Platelia GM EIA, focusing on samples with low-level GM indices (<1.5). A multilaboratory study was conducted to evaluate intra-assay and interassay reproducibility using multiple kit lots tested on multiple days.
MATERIALS AND METHODS
Study design. Three clinical microbiology laboratories participated in the study: the University of Washington Medical Center (UWMC), St. Jude Children's Research Hospital, and Duke University Medical Center. Each laboratory received three different lots of Bio-Rad Laboratories Platelia Aspergillus EIA test kits and the frozen nine-membered panels. Prior to starting the study, proficiency with the GM assay was confirmed at each laboratory by testing a proficiency panel with known OD values. One operator in each laboratory performed all the testing. Each member of the panel was tested in triplicate (in different microplate well locations within each run) on three different days (n = 9) using three different kit lots (n = 27) at the three different laboratories (n = 81). Results, interpreted as index values, were entered into an Excel spreadsheet along with the coded identifiers and sent to the Fred Hutchinson Cancer Research Center (FHCRC) for analysis.
To obtain information regarding reproducibility of clinical samples analyzed "real time" in the UWMC microbiology laboratory, we prospectively collected data on all clinical specimens for which a GM index (GMI) value was determined between 1 September 2003 and 21 July 2004. When a specimen had a GMI value of 0.5, the test was repeated on the same specimen and reported as a "confirmed positive on repeat testing" only if both results yielded a GMI of 0.5. Microbiology laboratory records were reviewed to determine the proportion of tests that were reproducibly positive, relative to the first GMI. For patient specimens not confirmed positive, clinical records were reviewed and patients were given a diagnosis of proven, probable, possible, or no IA according to standardized criteria (1). Informed consent was obtained under an FHCRC Institutional Review Board approved protocol.
Samples. Serum samples were prepared at FHCRC by pooling different samples from patients with known positive GM EIA results in the setting of proven and probable invasive aspergillosis, according to standardized criteria (1). Six panel members (A to F) with target index values ranging from 0.1 to 1.2 were prepared. Samples were pooled to provide sufficient volume for testing multiple times in different laboratories. Three additional panel members (G, H, and I) prepared with purified galactomannan were provided by Bio-Rad Laboratories (Edmonds, WA); the samples had target indices of 0.5, 0.9, and 1.5. The panel members were coded to blind investigators to the target values, frozen at –70°C, and transported to the participating laboratories on dry ice.
GM EIA. The Bio-Rad Laboratories Platelia Aspergillus EIA was performed according to the manufacturer's instructions. Briefly, after thawing, sera were mixed well, 300 μl was added to 100 μl of 4% EDTA treatment solution, and the mixture was boiled for 3 min and centrifuged at 10,000 x g for 10 min. Fifty microliters of horseradish peroxidase-conjugated anti-galactomannan EB-A2 was added to each well, followed by 50 μl of treated supernatant. Microwell plates were incubated at 37°C for 90 min, wells were washed using an automated washer, and mixtures were incubated with 200 μl of tetramethylbenzidine solution for an additional 30 min at room temperature in the dark. Reactions were stopped with 100 μl of 1.5 N sulfuric acid, and ODs were read at 450 and 620 nm. Each plate contained a negative control (index, <0.4), a positive control (index, >2.0), and a threshold control (OD between 0.3 and 0.8) provided in the kit. Results were recorded as an index value relative to the OD of the threshold control (GM index = OD sample/mean OD threshold controls).
Statistical analysis. The reproducibility study design evaluated four main sources of variability in assay results: within lot/run, between lot/run, between days, and between laboratory sites. Data are presented in figures that use box-and-whisker plots to display the average, and dispersion tendencies of the GMI values within the factors varied in this study. The solid box covers the interquartile range, the central line is at the median value, and the lines (whiskers) extend from the 25th and 75th percentiles to two-thirds the width of the interquartile range. Individual points that fall outside the lines are plotted separately (2). The impacts of site, day, and lot/run on the variability of the GMI were evaluated using a nested random effects analysis of variance model. In this model, days are nested within site, lots/runs within a day, and repetitions within lot/run (12). A separate model was fitted for each sample concentration level. Two-sided significances of between-site, between-day, and between-lot/run variability were assessed using F tests, and two-sided P values less than 0.05 were considered significant. These models were also utilized to obtain the variance components due to each of the above-described factors. Using each of these variance component estimates (2) and the mean index value (μ), the percent coefficient of variation (CV%) for each factor was calculated at each index level (CV% = /μ x 100). In order to evaluate whether there were significant sources of variability in the threshold cutoff values, we fitted similar nested random effects models evaluating between-site, between-day, and between-lot/run variability.
RESULTS
Multilaboratory reproducibility. In the multilaboratory reproducibility study, a total of 836 results were available for evaluation: 486 were from the panels prepared from the clinical specimens, 260 were from the panels prepared with spiked purified galactomannan, and 90 were from the kit controls. Due to a deviation from protocol, one laboratory ran sample A three additional times, sample B three fewer times, and sample G five additional times.
Figure 1A demonstrates the variability in test results between sites. In general, variability was low, although it increased for tests with higher indices. Examination of Fig. 1 suggests that one laboratory (site 3) generated more variability than sites 1 and 2; however, the between-site variability was not statistically significant when the variance components and associated significance were estimated from random effects models at each index level between sites. (Table 1). Only sample B, which was not run correctly in one laboratory, was associated with a significant variance component (P < 0.05) between sites. The between-site CV% values were generally low, although they were higher for low index values (4.1 to 24.65%).
Figure 1B demonstrates the variability in test results between days. Again, variability and the CV% values were low overall but more pronounced (although not significant) at the higher range of test results (Table 1).
Figure 1C demonstrates the variability in test results between lots/runs. While variability was low for samples C and G, there was significant variability (P < 0.01) in the test results for the other seven samples. This variability is also demonstrated in Table 1. The between-lot/run CV% values are high (7.6 to 66.7%), particularly for low index values (19.1 to 66.7%).
As within-lot/run variability was equal to the error variability, the statistical significance of this component was not evaluable. However, the variance components and the within-lot/run CV% values were relatively high for the lower index values (Table 1).
Table 1 also demonstrates the variability in threshold control index results. The variability between days was low; however, there was significant between-site variability (P < 0.05). The variance components and CV%s for between-lot/run variability for the threshold controls were also high, but as this variability was equal to the error variability, the statistical significance of this component was not evaluable.
Clinical laboratory experience. To evaluate reproducibility of initial indices reported from the clinical microbiology laboratory, test results from the UWMC were examined. During the time period evaluated, a total of 1,410 tests from patient specimens were performed, and 187 (13.3%) first tests were positive, with a GMI of 0.5. Nineteen (10.2%) samples from 17 patients with an initial positive test were not confirmed on repeat testing. As shown in Fig. 2, the majority of these (89.5%) had an initial GM index between 0.5 and 0.7. Only two tests with initial GM indices 1.0 were negative (<0.5) upon testing in a subsequent run. Of these 17 patients, 13 had clinical data available. A diagnosis of proven, probable, possible, and no IA was made in one, one, three, and eight cases, respectively.
DISCUSSION
Confidence in the reproducibility of the Platelia GM EIA is vital for the interpretation of results in the clinical setting, where the assay may be used not only to facilitate the early diagnosis of IA but also to monitor the course of infection. This study was performed to evaluate the reproducibility of test results, focusing on samples that have low-negative (<0.5) and low-positive (0.5) indices. The results of this study suggest that the assay demonstrates excellent reproducibility between laboratories and between days; however, some variability in results appeared to be associated with the use of different kit lots between runs. Analysis of the "real-time" performance of repeat testing of patient sera also demonstrates some lack of reproducibility when a sample with a GM index from 0.5 to 0.7 is tested in a different run. These results suggest that some caution may be needed when interpreting a single test with an index value between 0.5 and 0.7 as positive.
Test results were generally reproducible between laboratories and between days, even for samples with low index values. However, we did find significant variation between results generated in different runs using different kit lots, at all index values, with particularly high CV%s at low index values. The within-lot/run CV% was also high for these index values. Unfortunately, because of our study design, we were not able to distinguish between interlot and interrun variability. As we did not find a large variance between days, the variability is likely to reflect differences in kit lots. Lot variables that may be different include properties of the monoclonal antibody and, perhaps more likely, OD values of the threshold controls. The relatively high total CV% (19.5%) for the threshold controls, almost all of which represented between-site and between-lot/run variability, supports this hypothesis (Table 1). These results add to the findings of Verweij et al., who demonstrated significant between-run variability using the same kit lot (17). Measurement of serum GM levels over time to monitor the course of aspergillosis is not an FDA-approved indication for this assay; however, there is evidence that favorable outcomes are associated with decreasing GMIs and that poor outcomes are associated with increasing GMIs (8, 9). The interlot/interrun variation seen in this study may be particularly relevant for clinicians when comparing patients' GMIs over time using different kit lots.
With the exception of sample B and the threshold controls, between-laboratory results were reproducible, although one laboratory (site 3) did generate more variability than sites 1 and 2. EIAs are multistepped tests prone to variation and error. Explanations for between-site variability include different equipment and calibration and factors associated with between-run variability: mainly, methodological inconsistencies including the timing of each step, reagent temperature, pipetting errors, and the effectiveness of the washing step. In our reproducibility study, we sought to minimize these factors by ensuring proficiency of the operator before the study was started.
As shown in Table 1, the variance components appeared to increase as the index value increased. With the exception of between-lot/run variability, the increase in variance was not significant. As variance is a function of the mean value, this increase in variance is expected. Indeed, when the variance component is standardized by the mean index value to create the CV%, the highest CV% is seen at the lower index values.
In the clinical study, 10.2% of initially positive tests were not confirmed on subsequent testing. Lack of confirmation was almost exclusively seen when the initial GM index was between 0.5 and 0.7. The majority (85%) of the patients from whom these specimens were drawn did not have IA according to published criteria. A weakly positive GMI may represent one of the following: low fungal burden, cross-reactivity with other serum antigens, gut translocation of galactomannan from dietary sources, and specimen contamination with environmental Aspergillus species. Recently, we have demonstrated that concomitant administration of antifungal agents is associated with lower GMIs and decreased sensitivity of the assay (11). One of 61 (1.6%) tests with high initial GM indices (1.5) was not positive in the confirmity run, suggesting that laboratory contamination was not a large issue during the course of the study. The finding of variability in the low-positive range may reflect between-run variability, to which the clinical laboratory is particularly susceptible, given that different operators perform different test runs. Unfortunately, we do not know how many different kit lots were used to test the samples and the impact of such on the data.
The optimal cutoff for defining a positive result continues to be a subject of investigation and discussion. The current FDA recommendation is to use a cutoff index of 0.5, to confirm an initial positive test by repeating the assay on the same specimen, and to report the result as positive only if the subsequent index is also 0.5. In a recent paper, Maertens et al. demonstrated improved sensitivity (96.5%) and specificity (98.6%) when a dynamic cutoff of two tests with GM indices of 0.5 is employed, compared with sensitivity and specificity results when a static cutoff of a single test with a GM index of 1.5 and 0.5 is used (82.7% and 100%, and 96.5% and 85.1%, respectively) (6). Mindful of the lowered cutoff (0.5), we designed this study to investigate reproducibility at low levels of GM positivity, where the lack of reproducibility might have the greatest impact on diagnostic and treatment decisions. Our data suggest that between-lot/run and within-lot/run variability have an impact on the reproducibly of tests with index values around the low-positive range, and analysis of "real-time" results from the clinical microbiology laboratory confirmed some lack of reproducibility for tests having initial low-positive results (0.5 to 0.7). These results highlight the need for repeat testing of the same sample prior to reporting a positive test result.
There are some study limitations that need to be considered when analyzing our results. First, one site tested samples A and G three and five additional times, respectively, and sample B three fewer times than required by the study protocol; as each sample was tested approximately 81 times, we believe that the impact of this deviation on the final results is negligible. The serum samples used to prepare the panels had been frozen prior to use in this study. Although freezing does not alter the stability of GM (C. Bentsen, personal communication), it might impact the stability of other cross-reactive antigens.
In conclusion, the data from our reproducibility study demonstrate excellent reproducibility of the Platelia GM assay at low levels of GM positivity between laboratories and days. Some variability in results was seen with the use of different kit lots between runs and within runs, particularly at low index values. In the clinical study, the majority of results that were not reproducibly positive had low initial GM positivity with indices between 0.5 and 0.7. These findings indicate good overall reproducibility of the Platelia GM EIA; however, caution should be exercised when making clinical decisions based on changes in a patient's serum GM level when different kit lots are used or when interpreting single samples with indices between 0.5 and 0.7.
ACKNOWLEDGMENTS
This work was supported by grant U01 AI 054736 from the National Institutes of Health. Platelia Aspergillus EIA test kits and GM panel members were donated by Bio-Rad Laboratories, Redmond, WA.
We acknowledge Christopher Bentsen and Lisa McLaughlin (Bio-Rad Laboratories) for assistance with study design and preparation of spiked samples. We also acknowledge the assistance of Bruce Ullness, Gray Metzger, and Tom Novicki (University of Washington).
K.M. has served as a consultant and received grant support from Bio-Rad Laboratories.
Present address: Swedish Family Medicine/Providence Campus, Seattle, WA 98122.
REFERENCES
Ascioglu, S., J. H. Rex, B. De Pauw, J. E. Bennett, J. Bille, F. Crokaert, D. W. Denning, J. P. Donnelly, J. E. Edwards, Z. Erjavec, D. Fiere, O. Lortholary, J. Maertens, J. F. Meis, T. F. Patterson, J. Ritter, D. Selleslag, P. M. Shah, D. A. Stevens, and T. Walsh. 2002. Defining opportunistic invasive fungal infections in immunocompromised patients with cancer and hematopoietic stem cell transplants: an international consensus. Clin. Infect. Dis. 34:7-14.
Cleveland, W. S. 1993. Visualizing data. Hobart Press, Summit, N.J.
Herbrecht, R., V. Letscher-Bru, C. Oprea, B. Lioure, J. Waller, F. Campos, O. Villard, K.-L. Liu, S. Natarajan-Ame, P. Lutz, P. Dufour, J.-P. Bergerat, and E. Candolfi. 2002. Aspergillus galactomannan detection in the diagnosis of invasive aspergillosis in cancer patients. J. Clin. Oncol. 20:1898-1906.
Lin, S.-J., J. Schranz, and S. M. Teutsch. 2001. Aspergillosis case-fatality rate: systematic review of the literature. Clin. Infect. Dis. 32:358-366.
Machetti, M., M. Feasi, N. Mordini, M. T. Van Lint, A. Bacigalupo, J. P. Latge, J. Sarfati, and C. Viscoli. 1998. Comparison of an enzyme immunoassay and a latex agglutination system for the diagnosis of invasive aspergillosis in bone marrow transplant recipients. Bone Marrow Transplant. 21:917-921.
Maertens, J., K. Theunissen, E. Verbeken, K. Lagrou, J. Verhaegen, M. Boogaerts, and J. Van Eldere. 2004. Prospective clinical evaluation of lower cutoffs for galactomannan detection in adult neutropenic cancer patients and haematological stem cell transplant recipients. Br. J. Haematol. 126:852-860.
Maertens, J., J. Verhaegen, H. Demuynck, P. Brock, G. Verhoff, P. Vandenberghe, J. V. Eldere, L. Verbist, and M. Boogaerts. 1999. Autopsy-controlled prospective evaluation of serial screening for circulating galactomannan by a sandwich enzyme-linked immunosorbent assay for hematological patients at risk for invasive aspergillosis. J. Clin. Microbiol. 37:3223-3228.
Maertens, J., J. Verhaegen, K. Lagrou, J. van Eldere, and M. Boogaerts. 2001. Screening for circulating galactomannan as a noninvasive diagnostic tool for invasive aspergillosis in prolonged neutropenic patients and stem cell transplantation recipients: a prospective validation. Blood 97:1604-1610.
Marr, K. A., A. Balajee, L. McLaughlin, M. Tabouret, C. Bensten, and T. Walsh. 2004. Detection of galactomannan antigenemia by enzyme immunoassay for the diagnosis of invasive aspergillosis: variables that affect performance. J. Infect. Dis. 190:631-641.
Marr, K. A., R. A. Carter, M. Boeckh, P. Martin, and L. Corey. 2002. Invasive aspergillosis in allogeneic stem cell transplant recipients: changes in epidemiology and risk factors. Blood 100:4358-4366.
Marr, K. A., M. Laverdiere, A. Gugel, and W. Leisenring. 2005. Antifungal therapy decreases sensitivity of the Aspergillus galactomannan enzyme immunoassay. Clin. Infect. Dis. 40:1762-1769.
Montgomery, D. C. 2001. Design and analysis of experiments. John Wiley & Sons, New York, N.Y.
Ninin, E., N. Milpied, P. Moreau, B. Andre-Richet, N. Morineau, B. Mahe, M. Vigier, B.-M. Imbert, O. Morin, J.-L. Harousseau, and H. Richet. 2001. Longitudinal study of bacterial, viral, and fungal infections in adult recipients of bone marrow transplants. Clin. Infect. Dis. 33:41-47.
Patterson, T. F., W. R. Kirkpatrick, M. White, J. W. Hiemenz, J. R. Wingard, B. Dupont, M. G. Rinaldi, D. A. Stevens, and J. R. Graybill. 2000. Invasive aspergillosis: disease spectrum, treatment practices, and outcomes. Medicine 79:250-260.
Pinel, C., H. Fricker-Hidalgo, B. Lebeau, F. Garban, R. Hamidfar, P. Ambroise-Thomas, and R. Grillot. 2003. Detection of circulating Aspergillus fumigatus galactomannan: value and limits of the Platelia test for diagnosing invasive aspergillosis. J. Clin. Microbiol. 41:2184-2186.
Sulahian, A., F. Boutboul, P. Ribaud, T. Leblanc, C. Lacroix, and F. Derouin. 2001. Value of antigen detection using an enzyme immunoassay in the diagnosis and prediction of invasive aspergillosis in two adult and pediatric hematology units during a 4-year prospective study. Cancer 91:311-318.
Verweij, P. E., Z. Erjavec, W. Sluiters, W. Goessens, M. Rozenberg-Arska, Y. J. Debets-Ossenkopp, H. F. L. Guiot, and J. F. G. M. Mies. 1998. Detection of antigen in sera of patients with invasive aspergillosis: intra- and interlaboratory reproducibility. J. Clin. Microbiol. 36:1612-1616.
Von Eiff, M., N. Roos, R. Schulten, M. Hesse, M. Zuhlsdorf, and J. Van de Loo. 1995. Pulmonary aspergillosis: early diagnosis improves survival. Respiration 62:341-347.