点击显示 收起
1 Department of Paediatrics, Christchurch School of Medicine and Health Sciences, Christchurch, New Zealand
2 Australian and New Zealand Neonatal Network (ANZNN), Centre for Perinatal Health Services Research, University of Sydney, NSW, Australia
3 School of Public Health, University of Sydney, NSW, Australia
4 Department of Neonatal Medicine, Royal Prince Alfred Hospital, Sydney, NSW 2050, Australia
ABSTRACT
Aim: To analyse variations in rates of severe retinopathy of prematurity (ROP) among neonatal intensive care units (NICUs) in the Australian and New Zealand Neonatal Network (ANZNN), adjusting for sampling variability and for case mix.
Methods: 25 NICUs were included in the study of 2105 infants born at less than 29 weeks in 1998 and 1999, who survived to 36 weeks post-menstrual age and were examined for ROP. The observed NICU rates of severe ROP were adjusted for case mix using logistic regression on gestation, weight for gestational age and sex, and for sampling variability using shrinkage estimates. The corrected rate in the best 20% of NICUs was identified and NICU variations in rates were compared with those in 2000–1.
Results: The overall (unadjusted) rate of severe ROP in the NICUs was 9.6% (interquartile range 5.4–12.8%). After adjusting for both case mix and sampling variability there remained significant variation among the NICUs. 20% of NICUs had a rate of severe ROP 5.9%. Variation in rates among NICUs showed a similar pattern in both time periods. If the overall network rate was reduced to 5.9%, the 20th centile of the adjusted rates, there would be 79 fewer cases in a 2 year period, in contrast with 26 fewer if rates in the two units with excess rates improved to the average.
Conclusions: Considerable variation in rates of severe ROP among NICUs remained after adjustment for case mix and sampling variability. These data will facilitate investigation of potentially better practices associated with a reduced risk of severe ROP.
Abbreviations: ANZNN, Australian and New Zealand Neonatal Network; CRIB, Clinical Risk Index for Babies; GA, gestational age; IQR, interquartile range; NICU, neonatal intensive care unit; PMA, post-menstrual age; ROP, retinopathy of prematurity; SNAP-PE, Score for Neonatal Acute Physiology
Keywords: retinopathy of prematurity; premature infants; quality improvement
Since the first description of retrolental fibroplasia by Terry in 1942, a notable feature of the condition (now known as retinopathy of prematurity, ROP) has been the variation in incidence among hospitals.1 Recent hospital cohorts2,3 and population based reports4–8 continue to demonstrate widespread variability in reported incidence and severity of ROP.
Such variation in outcome, across different specialties and disease entities, has been the subject of considerable debate. Differences in outcome ascertainment, case mix, and sampling variability, as well as different clinical practices may all contribute to the observed differences.9 In order to be able to identify NICUs with significantly better outcomes, and thus investigate potentially better practices associated with those outcomes, it is necessary to control for these sources of variation. No previous study has taken this approach to reduction in the incidence of severe ROP.
The Australia and New Zealand Neonatal Network (ANZNN) consists of all 29 level III neonatal intensive care units (NICUs) in Australia and New Zealand. Since 1995 a dataset of 60 perinatal variables, using agreed definitions, has been collected prospectively on all infants of less than 32 weeks gestation or birth weight less than 1500 g. Both countries have a regionalised system of neonatal intensive care, meaning that the ANZNN dataset for infants with gestation less than 29 weeks represents essentially population based data. Previously, we analysed data on 21 antenatal and intrapartum variables for all infants <29 weeks gestation born in the 2 year period 1998–9 and who survived to 36 weeks post-menstrual age (PMA) to identify risk factors for severe (stage 3 or more10) ROP.11 That study showed that decreasing gestation, lower birth weight for gestation, and male sex were significantly associated with increased risk.11
We have now analysed the differences in severe ROP among NICUs of the ANZNN, adjusting for differences in case mix and sampling variability, using our previously reported methodology.9 Our aim was to identify true variations in severe ROP in order to provide a framework for reducing the overall network incidence of severe ROP.
METHODS
The four children’s hospitals out of the 29 NICUs in the ANZNN were excluded from the study since they represent a different population of infants (all outborn and many requiring surgery). The remaining 25 NICUs included 2830 infants born at <29 weeks in 1998–9, of whom 2286 (80.8%) survived to 36 weeks PMA. The 181 (7.9%) infants who did not have an eye examination reported were excluded, leaving 2105 eligible infants.
To analyse variation in outcome rates among NICUs we compared the observed frequency (O) of severe ROP in each unit with the expected frequency (E).9 The unadjusted value of E for each NICU is simply the overall rate of severe ROP for all NICUs multiplied by the number of eligible admissions to that NICU. To adjust for bias caused by case mix, we used the logistic regression model to predict the probability of severe ROP for each infant from its gestation, weight for gestational age and sex.11 The adjusted value of E for each NICU is the sum of the predicted probabilities for all infants in that NICU.
We also allowed for the effect of sampling variability, which is smaller when the outcome rate is closer to 0% or to 100%, assuming the rates have a binomial distribution. It is also smaller for larger NICUs, because the standard error of the estimated rate decreases as sample size increases. Because of sampling variability the observed rate varies about the true rate for each NICU. We allowed for this by calculating shrinkage estimators using a Bayesian approach. Details are given elsewhere9 but, briefly, each NICU is assumed to have a true underlying rate of severe ROP. Shrinkage estimators use data from all NICUs to improve the estimate of the true rate of severe ROP for each individual NICU, by shrinking the observed rate towards the overall rate for all units. The shrinkage is greater for smaller NICUs, so shrinkage estimates are more conservative and less variable than the observed rates. The shrinkage estimates were obtained using maximum likelihood methods assuming a gamma Poisson model, which also gave an estimate, v, of the standard deviation of the true incidence ratios (O/E) for all NICUs.12
The corrected excess (O – E) number of infants with severe ROP was calculated for each NICU using the shrunken estimators. The 95% control limits were obtained similarly, to show the limits within which this excess should lie when the expected number of cases is E.9
To inform quality improvement, we explored the benefit of a network-wide approach rather than targeting NICUs performing poorly. The 20th centile was estimated as the (n+1)/5th value of the adjusted rates of severe ROP for all NICUs sorted in ascending order. This is used as an achievable estimate of "best practice," because it is already achieved by 20% of NICUs. The "centile gain" is the number of potential cases of severe ROP that could be prevented in a 2 year period if the overall NICU rate was reduced to the 20th centile rate. This was compared with the "outlier gain," the reduction in the number of cases that would result if NICUs that were exceeding their 95% limits improved to their expected number of cases.
The same methods were applied separately to data for a similar cohort born in 2000–1 (that is, infants of <29 weeks gestation, registered with ANZNN in 2000–1, who survived to 36 weeks PMA and were examined for ROP) for comparison (n = 2277) (full data for 2000–1 not shown).
We also explored the potential measurement bias resulting from the failure to examine eligible infants. Kendall’s rank correlation coefficient was used to assess the relations between the rate of severe ROP and the observed mortality rate (that is, the proportion of infants of <29 weeks gestation who died before 36 weeks PMA), the number of infants of <29 weeks gestation alive at 36 weeks PMA, and the percentage of these eligible infants not examined for ROP per NICU. In addition, sensitivity analysis was performed using best and worst case scenarios (worst case treating babies not examined for ROP as cases of severe ROP, best case as no ROP11).
Lastly, the NICUs were surveyed on their clinical practice with respect to indirect ophthalmoscopy. All NICUs in the ANZNN confirmed that examination guidelines13,14 with respect to timing and technique, including adequate pupil dilatation, were followed and that infants are screened until the retina is fully or nearly fully (extreme zone 3) vascularised. Examination findings are reported using the International Classification of ROP.10
Approval for the project was given by Royal Prince Alfred Hospital ethics review committee and University of Sydney human ethics committee.
RESULTS
In 1998–9 ROP of any stage was reported in 42% of examined infants, stage 3 or more in 9.6%, and 57% of the latter received treatment by laser or cryotherapy. The unadjusted rates of severe ROP for individual NICUs ranged from 0% to 25% (column 4, table 1), interquartile range (IQR) 5.4–12.8%. The effects of both the shrinkage and case mix adjustment are shown in table 1. Adjusting for sampling variability alone (column 6) has a relatively minor impact on the order of the NICUs (except for "H"). Adjusting for case mix has a more marked effect on the order, most notably for "P" and "T." The relative impact of the adjustments is best demonstrated graphically. Figure 1 compares the observed rate of severe ROP with the rates adjusted for (a) case mix only and (b) both case mix and sampling variability (corrected rates).
Table 1 Rates of severe ROP in infants born at <29 weeks gestational age who survived to 36 weeks PMA and had eyes examined, for 25 ANZNN NICUs (1998–9)
Figure 1 Comparison of observed percentage of cases with severe ROP in each NICU (the NICUs are identified by letter on the X-axis, as in table 1) by decreasing number of eligible infants (1998–9) and (A) percentage adjusted for case mix only. (B) percentage adjusted for case mix and sampling variability.
Figure 2 shows the number of infants involved in this variation by depicting the corrected excess (O – E) number of severe ROP cases for each NICU in 1998–9 (black diamonds), with the NICUs ordered by decreasing number of examined infants and the upper and lower 95% control limits shown. NICU "X" had almost 20 more cases of severe ROP than expected over the 2 year period. NICU "Y" had six excess cases and NICU "B" had five fewer cases of severe ROP than expected.
Figure 2 Excess number of cases of severe ROP (O – E) by NICU, calculated using estimates adjusted for both case mix and sampling variability, for 1998–9 and 2000–1 cohorts. NICUs are ordered by decreasing number of eligible cases in 1998–9. Upper and lower 95% control limits are shown for each cohort. (The NICUs are identified by letter on the X-axis, as in table 1).
For infants in the 2000–1 cohort, the overall rate of severe ROP was 10.2% (60% of these infants receiving treatment) and the variability among NICUs increased slightly (fig 2, open squares). There were an additional two NICUs ("O" and "G") with fewer than expected cases and another two NICUs ("R" and "N") with more than expected. Individual unit performance for this outcome can be easily compared over the two time periods using figure 2 because the NICUs are in the same order. The NICU with almost 20 more cases than expected in 1998–9 had 25 excess cases in 2000–1.
Estimates of the variation (v) among the NICUs’ true incidence ratios are given in table 2 for three different scenarios: with adjustment for none, two, and three predictor variables. The variability among NICUs actually increased as we adjusted for more variables. Adjusting for the three predictors (gestational age, birth weight for gestational age, and sex), the best 20% of NICUs were estimated to have a rate of 5.9% or less while the overall rate in this period was 9.6%. If the network rate was reduced to 5.9%, there would be 79 fewer cases of severe ROP over a 2 year period ("centile gain"), in contrast with only 26 fewer cases by taking the "outlier gain" approach—that is, by improving the outcome in the two NICUs that were in excess of their 95% limits.
Table 2 Estimates of variation (v) in severe ROP rates among 25 ANZNN NICUs (1998-9), based on adjustment for 0, 2, and 3 predictor variables (gestational age (GA), weight for gestational age, sex)
There was no relation between the observed rate of severe ROP and a NICU’s mortality rate before 36 weeks PMA (Kendall’s rank correlation, T = 0.10, p = 0.47), or size of NICU, as judged by the number of eligible infants (T = 0.04, p = 0.78). Measurement bias could also result from excluding those infants not examined for ROP, with the potential for NICUs with high rates of "not examined" to have low rates of severe ROP. Figure 3 displays these data with the size of each point proportional to the number of infants in the cohort from each NICU. There was a significant negative correlation between the rate of severe ROP and the rate of "not examined" (Kendall’s rank correlation, T = –0.30, p = 0.04). The sensitivity analysis using the worst case scenario—that is, assuming all non-examined infants had severe ROP, resulted in an additional two NICUs having corrected excess above their 95% limits. However, these NICUs had a greater than average proportion of larger, more mature infants, who have a relatively low risk of severe ROP.
Figure 3 Percentage of infants not examined for ROP by percentage with severe ROP for 25 ANZNN NICUs (1998–9). Circle size is proportional to number of eligible babies (those born at less than 29 weeks gestational age and surviving to 36 weeks). The NICUs are ordered left to right on the X-axis, by increasing percentage of unadjusted severe ROP—that is, A to Y (see table 1).
DISCUSSION
This study has demonstrated that even when data are corrected for case mix and sampling variability there remains considerable variation in rates of severe ROP among 25 NICUs in the ANZNN.
Our analysis did demonstrate a significant relation between the percentage of infants not examined for ROP and the rate of severe ROP. NICUs with lower rates of ROP tended to have a greater proportion of eligible infants not examined. Overall, only a small proportion of eligible infants (7.9%) were not examined and our previous analysis showed the majority of these were larger and more mature, with 82% being in the upper quartile for birth weight and 75% being born at 27 weeks gestation or beyond.11 One possibility is that NICUs with lower rates of ROP in infants most at risk were more likely not to examine larger and more mature infants. However, these data do emphasise the need for all infants to have appropriate screening for ROP according to the national guidelines.
To examine variation in practice among NICUs, we focused on antenatal and intrapartum variables that affect the infant’s condition before arrival in the NICU when adjusting for case mix. One possible source of variation is that some infants were intrinsically sicker than others. A number of scores have been developed as a measure of severity of illness, including the Clinical Risk Index for Babies (CRIB)15 and Score for Neonatal Acute Physiology (SNAP-PE).16,17 The ANZNN does collect data to calculate the CRIB score but not the SNAP-PE. However, the CRIB score includes birth weight and gestation, which were both used in our logistic regression model, together with highest and lowest inspired oxygen in the first 12 hours of life, which may be influenced by care practices. Additionally, both scores are reported to predict hospital mortality better than they predict morbidity.18
Parry et al19 reported on a longitudinal study of mortality among 2671 infants with birth weight <1500 g admitted to one of nine NICUs in the United Kingdom between 1988 and 1994. While in most years there were no significant differences in outcome among hospitals, the apparent performance of individual hospitals fluctuated substantially from year to year. One interesting feature of the present study is that when the same model was applied to 2000–1 data most units had similar results, in particular the two NICUs that had an excess of cases in 1998–9 still had this excess in the later period. This does suggest the possibility that there may be some systematic or consistent element(s) in clinical practice in these NICUs, which may be relevant. Equally, the one NICU with fewer than expected cases in 1998–9, continued in this situation in 2000–1. A focus on systemic quality improvement measures, aimed at reducing the incidence of severe ROP in all NICUs, is likely to be more productive than an approach that focuses solely on poorly performing NICUs. In particular, reducing the overall network rate to the 20th centile rate is likely to prevent more cases of severe ROP in 2 years (79) than if the two NICUs with excess rates improved to their expected number of cases (26 cases prevented).
One source of variation we are unable to correct for is observer differences in the assessment of retinal changes during indirect ophthalmoscopy. Such measurement bias is harder to quantify for ROP than for some other neonatal outcomes, such as intraventricular haemorrhage. At present it is not straightforward or practical for independent observers to audit the results of individual indirect ophthalmoscopy examinations, although the development and availability of contact digital retinal cameras may change this situation in the future.20 However, the NICUs in the ANZNN are all served by experienced ophthalmologists, who follow explicit examination standards guidelines,13,14 which should help to minimise this source of variation.
In conclusion, we have analysed data on the incidence of severe ROP among NICUs in the ANZNN and adjusted for both sampling variation and case mix to explore variations in rates among hospitals. Greater reductions in rates of severe ROP could be achieved if the overall rate was reduced to the 20th centile than if the focus were on outliers with excess rates. There has been much recent debate about optimal oxygen therapy in extremely premature infants21,22 and two recent trials23,24 have compared oxygen saturation targets after the first few weeks of life in these infants. Other studies have found a relation between greater variability of oxygen levels and an increased risk of severe ROP,25,26 while strategies to decrease some of this variability have also been reported.27 These and other evidence based factors should be explored in efforts to achieve network-wide improvement in rates of severe ROP.28
APPENDIX
The ANZNN Advisory Committee and Executive (*) members:
Australia: Centre for Perinatal Health Services Research, NSW: David Henderson-Smart*; Flinders Medical Centre, SA: Peter Marshall; John Hunter Hospital, NSW: Chris Wake; King Edward Memorial and Princess Margaret Hospitals, WA: Noel French, Ron Hagan and Karen Simmer; Launceston General Hospital, Tas: Chris Bailey; Liverpool Health Service, NSW: Robert Guaran; Mater Mother’s Hospital, Qld: David Tudehope; Mercy Hospital for Women, Vic: Andrew Watkins; Monash Medical Centre, Vic: Kaye Bawden*, Andrew Ramsden, Victor Yu; National Perinatal Statistics Unit, NSW: Paul Lancaster*; Nepean Hospital, NSW: Lyn Downe; Newborn Emergency Transport Service (Vic): Michael Stewart; NSW Newborn and Paediatric Emergency Transport Service: Andrew Berry; Perinatal Research Centre, Qld: Paul Colditz; Royal Children’s Hospital, Vic: Linda Johnstone, Peter McDougall; Royal Darwin Hospital, NT: Charles Kilburn; Royal Hobart Hospital, Tas: Peter Dargaville; Royal Hospital for Women, NSW: Kei Lui; Royal North Shore Hospital, NSW: Jennifer Bowen; Royal Prince Alfred Hospital, NSW: Nick Evans Royal Women’s Hospital, Qld: David Cartwright*; Royal Women’s Hospital, Vic: Lex Doyle, Colin Morley, Neil Roy; Sydney Children’s Hospital, NSW: Barry Duffy; The Canberra Hospital, ACT: Graham Reynolds; The Children’s Hospital at Westmead, NSW: Robert Halliday; The Townsville Hospital, Qld: John Whitehall; Western Australia Neonatal Transport Service: Jenni Sokol; Westmead Hospital, NSW: William Tarnow-Mordi; Women’s and Children’s Hospital, SA: Ross Haslam; Deborah Donoghue is the ANZNN Coordinator;
New Zealand: Christchurch Women’s Hospital: Nicola Austin; Christchurch School of Medicine: Brian Darlow*; Dunedin Hospital: Roland Broadbent; Gisborne Hospital: Graeme Lear; Hastings Hospital: Jenny Corban; Hutt Hospital: Robyn Shaw; Middlemore Hospital: Lindsay Mildenhall; National Women’s Hospital: Carl Kushell; Nelson Hospital: Peter McIlroy; Palmerston North Hospital: Jeff Brown; Rotorua Hospital: Stephen Bradley; Southland Hospital: Paul Tomlinson; Taranaki Hospital: John Doran*; Tauranga Hospital: Hugh Lees; Timaru Hospital: Philip Morrison; University of Auckland: Jane Harding; Waikato Hospital: David Bourchier; Wairau Hospital: Ken Dawson; Wanganui Hospital: John Goldsmith; Wellington Women’s Hospital: Vaughan Richardson; Whakatane Hospital: Chris Moyes; Whangarei Hospital: Peter Jankowitz.
ACKNOWLEDGEMENTS
Supported by the National Health and Medical Research Council of Australia (grant 211088 to DRH-S, DAD, JMS, and NJE) and the Centre for Perinatal Health Services Research, University of Sydney, Sydney, Australia.
REFERENCES
Silverman WA. Retrolental fibroplasia: a modern parable. New York: Grune and Stratton, 1980.
Vyas J, Field D, Draper ES, et al. Severe retinopathy of prematurity and its association with different rates of survival in infants of less than 1251 g birth weight. Arch Dis Child Fetal Neonatal Ed 2000;82:F145–9.
Vohr BR, Wright LL, Dusick AM, et al. Center differences and outcomes of extremely low birth weight infants. Pediatrics 2004;113:781–9.
Darlow BA, Horwood LJ, Clemett RS. Retinopathy of prematurity: Risk factors in a prospective population based study. Paediatr Perinat Epidemiol 1992;6:62–80.
Finnstr?m O, Olausson PO, Sedin G, et al. The Swedish national prospective study on extremely low birthweight (ELBW) infants. Incidence, mortality, morbidity and survival in relation to level of care. Acta Paediatr 1997;86:503–11.
Lee SK, McMillan DD, Ohlsson A, et al. Variations in practice and outcomes in the Canadian NICU Network: 1996–1997. Pediatrics 2000;106:1070–9.
Tin W, Milligan DWA, Pennefather P, et al. Pulse oximetry, severe retinopathy, and outcome at one year in babies of less than 28 weeks gestation. Arch Dis Child Fetal Neonatal Ed 2001;84:F106–10.
Donoghue DA, the A N Z N N. Report of the Australian and New Zealand Neonatal Network 2001. Sydney: ANZNN, 2003.
Simpson JM, Evans N, Gibberd RW, et al. Analysing differences in clinical outcomes between hospitals. Qual Saf Health Care 2003;12:257–62.
International Committee for the Classification of Retinopathy of Prematurity. An international classification of retinopathy of prematurity. Arch Ophthalmol 1984;102:1130–4.
Darlow BA, Hutchinson JL, Henderson-Smart DJ, et al. Prenatal risk factors for severe retinopathy of prematurity in very preterm infants of the Australian and New Zealand Neonatal Network (ANZNN). Pediatrics 2005;115:990–6.
Coory M, Gibberd R. New measures for reporting the magnitude of small-area variation in rates. Stat Med 1998;17:2625–34.
NHMRC. Clinical practice guidelines: care around preterm birth. Canberra: Australian Government Publishing Service, 1997.
Darlow BA, Clemett RS. Retinopathy of prematurity; screening and optimal usage of the ophthalmologists time. Aust NZ J Ophthalmol 1990;18:41–6.
The International Neonatal Network. The CRIB (clinical risk index for babies) score: a tool for assessing initial neonatal risk and comparing performance of neonatal intensive care units. Lancet 1993;342:193–8.
Richardson DK, Gray JE, McCormick MC, et al. Score for Neonatal Acute Physiology: a physiologic severity index for neonatal intensive care. Pediatrics 1993;91:617–23.
Richardson DK, Phibbs CS, Gray JE, et al. Birth weight and illness severity: independent predictors of neonatal mortality. Pediatrics 1993;91:969–75.
Eriksson M, Bodin L, Finnstr?m O, et al. Can severity-of-illness indices for neonatal intensive care predict outcome at 4 years of age? Acta Paediatr 2002;91:1093–100.
Parry GJ, Gould CR, McCabe CJ, et al. Annual league tables of mortality in neonatal intensive care units: longitudinal study. BMJ 1998;316:1931–5.
Gallagher K, Moseley MJ, Tandon A, et al. Nasotemporal asymmetry of retinopathy of prematurity. Arch Ophthalmol 2003;121:1563–8.
Cole CC, Wright KW, Tarnow-Mordi W, et al. Resolving our uncertainty about oxygen therapy. Pediatrics 2003;112:1415–9.
Silverman WA. A cautionary tale about supplemental oxygen: the albatross of neonatal medicine. J Pediatr 2004;113:394–6.
STOP-ROP. Supplemental therapeutic oxygen for prethreshold retinopathy of prematurity (STOP-ROP). A randomized controlled trial. I: primary outcomes, Pediatrics 2000;105:295–310.
Askie LM, Henderson-Smart DJ, Irwig L, et al. Oxygen-saturation targets and outcomes in extremely preterm infants. N Engl J Med 2003;349:959–67.
Cunningham S, Fleck BW, Elton RA, et al. Transcutaneous oxygen levels in retinopathy of prematurity. Lancet 1995;346:1464–5.
York JR, Landers S, Kirby RS, et al. Arterial oxygen fluctuation and retinopathy of prematurity in very-low-birth-weight infants. J Perinatol 2004;24:82–7.
Chow LC, Wright KW, Sola A, et al. Can changes in clinical practice decrease the incidence of severe retinopathy of prematurity in very low birth weight infants? Pediatrics 2003;111:339–45.
Horbar JD, Plesk PE, Leahy K. NIC/Q 2000: establishing habits for improvement in neonatal intensive care units. Pediatrics 2003;111 e397–e410 (www.pediatrics.org/cgi/content/full/111/4/e397).