点击显示 收起
the Department of Neurology, Rudolf Magnus Institute (P.H.A.H., J.v.G., L.J.K., A.A.), and the Julius Center for Health Sciences and Primary Care (A.A.), University Medical Center Utrecht, The Netherlands
the Department of Neurology (P.J.K.), Erasmus Medical Center Rotterdam, The Netherlands.
Abstract
Background and Purpose— Classification of outcome events is essential in clinical research. The Executive Committee of the European/Australasian Stroke Prevention in Reversible Ischaemia Trial (ESPRIT), a secondary prevention trial in patients with cerebral ischemia, repeatedly encountered problems in classifying the cause of death after a stroke if the interval between these events was relatively long. We aimed to develop guidelines for classifying such events.
Methods— Twenty-nine neurologists with a special interest in stroke filled out a questionnaire and audited 5 case vignettes. On the basis of this information, we developed a proposal for classifying causes of death after stroke. This proposal was evaluated in an interobserver analysis in which 10 neurologists or residents in neurology assessed 20 of 100 case vignettes.
Results— Initially, there was great variation in classifications of the case vignettes, mainly because the correspondents strongly disagreed about the relative importance of the interval between stroke and death, the degree of disability after stroke, the discharge destination (home or institutional care), and the coexistence of infection. In the new proposal, the main criteria were "interval after stroke" (cutoff point at 1 month) and "best Rankin grade after stroke" (cutoff at 3). In the interobserver analysis, good agreement was obtained among the 5 pairs of neurologists who assessed the 20 case vignettes ( 0.80 95% CI, 0.68 to 0.92).
Conclusions— In the absence of guidelines, neurologists show striking variation in the classification of causes of death in patients who die after a stroke. With precise rules, agreement in the classification of death after stroke strongly improved.
Key Words: classification death stroke
Introduction
In clinical trials of secondary stroke prevention, the main outcome events are recurrent stroke, myocardial infarction, and death. Such outcome events are also often used in observational studies. In most such studies, an auditing committee classifies these events according to prespecified criteria. The executive committee of the European/Australasian Stroke Prevention in Reversible Ischaemia Trial (ESPRIT)1 repeatedly encountered problems in classifying the cause of death in patients who had previously experienced a stroke. An example is a patient who remained dependent in a nursing home after a brain infarct, developed pneumonia and heart failure, and eventually died 4 months after the stroke (Table 1, case vignette 2). Members of the auditing committee could not agree whether the subsequent pneumonia, heart failure, and ultimately death should be attributed to the initial stroke or whether these events should be regarded as separate complications.
The difficulty in reaching consensus arose because some neurologists tended to attach most importance to the long interval between stroke and death, whereas others put the emphasis on stroke-related disability. Because we could not find appropriate references in the literature, we decided to consult with stroke experts from all over the world, with the aim to formulate a practical guideline for the auditing of death after stroke in clinical research.
Methods
We sent a message to 43 experts with a special interest in secondary prevention after stroke and with expertise in auditing outcome events in clinical trials. We explained our problem and asked whether they were willing to answer 5 questions on the classification of the cause of death after a stroke (Table 2) and to audit 5 cases. All cases were based on true patient data from the ESPRIT trial. The case vignettes are described in Table 1. A single item could be chosen from a list with 12 predefined causes of death (Table 3).
After having studied the completed questionnaires and opinions, we formulated a proposal with guidelines for classifying the cause of death after stroke (Table 4). This proposal was sent to all participating neurologists for approval.
In the second stage of this study, the proposed guidelines were tested by means of an interobserver analysis. From several stroke trials and studies coordinated by the stroke trial office in Utrecht (ESPRIT,1 Dutch TIA Trial ,2 and Life Long After Cerebral Ischaemia (LiLAC)3), case histories of 100 patients were selected who experienced a stroke and died during follow-up. For each patient, a short report summarized the qualifying event, the stroke during follow-up, and the circumstances at the time of death. Seven Dutch neurologists and 3 residents in neurology, all with a special interest in stroke (see Appendix), consented to audit 20 case reports and to classify the causes of death as "stroke" or "other cause" on the basis of the proposed guidelines in Table 4. In this way, each case report was classified by 2 different physicians. Interobserver agreement on causes of death was assessed with statistics.
To compare the agreement of the initial classifications of the participants in the questionnaire with the agreement in the classifications in the interobserver analysis, based on the new criteria, we reclassified the initial classifications. If the classified cause of death was "fatal cerebral infarction," "fatal cerebral hemorrhage" or "fatal stroke," we reclassified it as "stroke"; if another classification was chosen, we classified it as "other cause." The neurologists were divided in 2 groups according to order of response (first responder in group 1, second in group 2, third in group 1, etc), resulting in 70 classification couples (one responder was excluded for this analysis to obtain an even number of responders). Interobserver agreement on these classified causes of death was assessed with statistics.
Results
Twenty-nine (see Appendix) of 43 international experts filled out the questionnaire. The answers on the questions are summarized in Table 2, and the assigned causes of death for the 5 test cases are shown in Table 3.
The length of the interval between stroke and death appeared to be important for 26 of 29 of adjudicators, but there was no consensus on the interval between stroke and death, beyond which stroke no longer should be regarded as a cause of death. Five experts commented that the length of this period depends on the type and severity of the stroke and on the so-called "direct cause of death" (for example, brain herniation, aspiration pneumonia, or heart failure). Three neurologists who did specify a maximum interval breached their own rule when they audited the cases. With regard to the degree of disability according to the modified Rankin scale,4,5 most neurologists indicated that it should be taken into account, but again, there was no agreement on the cutoff point for dependence below which stroke always should be taken as cause of death. Also, there was much difference of opinion on the importance of the discharge destination of the patient (home versus nursing home). There was more agreement about the importance of comorbidity at the time of the stroke and on whether the patient experienced an infection at the time of death; a clear majority thought these conditions should be taken into account.
Not surprisingly, there was great variation in the way neurologists classified the cause of death in the 5 case vignettes (Table 3). Only in cases 1 and 5 was there substantial agreement. The of the interobserver analysis of these initial classifications was 0.31 (95% CI, 0.10 to 0.54), which reflects poor agreement.
After having weighed all different opinions and answers, we proposed a new guideline for classifying the cause of death after stroke (Table 4). All but 2 of the 29 correspondents agreed with the proposal without major objections.
In the second phase of the study, in which 5 pairs of neurologists each classified 20 different case histories, there was agreement on the cause of death in 90 of the 100 patients (Table 5), resulting in a value of 0.80 (95% CI, 0.68 to 0.92).
Discussion
Our study shows that in the absence of clear guidelines, there was very little agreement among world experts on the classification of cause of death in patients who die after a stroke in the setting of a clinical trial. On the basis of all these opinions, we designed a simple set of criteria (Table 4) on which most experts who participated in our study (27 of 29) agreed. Moreover, a subsequent interobserver study among neurologists in a single country showed there was excellent agreement in classification of cause of death when this set of criteria was used. In our study, a of 0.80 was obtained, whereas in general, a between 0.61 and 0.80 is considered to reflect substantial agreement, and a between 0.81 and 1.00 reflects almost perfect agreement.6,7
The World Health Organization (WHO) published the International Statistical Classification of Diseases and Related Health Problems,8 a manual with rules and guidelines for the coding of mortality and morbidity for the purpose of statistical analysis. According to their general principles, all deaths should be attributed to the primary cause of a sequence leading to the fatal disease. In this way, death in a patient dying of an aspiration pneumonia after a stroke should, for example, be attributed to atherosclerosis. However, modification rules can override these general principles, and special "Sequelae of" codes are provided, for instance, a "Sequelae of cerebrovascular disease" code, which can be used if a patient dies as a consequence of the stroke. There is no time limit or other restriction to this WHO code that can be consulted if "there is evidence that death occurred from residual effects of this condition rather than from those of its active phase." This leaves a wide margin of uncertainty for all physicians when classifying and reporting causes of death in a given patient.
Although the number of neurologists who participated in the questionnaire was limited to 29, they represent a fair sample of the 43 world experts we consulted, and the range of answers and classifications could have been only wider in a larger sample. Most disagreement was related to the interval between stroke and death, the disability level of the patient after the stroke, the existence of infection at the time of death, and of the discharge destination after stroke. The disagreement on this last item may partly be caused by intercultural differences because care for the elderly and disabled differs between countries, even within the Western world. The other items seem to be more essential for the purpose of this questionnaire, but the answers we initially received differed not only between but also within correspondents, given some discrepancies between opinion and actual classification.
The agreement in the interobserver analysis may have been influenced by the single country background of the participants, resulting in an exaggeration of the agreement. Although the observers work in 5 different hospitals in The Netherlands, their Dutch medical training may have resulted in a better agreement than would have been obtained in an international analysis. However, from the difference in between this analysis and the analysis of the initial classifications, we can infer that the "gut feeling" of stroke experts has a strong individual basis and does not result in agreement in consensus-based classifications.
To study whether application of the new criteria changes the original classifications, we compared the classifications of the cases in the interobserver analysis with those in the studies from which they were derived (DTT, LiLAC, and ESPRIT). Of the 90 cases in which there was agreement on classification in the interobserver analysis, 20 were differently classified in the original study. In all these, death was classified as "stroke" in the interobserver analysis, whereas the original classification was "other cause."
The results of previous secondary prevention trials probably have not been substantially influenced by disagreement about the cause of death after stroke because in most of these studies, only the first vascular event a patient experiences is included in the primary analysis. In the study cases, all first vascular events would be stroke, and the event of death would be analyzed only for secondary survival analysis.
When our new set of criteria is applied to the 5 original case vignettes in the questionnaire, all 5 would be classified as stroke deaths (cerebral infarction, cerebral hemorrhage, or stroke of unspecified nature). This is in contrast with the classification of the majority of participating neurologists except in 2 cases (vignettes 1 and 5) in which death occurred within 2 weeks after stroke. In the other cases, other classifications were initially prompted by the presence of comorbidity and a long interval between stroke and death. There is no universal truth in this matter. To quote one of our correspondents: I always ask myself the question "Would this patient have died if he would not have suffered that stroke" To answer that question, we devised an admittedly arbitrary but pragmatic set of criteria to simplify the work of auditing committees in clinical studies. The proposed guideline was developed and tested by neurologists, whereas in some countries, stroke patients are cared for by general physicians or geriatricians. Nevertheless, we think the weighing of causal factors in the chain of events between stroke and death is not likely to depend on medical discipline. It is not designed to determine the "one and only" true cause of death in patients participating in a clinical study. Nevertheless, if the same criteria are used in different studies, the results of these studies can be compared more reliably. Moreover, from the perspective of internal validity, it is no problem to use criteria with some arbitrary aspects in clinical trials as long as these rules are applied in the same way to all treatments. Because the majority of correspondents agreed with our proposed criteria, and the criteria proved to be workable and reliable in the interobserver analysis, we conclude that we succeeded in our goal to formulate a practical guideline for the auditing of death after stroke in clinical research.
Appendix
Participants in Questionnaire
Australia: G.A. Donnan, G.J. Hankey
Belgium: G. Vanhooren
Finland: M. Kaste
France: D. Leys, J.M. Orgogozo, M.G. Bousser
Germany: W. Hacke
Italy: L. Candelise, S. Ricci
Portugal: J.M. Ferro
Singapore: C.P.L.H. Chen
Spain: A. Chamorro
Sweden: B. Norrving
Switzerland: J. Bogousslavsky, H.P. Mattle
The Netherlands: A. Algra, P.J. Koudstaal, J. van Gijn, G.J.E. Rinkel, M. Vermeulen
United Kingdom: M.M. Brown, M.S. Dennis, P.M. Rothwell, P.A. Sandercock, G.S. Venables, C.P. Warlow
United States of America: H.P. Adams, R.G. Hart
Participants in Interobserver Analysis
S.L.M. Bakker, E.L.L.M. De Schryver, D.W.J. Dippel, C.L. Franke, J. van Gijn, L.J. Kappelle, P.J. Koudstaal, V.I.H. Kwa, D.J. Nieuwkamp, M.D.O. Pruissen.
Acknowledgments
We thank all participating neurologists (see Appendix) for their response on the questionnaire and on our proposal for a classification schedule, and we are also grateful to all participants in the interobserver analysis (see Appendix) for their cooperation.
References
De Schryver EL. Design of ESPRIT: an international randomized trial for secondary prevention after non-disabling cerebral ischaemia of arterial origin. European/Australian Stroke Prevention in Reversible Ischaemia Trial (ESPRIT) group. Cerebrovasc Dis. 2000; 10: 147–150.
A comparison of two doses of aspirin (30 mg vs. 283 mg a day) in patients after a transient ischemic attack or minor ischemic stroke. The Dutch TIA Trial Study Group. N Engl J Med. 1991; 325: 1261–1266.
van Wijk I, Kappelle LJ, van Gijn J, Koudstaal PJ, Franke CL, Vermeulen M, Gorter JW, Algra A. Long-term survival and vascular event risk after transient ischaemic attack or minor ischaemic stroke: a cohort study. Lancet. 2005; 365: 2098–2104.
Bamford JM, Sandercock PA, Warlow CP, Slattery J. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1989; 20: 828.
van Swieten JC, Koudstaal PJ, Visser MC, Schouten HJ, van Gijn J. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1988; 19: 604–607.
Cohen J. A coefficient for agreement for nominal scales. Educ Psychol Meas. 1960; 20: 37–46.
Veldhuyzen van Zanten SJ, Hijdra A. [The use of kappa in the study of variability between observers]. Ned Tijdschr Geneeskd. 1988; 132: 199–202.
World Health Organization. International Statistical Classification of Diseases and Related Health Problems. 1993. Ref type: generic.