点击显示 收起
1 From the Tufts-New England Medical Center Evidence-based Practice Center, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Boston, MA (EMB and JL); the University of Ottawa Evidence-based Practice Center (TAH and DM), the Chalmers Research Group (TAH and DM), and the Department of Pediatrics (HMS), Children's Hospital of Eastern Ontario Research Institute, Ottawa, Canada; the Southern California/RAND Evidence-based Practice Center, RAND Health, Santa Monica, CA 90407 (SJN and CHM); the Cardiovascular Nutrition Laboratory, Jean Mayer USDA Human Nutrition Research Center on Aging, Tufts University, Boston, MA (AHL); the Office of Dietary Supplements, National Institutes of Health, Bethesda MD 20892 (EAY); and the Faculty of Medicine, University of Ottawa, Ottawa, Canada (HMS)
2 Supported by contracts no. 290-02-003, 290-02-0021, and 290-02-0023 from the Agency for Healthcare Research and Quality and by the Office of Dietary Supplements, National Institutes of Health. 3 Reprints not available. Address correspondence to E Balk, Tufts-New England Medical Center Evidence-based Practice Center, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, NEMC #63, 750 Washington Street, Boston, MA 02111. E-mail: ebalk{at}tufts-nemc.org.
ABSTRACT
Evidence-based systematic reviews evaluating dietary intake and nutritional interventions are becoming common but are relatively few compared with other applications. Concerns remain that systematic reviews of nutrition topics pose several unique challenges. We present a successful collaboration to systematically review the health effects of a common nutrient, n–3 (or omega-3) fatty acids, across a wide range of clinical conditions. More generally, we discuss the challenges faced and the lessons learned during the review, the benefits of systematic review of nutritional topics, and recommendations for conducting and reviewing nutrition-related studies. Through a structured but flexible process, 3 Evidence-based Practice Centers in the Agency for Healthcare Research and Quality program produced 11 reports on a wide range of n–3 fatty acid–related topics. An important resource has been created, through which nutrition and dietetics researchers, clinical dietitians and nutritionists, clinicians, and the general public can understand the state of the science. The process identified challenges and problems in evaluating the health effects of n–3 fatty acid consumption, highlighted challenges to reviewing the human nutrition literature, and yielded recommendations for future research. The goals of these systematic reviews, the processes that were used, the benefits and limitations of the collaboration, and the conclusions of the reviews, including recommendations for future research, are summarized here.
Key Words: Evidence-based medicine • systematic review • nutrition • dietary supplements • diet • fatty acids • n–3 fatty acids
INTRODUCTION
The process of systematic review has become well established for the evaluation of specific medical treatments and diagnostic tests. The use of this approach for the evaluation of dietary intake and nutritional interventions has become more common recently, but the process has had mixed acceptance by the nutrition and dietetics scientific community because of concerns that nutrition-related topics pose several distinct challenges that are not encountered with systematic reviews of traditional medical interventions. The purposes of this article are to present an example of a large series of systematic reviews of n–3 (or omega-3) fatty acid (FA) consumption and many clinical conditions; to elaborate on the challenges faced and the lessons learned during this review; and to discuss important caveats about and recommendations for conducting and reviewing nutrition research in general and, more specifically, n–3 FA research. This review does not aim to reproduce conclusions about the effects of n–3 FAs on human health. We also summarize issues related to the collaboration that produced the systematic reviews.
Three centers in the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) program produced 11 evidence reports covering 17 clinical topics and examining the effects of n–3 FAs on clinical outcomes (Table 1). The Office of Dietary Supplements (ODS) and other institutes and centers within the National Institutes of Health (NIH) requested and funded the systematic reviews to summarize the substantial body of evidence on the clinical effects of this particular group of FAs.
View this table:
TABLE 1. Summary of populations, numbers of articles reviewed, interventions, outcomes, and methods used1
Nine reports focused solely on human studies (1-9); 2 either focused on or included questions on animal and in vitro models of arrhythmogenesis or tumor behavior (10, 11). Only the reviews of human studies are discussed here. All reports included studies that investigated n–3 FA intakes with a range of sources and doses of n–3 fatty acids. The reports focused on the very-long-chain FAs found in fish oils—ie, eicosapentaenoic acid and docosahexaenoic acid—and the n–3 FA found primarily in plant oils—ie, -linolenic acid (ALA)—in the form of both dietary supplements (eg, encapsulated oils) and food (eg, fish or high-ALA vegetable oils). A structured but flexible systematic review process was used to both maximize consistency across reports and to maintain focus on the questions of interest for each report. The reports can be accessed on the Internet (www.ahrq.gov/clinic/epcquick.htm#Otopics).
SYSTEMATIC REVIEW APPROACH
The primary goal of the systematic reviews was to summarize the evidence to inform the research community of gaps and limitations in the evidence and to provide a basis for identifying priorities for future research on the effects of n–3 FAs on various clinical conditions, mechanisms of action, and associations between intake and tissue concentrations (eg, serum or platelet phospholipids). The systematic review approach was used to minimize biases by evaluating all the evidence and providing transparency with respect to the method used. The objective and comprehensive approach of a systematic review, in which the data are evaluated with minimal biases about the subject area, is ideally suited for clarifying the nature and strength of the available evidence in controversial areas, such as the health benefits or risks of n–3 FAs. The process allows for conclusions to be drawn on the whole body of evidence rather than on selected evidence and opinions. Systematic reviews, by design, do not attempt to address all possible facets of a topic but, rather, address focused, answerable, and defined questions. Furthermore, they focus on the types of evidence that best or most completely address the specific questions of interest, and they generally do not include theoretical constructs or anecdotes as evidence of effect.
SYSTEMATIC REVIEW PROCESSES AND METHODOLOGIC COLLABORATION
ODS and other involved parts of NIH identified the primary clinical areas of interest and developed a list of potential key questions. Technical Expert Panels, independent of the EPCs, were convened; they included clinician and researcher domain experts (eg, nutrition scientists, basic scientists, and medical specialists), and representatives from NIH and AHRQ. The technical expert panels played only an advisory and consultative role. Through an iterative process, together with the technical experts, each EPC explicitly refined the key questions for each clinical outcome by defining the relevant populations, interventions, comparators, outcomes, and study designs of interest for that outcome (12). The process provided sufficient flexibility to ensure that the reviews addressed the questions of greatest interest to the research community and funding agencies and also accommodated differences in the state of science across the wide range of clinical outcomes.
The reports included common approaches, when feasible, to ensure consistency in capturing n–3 exposure information, in conducting literature searches, and in the use of methodologic quality ratings. However, to accommodate differences across the wide range of clinical outcomes of interest and differences in the needs and interests of sponsoring NIH institutes and centers, the reports exhibited flexibility through variations in the framework questions, eligibility criteria for study designs, populations, and types of outcomes.
The EPCs collaborated on several methodologic elements, including centralized literature searching, common approaches to assessing and reporting the quality and applicability of the evidence, standardized evidence tables, and the use of what we called "summary matrixes" to summarize the overall quality and applicability of the literature. We also sought common a priori definitions of variables, including interventions, and, where appropriate, common constructs within analytic frameworks. For example, we agreed on allowable sources of n–3 FAs, excluding fried fish. However, each EPC supplemented the electronic literature searches and independently developed specific screening criteria for each topic. Cognizant of the need to review the breadth of evidence and of the fact that including only randomized trials would have been insufficient for reviews of diet and chronic diseases (13), we generally included both randomized trials and observational studies.
We developed a common approach to formulating analytic frameworks for each report (Figure 1), which were based on the US Preventive Service Task Force recommendation (14). Analytic frameworks provide several advantages to both the reviewers and the readers of systematic reviews by providing a visual map outlining specific linkages among the populations of interest, exposures, modifying factors, and outcomes of interest.Specifically, these frameworks depict the chain of logic that evidence must support to link the exposure (eg, consumption of n–3 FAs) to clinical outcomes. This process helps guide the development of key questions, determine study eligibility criteria, and interpret and contextualize relevant studies.
FIGURE 1.. Generalized analytic framework of the effects of n–3 fatty acid exposure on outcomes. FAs, fatty acids; RBC, red blood cell; WBC, white blood cell.
Although it is generally accepted that deficiencies in study design, conduct, analysis, and reporting may lead to biased results, there is little agreement on whether any specific study factor consistently biases or restricts the applicability of given studies (15-19). All 13 AHRQ-sponsored EPCs use a similar set of basic criteria to assess study quality; however, no uniform approach is used to determine which specific criteria are used. For the purposes of this simultaneously generated set of reports, we agreed a priori to standardize our approaches to allow better comparisons across reports. Thus, for all reports, the validity of included randomized trials was assessed by using the Jadad score, which assesses randomization, double blinding, and dropouts and withdrawals on a 5-point scale (20); adequacy of allocation concealment as described by Schulz et al (18); and a 3-category system (good, fair, or poor) that qualitatively grades the likelihood of bias based on a range of potential quality factors. The 3 EPCs did not, however, use the same approaches for grading nonrandomized studies: some used the 3-category system and some used other scales (21). In addition, given the unique nature of the data in some reports, topic-specific quality scales were used. Each study was also given an applicability score(broad, moderate, or narrow) to summarize its relevance to the populations of interest for each report question.
Among the strengths of coordinating a large number of reviews across several EPCs was that we could achieve consistency in several centralized processes for core components of the reviews. However, because each EPC has its own systems and methodologic approaches, we were unable to achieve complete consensus regarding data analysis (eg, evaluation of net changes or of final values), types of outcomes of interest (eg, definition of clinical or intermediate outcomes), or style of reporting (eg, exact format of tables).
OVERVIEW OF EVIDENCE-BASED REPORT FINDINGS
The primary clinical conditions reviewed by the 3 EPCs are summarized in Table 1, Table 2, and Table 3. The specific questions varied by topic, but they were intended to cover the effects of n–3 FA consumption either on prevention or treatment of specific disease states or on the risk factors or markers of the diseases.
View this table:
TABLE 2. Summary of body of evidence and overall results for fish oils and fish1
View this table:
TABLE 3. Summary of body of evidence and overall results for -linolenic acid (ALA) (supplements and plant-source diet)1
When the included studies are grouped by topic, the largest groups were studies of cardiovascular disease (CVD)–related endpoints (32%), of child and maternal health (17%), and of mental health (15%) (Table 1) Most interventions evaluated the effect of fish-oil supplementation (57%) or fish diets (26%) (Tables 1 and 2). The reports summarized the evidence of adverse events and of any differential effects among populations or risk groups.
The reports included studies documenting dietary intakes of n–3 FAs from both marine and plant sources, as either nutritional supplements or dietary components. Approximately two-thirds of the study arms evaluated n–3 FA supplements; the remaining study arms evaluated food sources (eg, plant oil spreads or fish). Fish oils constituted the source of n–3 FAs in more than half of the studies, and fish did so in approximately one-quarter of the studies. Only 10% of interventions provided ALA alone (without fish oils) (Tables 1 and 3); thus, for approximately one-third of the clinical conditions examined, no studies evaluated ALA. The types, designs, and durations of reviewed studies varied widely across reports.
In part, this variability reflects the intensity with which each clinical area has been researched and the variations in eligibility criteria used by the different reports. For most reports, randomized controlled trials were the studies most commonly reviewed, although prospective and retrospective observational studies also were commonly included. Studies of fish and fish oil were more commonly randomized controlled trials than studies of ALA. Overall, approximately one-quarter of the studies were deemed to be of good quality, and one-quarter were of poor quality. Among the randomized trials, only about one-quarter had a 4 or 5 (out of 5) Jadad score. Similarly, approximately one-quarter of the studies were of broad applicability, and one-quarter were of limited applicability. The overall quality and applicability of studies varied substantially across clinical topics. Fish-oil studies were similar to ALA studies in their overall methodologic quality and applicability, except that few ALA randomized trials received high Jadad scores.
For most topics reviewed, the evidence of health benefits of either fish oils or ALA was inconclusive. The paucity of studies, particularly of randomized trials, and the heterogeneity of interventions evaluated (ie, the wide variety of FA doses, combinations, forms tested, and outcomes measured) prevented firm conclusions. Many studies were small, methodologically flawed, and of limited applicability, and they frequently were heterogeneous in their results. However, incompletely reported data, failure to fully describe study designs or conditions, and differences among participant populations often made it difficult to meaningfully explain the heterogeneity in findings.
The evidence was adequate to attribute a health benefit to consumption of supplemental fish oil or dietary fish only with respect to a reduction in cardiovascular events, principally as secondary prevention. In certain persons and at various doses, fish oils reduce serum triacylglycerol concentrations, reduce blood pressure by a small but significant amount (2 mm Hg), reduce corticosteroid use in patients with rheumatoid arthritis, and reduce the risk of non-Alzheimer dementia. No association was found between n–3 FA consumption and cancer incidence, although, for most types of cancer, the data were not sufficient to exclude possible associations.
Most of the evidence of adverse events was found in the studies of CVD-related outcomes. Even among these studies, however, only 148 of 395 (37%) evaluated articles provided any information on adverse events, and the reporting of adverse events was generally inadequate and incomplete. In particular, rates of adverse events were not systematically or consistently monitored and reported. Overall, given these caveats, no evidence of substantial adverse events related to n–3 FA consumption beyond mild gastrointestinal symptoms was found. However, the lack of protocols for monitoring adverse events could have resulted in underreporting or biased reporting of actual effects.
ADDED VALUE OF SYSTEMATIC REVIEW REPORTS
The dissemination and added value of these reports have already been observed. Since their release, several national and interagency meetings have specifically used these reports to guide research agendas. ODS and the National Heart, Lung, and Blood Institute gathered domain experts in a workshop to help the NIH determine the future research on n–3 FAs and CVD that should be funded and the general design that should be used for those studies. By integrating information from the evidence reports with the participants’ expertise, the workshop concluded that the need for further studies of the effects on established cardiovascular risk factors (eg, lipoproteins and blood pressure) is limited, but that a well-designed, long-term randomized trial of both fish-oil and ALA supplements should be conducted to assess their effects in a high-risk population (22).
Similarly, the National Institute of Environmental Health Sciences, the Office of Rare Diseases, and ODS convened a workshop to evaluate the possible development of a prevention trial designed to assess the effect of n–3 FA supplementation on asthma and allergy during pregnancy and infancy. Findings from these reports have also been presented to researchers at other NIH-sponsored meetings and at meetings such as Experimental Biology, Translating Research into Practice and Policy, and SupplySide East, where the goals of the presentation were to highlight the issues related to the value of the current evidence and to offer suggestions to improve future research. A larger audience has also been reached through journal publications, press releases, and the general press (23-33).
LIMITATIONS OF THE EVIDENCE AND RECOMMENDATIONS FOR FUTURE RESEARCH
Many deficiencies that are common in the current medical and health literature were also present in this field, as shown in Table 4. For example, inadequate reporting of randomized trials can introduce bias into effect estimates of interventions and clearly makes interpretation of studies and their possible biases problematic (34, 35). Other deficiencies were more specific to both n–3 FA research and nutrition research in general (Table 4).
View this table:
TABLE 4. Common limitations of current studies of n–3 fatty acids
Considerable energy has been devoted to providing guidance to authors in improving their reporting of randomized trials, such as the CONSORT Statement (35), which is associated with better reporting of randomized trials (36). Thus, to improve the utility of publications, the CONSORT criteria should be used as a guideline for reporting all future trials. The CONSORT Statement has recently been extended to help with reporting trials of herbal interventions (37). Whereas many of the statement's considerations may apply to trials of nutritional supplements and diets, further refinements to the criteria may be needed for this field of study.
Additional future research recommendations include improving and developing plausible models of disease interactions in animal and in vitro models, and ensuring that hypotheses are tested for both primary and secondary prevention (ie, in persons with and without a history of the conditions of interest). For several clinical conditions, such as dementia and cancer, dietary assessments starting at a relatively young age with very long-term follow-up are necessary to evaluate primary prevention. Such designs could feasibly be implemented by converting randomized trials of dietary or supplement interventions into long-term observational studies.
The applicability and strength of studies would be increased if research groups begin to collaborate on multicenter trials. Such study designs could also stretch scarce research dollars and simultaneously allow consideration of confounding factors. Furthermore, comprehensive and systematic adverse event–monitoring systems should be developed and incorporated into future studies. Many other specific recommendations that are also applicable to nutrition research in general were made. Notably, one EPC made the practical suggestion to piggyback the assessment of different sets of clinical outcomes (eg, those for mental health) onto studies primarily designed to evaluate other sets of clinical outcomes (7).
DISCUSSION
The series of evidence reports on n–3 FAs commissioned by AHRQ has provided an important resource for nutrition and dietetics researchers, clinical dietitians and nutritionists, clinicians, and consumers with respect to what is known—and not known—about the effects of increased consumption of n–3 FAs. They have also helped NIH set research agendas and priorities. This review elaborates on the benefits of the reports, lessons learned during the systematic review process, and recommendations for conducting and reviewing nutrition research.
These reports highlight an important message to the nutrition research community that, whereas future research on many clinical questions is necessary, our knowledge of the health effects of n–3 FAs (and by extension, other nutrients) will not significantly progress simply through the conduct of more studies. Advances in knowledge will require substantial improvements in the quality of study rationale, design, conduct, and documentation. Additional studies done without significant methodologic improvements will be unlikely to meaningfully improve the state of the science with respect to the health effects of nutrients.
For practitioners and consumers, whereas the reviews provide few definitive statements about known effects of n–3 FA consumption, they do provide a resource to help understand the evidence. They provide a comprehensive and rigorous evaluation of the literature that would be difficult for individuals to replicate. There was frequently a lack of sufficient information indicating that a particular intervention is effective, but the reviews provide a basis for fully informed decisions regarding n–3 FA consumption.
The reports highlighted important differences between fish oils and the plant oil ALA (eg, several benefits of fish oils for CVD and risk factors have not been found with ALA). They have also highlighted several important deficiencies in the research findings currently available. Despite a large evidence base, few clinical areas exist in which it is possible to make adequate assessments of the possible benefit or harm of n–3 FAs. This situation is largely a result of the small number of well-designed, high-quality studies—including both randomized controlled trials and observational studies—that have been conducted. Even fewer of these studies clearly defined interventions or adequately controlled for factors that are important in most nutrition-related studies, such as background diet and equivalent control subjects.
Interpreting human studies in the area of nutrition can be considerably more challenging than is interpreting drug trials. One factor is energy balance. To avoid changes in body weight that can confound the outcome measures (eg, LDL cholesterol), if a food is added to or subtracted from the diet, another food must be subtracted from or added to the diet to compensate. Thus, it is necessary to use controls that are energetically equivalent. Furthermore, the effects of increased consumption of a food product (eg, fish or fish-oil supplements) may differ depending on whether a person's background diet is already rich in that food (eg, fish). In observational studies, if a person reports eating a diet high in fish, he or she is likely to be eating less of another food, perhaps meat. Among observational studies, it is critical that potential confounding factors—such as background diet, food preparation method (eg, frying), weight, health status, and disease-specific risk factors—be accounted for (38). In addition, nutrients such as n–3 FAs are derived from sources that may contain other bioactive compounds that may affect the clinical outcomes of interest. The presence of these compounds may confound the effects of the nutrient of interest, particularly if these other compounds are not accounted for in the control interventions. It is also important to address the balance between different dietary fats (eg, n–3 and n–6 FAs; eicosapentaenoic acid, docosahexaenoic acid, and ALA) or altered intakes of other FAs (eg, saturated fatty acids). Diets or even nutritional supplements contain many nutrients that may interact, which adds a layer of complexity to their evaluation (39). In our review of the literature, it became apparent that only rarely were these factors taken into consideration.
Whereas, overall, better research and reporting are needed, the systematic reviews highlighted the possibility that adequate evidence for the effects of fish oils on a few outcomes, including improvements in lipoproteins, triacylglycerols, and blood pressure, may exist from which to base recommendations. Acknowledgment that these effects of n–3 FAs have been adequately studied could allow scarce resources to be better spent in evaluating underresearched areas.
Our evidence reports showed that evidence-based methods can be successfully applied to the evaluation of nutrition topics. These reports have highlighted the numerous clinical topics for which the evidence is inconclusive and on which further high-quality studies are needed. They also can provide the basis for planning future research. Our experience also serves to show the resource- and time-intensive nature of the systematic review process. The coordination among the EPCs allowed certain efficiencies and provided an excellent opportunity for each group to learn new approaches and improve its own processes. However, once data review began, coordination of each step was difficult to implement. Nevertheless, all groups found that conscientiously using a rigorous approach to define review parameters was essential to bringing focus to the clinically important questions being examined. Through consensus and collaboration, we ensured that the review questions covered the areas of interest but were clear and concise, which led to more focused reviews and made all aspects of the process, such as the searches, more efficient. The collaboration among the 3 EPCs, AHRQ, NIH, and the technical experts and the large scope of the overall project maintained a broad perspective precisely to avoid potential pitfalls of other systematic reviews of this topic (39-43).
The resource-intensive nature of these systematic reviews also raises practical questions. The benefits of an overarching set of reviews continually need to be balanced with the costs and delays inherent in such a large endeavor (almost 29 000 citations and 3300 full articles were screened, and 523 studies were reviewed). However, the breadth of these reviews should allow future updates to be sufficiently focused, both by clinical topic and on particular key questions, so that they can be completed relatively quickly. As time passes, we expect an increasing need for updates to these reports. Systematic reviews are most useful when they are kept current, because evidence is continually evolving as new research becomes available. Health care interventions that are currently accepted as effective may in the future be shown to be ineffective or harmful, or vice versa. Ignoring the emergence of new information could undermine the validity of even the most current, high-quality systematic reviews. As the number of newly published studies grows, depending on their findings, the current systematic reviews may become less authoritative.
The systematic review process, particularly with the coordination of the 3 EPCs, was the definitive approach to understanding the state of the literature on n–3 FAs and human health. These systematic reviews were able to confirm several beneficial health effects of increased n–3 FA consumption but they also found major deficiencies in the literature and highlighted the clinical areas, including adverse effects, in which the evidence is inadequate. The wide-ranging group of researchers involved in the reviews, both collectively and individually, capitalized on their experience to influence the thinking, both at NIH and among nutrition researchers, about the future research that is needed to address the gaps in the evidence, rather than to add further inadequate data to the literature.
ACKNOWLEDGMENTS
The authors’ responsibilities were as follows—all authors: conception of the collaboration; EMB, TAH, SJN, and AHL: data acquisition and the draft of the manuscript; and all authors: critical revision of the manuscript. None of the authors had a personal or financial conflict of interest.
REFERENCES