点击显示 收起
1 From the Institute of Public Health, The Panum Institute, University of Copenhagen.
2 Supported by grants from the Research Centre of the Danish Health Ministry, the Danish Research Councils, and the Novo Nordisk Foundation. 3 No reprints available. Address correspondence to P Møller, Institute of Public Health, The Panum Institute, University of Copenhagen, Blegdamsvej 3, DK-2200 Copenhagen N, Denmark. E-mail: fipm{at}farmakol.ku.dk.
ABSTRACT
Many epidemiologic studies have addressed the possible preventive effects of antioxidants in disease causation and progression. With the use of molecular techniques, it is feasible to investigate specific properties of antioxidants in intervention studies. The most widely used techniques to investigate oxidative DNA damage in white blood cells are the measurement of 7-hydroxy-8-oxo-2'-deoxyguanosine and the comet assay. The types of antioxidant intervention studies include those involving single or multiple supplementations of vitamin C, vitamin E, or carotenoids and those involving various natural food products (eg, carrot juice). In short-term intervention studies (usually weeks or a few months), results have been mixed. Single-dosing studies found that decreased oxidative DNA damage lasted only hours after antioxidant supplementation, suggesting that the preventive effect is relatively short. In addition, many of the positive studies were not placebo-controlled, thus leaving a possibility of false-positive results caused by period effects, eg, seasonal variation, changes in the lifestyles of the subjects, or variation in measurements over time. Because participation in an antioxidant intervention study may cause changes in dietary habits and because seasonal changes may have profound effects, it is recommended that future studies have a placebo-controlled, parallel design rather than a crossover design.
Key Words: Antioxidants • comet assay • humans • intervention studies • 7-hydroxy-8-oxo-2'-deoxyguanosine • oxidative DNA damage • vitamin C • vitamin E • carotenoids
INTRODUCTION
The cells of the human body are continuously attacked by reactive oxygen species (ROS), which arise as natural byproducts of normal cellular energy production or are generated in large amounts by exhaustive exercise or by chemical agents in the environment (1). In addition, cancer and chronic inflammatory diseases generate ROS as part of the pathophysiologic mechanism (2). Cells have evolved a complex network of defense barriers to counteract the generation of ROS and protect against the oxidation of macromolecules by scavenging ROS. The dietary intake of antioxidants is thought to play a major role in this network. Some compounds that have antioxidant properties, such as vitamin C, vitamin E, carotenoids, and flavonoids, have been identified in large quantities in some natural food products (3). However, these antioxidants are few in comparison with the many other antioxidants and other bioreactive substances in natural products.
The antioxidant capacity of fruit and vegetables plays an important role in their protective effect (4, 5). From epidemiologic studies it has been concluded that lifestyles characterized by a high consumption of fruit and vegetables are associated with lower incidences of cancers of the lung, stomach, breast, colorectum, oropharynx, and liver (6). However, the large antioxidant intervention studies did not show a decreased cancer risk associated with antioxidant consumption (7–9).
An alternative approach to investigate antioxidant effects is to study the behavior of oxidative stress biomarkers in controlled supplementation studies. Using DNA as the biological target molecule, 2 methods appear to be the tests of choice: measurement of 7-hydroxy-8-oxo-2'-deoxyguanosine (8-oxodG) and the comet assay. From a mechanistic perspective, dietary antioxidants are expected to inhibit the formation of oxidative DNA damage. When ROS are formed in vitro or by oxidative stress in vivo, several types of oxidative DNA lesions are formed, including small base lesions and exocyclic adducts (10). 8-OxodG is one of the most easily formed oxidative DNA lesions. It can be detected in both urine and tissues after oxidative stress (11). 8-OxodG can be measured with the use of several chromatographic techniques, including HPLC with electrochemical detection and tandem mass spectrometry (MSMS), gas chromatography–mass spectrometry (GC-MS), and thin-layer chromatography with 32P postlabeling, and with antibody-based immunoassays (12). There is large variation between the different assays in the measured concentrations of 8-oxodG, partly because of artificially generated 8-oxodG in the assays (13). In particular, GC-MS–based methods produce high values because of artifacts. Recently, however, a joint effort of laboratories mainly in Europe narrowed down the range of measured values of 8-oxodG obtained by the various methods (13). 8-OxodG was probably chosen as the prime oxidative DNA-damage biomarker because it can be sensitively measured by HPLC with electrochemical detection and because it appears to be more closely linked with mutagenesis than are, for example, oxidative adenine and thymine lesions (11). For other types of oxidative DNA damage, standardization of laboratory procedures and agreement on basal values are still warranted.
The comet assay detects DNA strand breaks (SBs). An enzyme-modified version of the comet assay was developed to detect oxidatively altered nucleotide bases by including a DNA digestion step using DNA glycosylase enzymes (14). Oxidized purines, including 8-oxodG, can be detected by formamidopyrimidine-DNA glycosylase (EC 3.2.2.23), and oxidized pyrimidines by endonuclease III (ENDOIII; EC 4.2.99.18). In addition, in vitro exposure of cells to DNA-breaking agents, such as hydrogen peroxide or ionizing radiation, has been used as a semiquantitative measurement of the donor’s antioxidative status. This modification of the comet assay is based on the notion that the intracellular content of antioxidants will inhibit DNA breakage.
Here, we review antioxidant supplementation studies in which oxidative DNA damage was detected in leukocytes, lymphocytes, or mononuclear blood cells. We chose to limit our review to only those studies in which damage was assessed in blood cells because most studies assessed damage in that cell population. We also limited the survey to only those studies in which damage was detected either by the comet assay (or related assays) or by measurement of 8-oxodG. Data from assays that require the growth of cells, eg, assays that detect micronuclei or chromosomal aberrations, are not included in the present review. Moreover, other types of lesions were included in very few studies, and potential methodologic problems in those studies (eg, possible artifacts for some lesions) limit the interpretation of their results. An article featuring an extensive review of urinary 8-oxodG excretions was recently published (12). In addition, reviews by Halliwell (15) and Collins (16) summarizing the effects of antioxidants on biomarkers of oxidative DNA damage have been published.
SINGLE-DOSING ANTIOXIDANT INTERVENTION STUDIES
A summary of the results of single-dosing antioxidant supplementation studies is provided in Table 1. Vitamin C supplementation decreased the number of SBs as well as the sensitivity to hydrogen peroxide and ionizing radiation (18, 19). The effect of vitamin C appeared to peak early, within 2–8 h after consumption, and smokers benefited more from dietary vitamin C supplementation than did nonsmokers (19). Consumption of vitamin E or ß-carotene decreased the number of SBs and hydrogen peroxide sensitivity 18–24 h after ingestion (19). Decreased 8-oxodG concentrations were observed 6 h after consumption of tomatoes (8 g/kg body wt) (21). The later onset of the carotenoid-related effect may have been because of a slower uptake of these antioxidants than of the water-soluble vitamin C.
View this table:
TABLE 1 . Oxidative DNA damage after single dosing of antioxidants1
Consumption of 200 g lightly fried onions/d as a source of flavonoids decreased the number of SBs and hydrogen peroxide sensitivity, whereas the number of ENDOIII sites was apparently unchanged (17). When uncooked cherry tomatoes were added to the onion diet, there were fewer ENDOIII sites but there was no decrease in the number of SBs or in hydrogen peroxide sensitivity. In any case, the effects on the biomarkers were apparently transient because they were observed 4–8 h after consumption but not at 24 h (17). A study of kiwi fruit juice, in which subjects consumed a single dose of 500 mL, showed decreased sensitivity to hydrogen peroxide, with a maximum after 8 h, but no effects on the number of SBs or ENDOIII sites (20).
MULTIPLE-DOSING, SHORT-TERM ANTIOXIDANT INTERVENTION STUDIES
Most antioxidant supplementation studies included intervention periods of weeks or months. For the sake of clarity, the studies are divided into single and multiple antioxidant studies and studies using natural food products. To date, 26 studies using either the comet assay or measurement of 8-oxodG have been published. Details of the studies are provided in Tables 2 (comet assay) and 3 (8-oxodG).
View this table:
TABLE 2 . Multiple dosing of dietary antioxidants with assessment of effects by the comet (or related) assay1
View this table:
TABLE 3 . Dietary antioxidant supplementation studies with assessment of 7-hydroxy-8-oxo-2'-deoxyguanosine (8-oxodG) in DNA from white blood cells1
Single antioxidant trials
Studies of vitamin C supplementation provide conflicting results. Supplementation with vitamin C (500 mg/d) for 3 wk decreased 8-oxodG concentrations as assessed by GC-MS after 3 and 6 wk in nonsmokers (36, 48). However, instead of a random sequence, the placebo treatment was carried out before the vitamin C supplement period. In a placebo-controlled crossover study, subjects who received either 60 mg or 6 g vitamin C/d for 2 wk had increased plasma concentrations of vitamin C; yet, there was no effect on the number of SBs or on hydrogen peroxide sensitivity (27). In an exceptionally well-controlled trial, subjects ate identical meals that initially included 250 mg vitamin C/d, and thereafter the dose of vitamin C was gradually reduced to 5 mg/d and maintained for 92 d; there was no effect on 8-oxodG concentrations as assessed by HPLC with electrochemical detection (39). The discrepancy between the studies may be related to the different initial vitamin C concentrations in the subjects or to the method of analysis. However, Rehman et al (42) conducted a study of healthy, nonsmoking subjects stratified by low (50.4 µmol/L) or high (71.9 µmol/L) initial plasma vitamin C concentration. In the subjects with high initial plasma vitamin C concentrations, low doses of vitamin C (60 mg/d) decreased 8-oxodG concentrations, whereas high doses of vitamin C (260 mg/d) had no effect (42). In the subjects with low initial plasma vitamin C concentrations, there was no effect of either a 6- or 12-wk supplementation (42). In a later placebo-controlled study, the 6-wk supplementation trial was repeated, and no difference was observed between placebo and vitamin C supplementation, albeit 8-oxodG concentrations decreased in both groups (44).
Dietary vitamin E supplementation (400 IU/d) to nonsmokers for 8 wk had no effect on the number of SBs or on hydrogen peroxide sensitivity, despite increased vitamin E concentrations in plasma (29). The effect of vitamin E was studied in combination with a diet either high or low (15% or 5% as food energy) in polyunsaturated fatty acids (PUFAs) among nonsmoking men (30). The results of this study indicated that the PUFA content had no effect on hydrogen peroxide sensitivity or on the number of SBs or ENDOIII sites in subjects supplemented with 80 mg vitamin E/d for 4 wk. However, in subjects who were supplemented with only 5–7 mg vitamin E/d, hydrogen peroxide sensitivity and the number of ENDOIII sites decreased when eating a 5% PUFA diet, whereas the same endpoints increased after consumption of the 15% PUFA diet (30). In a similar study, Chen et al (38) found no differential effect on 8-oxodG concentrations between groups receiving a high–vitamin E–low-PUFA diet or a low–vitamin E–high-PUFA diet. However, 8-oxodG concentrations decreased gradually in both groups throughout the total study period of 31 d (38). This probably indicates the effect of subjects entering a controlled dietary intervention study or a seasonal effect.
A study of carotenoid supplementation (lycopene, lutein, or ß-carotene) for 1 wk indicated no effect on hydrogen peroxide sensitivity (33). In the 5 subjects who had a 20% increase in plasma lycopene concentration, lycopene supplementation probably decreased the number of SBs (statistics were not included in the manuscript). Lutein and ß-carotene supplementation did not alter the number of SBs. Consumption of a mixture of - and ß-carotene (15 mg/d) for 12 wk did not affect the number of formamidopyrimidine-DNA glycosylase and ENDOIII sites or 8-oxodG concentrations in lymphocytes (31).
A placebo-controlled crossover study including both smokers and nonsmokers who were given single supplementations of vitamin C (350 mg/d), vitamin E (250 mg/d), ß-carotene (60 mg/d), selenium (80 µg/d), or vitamin C (350 mg/d) and vitamin E (250 mg/d) showed no effect on the number of SBs or on X-ray sensitivity after 4 wk of supplementation (26). 8-OxodG concentrations were influenced by ß-carotene supplementation: smokers had increased 8-oxodG concentrations after supplementation, whereas nonsmokers had decreased concentrations (26). A small study among smokers (3 subjects per group) found decreased 8-oxodG after vitamin E and red ginseng supplementation, whereas ß-carotene and vitamin C had no effect (37). A study in which an enzyme-linked immunosorbent assay was used to detect SBs found decreased hydrogen peroxide sensitivity after supplementation with vitamin C (1000 mg/d) or vitamin E (800 mg/d) for 42 d (28). Consumption of coenzyme Q10 had no effect on the number of SBs, ENDOIII sites, or formamidopyrimidine-DNA glycosylase sites in samples that were analyzed immediately after they were obtained (35).
Multiple antioxidant trials
Dietary supplementation with vitamin C (100 mg/d), vitamin E (280 mg/d), and ß-carotene (25 mg/d) for 20 wk had no effect on the number of SBs (24). However, the number of ENDOIII sites and hydrogen peroxide sensitivity decreased, although at intermediate time points (5 and 10 wk), there was no effect on the number of ENDOIII sites (24). In a study of subjects who were occupationally exposed to tobacco smoke, 8-oxodG concentrations decreased after a 60-d supplementation with an over-the-counter vitamin tablet (3000 mg ß-carotene, 60 mg vitamin C, 30 IU -tocopherol, 40 mg Zn, 40 mg Se, and 2 mg Cu) (41). Of the vitamins supplemented, the plasma vitamin C concentration increased, whereas plasma -tocopherol and ß-carotene concentrations remained unaltered (41). A 6-mo vitamin supplementation trial (500 mg vitamin C/d, 400 IU -tocopherol/d, and 12 mg ß-carotene/d or placebo) did not show any effect of antioxidant supplementation; yet, 8-oxodG concentrations decreased in both groups throughout the trial (more so in the placebo group than in the supplementation group) (43).
Natural food product trials
Studies of natural food products appear to be different from single and multiple antioxidant studies because in the former a wide range of different food products are investigated, and with the exception of carotenoid-rich foods, the food products are difficult to group according to the type of antioxidants that they contain. As for carotenoid-rich foods, subjects who consumed tomato purée (25 g/d for 2 wk) in addition to their normal diet had decreased hydrogen peroxide sensitivity, whereas the number of SBs was unaltered (23). Consumption of other carotenoid-rich food products (tomato juice, carrot juice, and spinach powder) decreased the number of SBs, whereas ENDOIII sites were decreased only by drinking carrot juice; hydrogen peroxide sensitivity was unaltered after consumption of any of the carotenoid-rich foods (25). Unfortunately, neither of these studies was placebo controlled. A sequential 1-wk supplementation with spaghetti sauce (126 g/d), tomato juice (450 mL/d), and oleoresin capsules (2.5 g/d) as sources of lycopene did not produce a significant effect on 8-oxodG concentrations, although the authors of the study argued that 8-oxodG concentrations decreased after the supplementation (40).
Drinking green tea decreased 8-oxodG concentrations in Chinese men who were smokers, and in an American population, drinking green tea decreased 8-oxodG concentrations more in smokers than in nonsmokers (45). Drinking red wine (240 mL/d) decreased 8-oxodG concentrations in 2 groups of men who received either a high-fat or a Mediterranean diet (47). Consumption of rye crisp bread, a source of phytoestrogenic antioxidants, for 2 wk had no effect on the number of SBs or ENDOIII sites (32). Consumption of another source of phytoestrogens, soya milk, decreased the number of ENDOIII sites after 4 wk of dietary supplementation, whereas the number of SBs and hydrogen peroxide sensitivity remained unchanged (22). A 3-wk placebo-controlled trial of vegetable and fruit consumption found that the number of SBs and ENDOIII sites, hydrogen peroxide sensitivity, malondialdehyde concentrations, and protein oxidation in male smokers were unchanged, even though plasma concentrations of antioxidants and total plasma antioxidant capacity were increased (34). A stratification of presupplementation plasma -carotene concentrations into low and high concentrations showed that women with low plasma -carotene concentrations had decreased 8-oxodG concentrations after consuming 12 servings of vegetables and fruit/d for 2 wk (compared with 5.8 servings/d at baseline) (46).
Evaluation of study design and power in short-term trials
To allow a critical review of the present short-term studies, we considered differences in several characteristics as being important for a significant outcome, including the type of antioxidant, the inclusion of smokers, the number of subjects, the ages of the subjects, the length of the supplementation and washout periods, the types of detection (for 8-oxodG), and the strength of the study. Period effects due to seasonal variation or changes in subject behavior during a trial and unrecognized changes in sampling or analysis may have profound effects that may severely bias sequential designs without control groups. Thus, eg, the comet assay shows covariation with solar flux (49). We defined optimal studies as randomized, placebo-controlled studies with parallel designs in which the blood samples from the placebo group were taken at the same time as those from the supplement group. However, parallel two-group (antioxidant and placebo) designs may suffer from baseline differences between the groups that may affect the outcome measures; ie, it is difficult at the start of a study to fully normalize 2 groups for antioxidant status, especially if the groups are small. The second-best choice for a design is a randomized, placebo-controlled crossover design, although it is vulnerable to period and carryover effects and a loss of the statistical power otherwise often inherent in the use of subjects as their own controls. In that respect, the intrasubject variation in the comet assay is no less than the intersubject variation (49), and no statistical power is gained in crossover designs over that of parallel-group designs. Of the parallel, placebo-controlled studies, those studies in which samples were taken after the end of the intervention were considered to have the strongest type of study design. The strength of the study design was scored from 0 to 3 on the basis of the following criteria: inclusion of a placebo group or period (1 point), parallel intervention series of placebo and antioxidant (1 point), and inclusion of samples taken after the end of the intervention (1 point). This design-strength score is outlined in Table 4. To evaluate the relation between the strength of the studies and outcome, we performed a statistical analysis (analysis of variance) with factors, including the biomarker method used, the number of subjects, smoking status, the types of antioxidant (single antioxidant, multiple antioxidants, or natural products), the duration of the intervention trial and washout period, and the strength of study design. The statistical analysis indicated that studies showing protective effects of antioxidants were more likely to have a low design-strength score (P < 0.05; n = 26), whereas other factors were nonsignificant. The mean ± SD strength scores of studies showing protective effects and those showing no effects were 1.1 ± 1.0 (n = 14) and 1.8 ± 0.6 (n = 12), respectively. A significant reduction in the number of SBs was only shown in 1 of 14 studies. For ENDOIII sites, hydrogen peroxide sensitivity, and 8-oxodG, the number of positive and negative results were evenly distributed: 4 and 4, 4 and 7, and 7 and 6, respectively.
View this table:
TABLE 4 . Power analysis of studies and score of study design sorted by endpoint [7-hydroxy-8-oxo-2'-deoxyguanosine (8-oxodG) or strand breaks (SBs)] and study outcome1
Low power in studies also may explain negative findings. To investigate this, we calculated how many subjects would be required to detect a 50% decrease in oxidative DNA damage on the basis of the mean ± SD values for the control or preintervention groups (Table 4). This calculation indicated that the positive and negative studies did not differ (6 of 12 and 6 of 9, respectively) in their power to detect a 50% effect (based on = 5% and ß = 20% for the difference between 2 groups). In fact, power to detect smaller effects, eg, differences of 20–30%, would be desirable.
DISCUSSION
In this review we distinguish between single-dosing and short-term studies of protective antioxidant effects in mononuclear blood cells, lymphocytes, or leukocytes. The short-term studies included intervention periods of weeks or a few months (never more than one-half of a year). Five single-dosing studies were identified, and all showed some protective effects of single antioxidants or natural products (17–21). The effect of a single dose of vitamin C seems to vanish within a few hours, whereas vitamin E and carotenoids appear to show protective effects a little later, possibly because of differences in bioavailability (19).
For short-term vitamin C intervention studies, results from Lunec and coworkers (36, 48) indicated decreased 8-oxodG concentrations; however, this trial was sequential, with the placebo period preceding the period with active treatment. In the proper placebo-controlled trials, no effect of vitamin C was observed with measurement of 8-oxodG (37, 39, 44) or with the comet assay (27). Trials investigating oxidative DNA damage with single carotenoids or vitamin E mainly showed no effect at the high doses (26, 29–31, 33, 37, 38). In the studies of vitamin C in which GC-MS was used for measurement of 8-oxodG, increases and decreases in the level of oxidation of some other DNA bases, including 8-oxoadenine, were also observed (42, 44, 48). However, this method is prone to artifactual oxidation of DNA bases, and until verified by other methods, those data should be interpreted with caution. On the basis of the available data, consumption of a single antioxidant or vitamin for weeks or months apparently provides little detectable effect on the level of oxidative DNA damage in white blood cells.
Combinations of vitamins C and E and ß-carotene had a protective effect after 20 wk of supplementation but no effect after 10 wk (24) or 6 mo of supplementation (43). The length or quality of the intervention period, smoking status, or amount of antioxidants cannot explain this discrepancy. In a study that was not placebo controlled, subjects occupationally exposed to smoking had decreased 8-oxodG concentrations after 6 wk of supplementation (41). Thus, firm conclusions cannot be reached regarding whether consumption of multiple vitamins provides better protection against oxidative DNA damage than does single-vitamin supplementation.
Ingestion of natural food products produced protective effects as assessed by the comet assay (22, 23, 25), but one study of vegetable supplementation did not show an effect (34). Unfortunately, two of the positive studies were not placebo controlled (23, 25). With regard to 8-oxodG concentrations, green tea and red wine had a beneficial effect (45, 47), whereas Rao and Agarwal (40) found a nonsignificant beneficial effect of lycopene-rich diets. In general, natural food products are apparently more likely to achieve beneficial antioxidant effects than are supplements. However, any conclusion should be made with caution because several of the studies were not placebo controlled. Furthermore, it is puzzling that there is no general pattern regarding the type of endpoint for which significant results occur, although it appears that the number of enzyme sites, 8-oxodG concentration, and sensitivity to hydrogen peroxide or ionizing radiation are more sensitive endpoints than the number of SBs. It is possible that various types of antioxidants exert different effects that are detected by different endpoints. The lack of a general pattern does not appear to be due to chance because chance should produce a maximum of 2 significant protective effects (given a total of 50 endpoints measured and on the basis of a two-tailed t test at a 5% significance level). A positive publication bias could be considered but is probably not likely because short-term intervention studies typically are large studies with many endpoints from which some information eventually becomes available.
It is possible that a preventive effect of antioxidants is seen only when accompanied by an oxidative-stress situation. Oxidative stress occurs under certain physiologic conditions, eg, after exhaustive exercise; in pathologic conditions, such as immunologic diseases; or as a result of environmental factors (smoking, air pollution, or radiation of the ultraviolet A waveband) (1). Interestingly, Hartmann et al (50) showed that vitamin E supplementation (1200 mg/d) for 14 d inhibited exercise-induced SB formation. It is possible that the presence of unrecognized oxidative stress may provide a greater likelihood of detecting protective effects. It is almost impossible to identify such disguised oxidative stress in a study population, and therefore such considerations remain elusive. Although we hypothesized that beneficial effects of antioxidant supplementation would be more likely among smokers than among nonsmokers, the results of intervention studies did not confirm our hypothesis. The effect of smoking as detected by the comet assay is in itself questionable and is probably influenced by regional smoking habits and the brands smoked (51). The diet consumed also plays a role: Jenkinson et al (30) reported that the PUFA content appeared to be a strong determinant of a vitamin E–related response.
Patients with chronic inflammatory diseases (rheumatoid arthritis and systemic lupus erythematous) and infection (hepatitis C) have been shown to have elevated concentrations of 8-oxodG in circulating leukocytes (52, 53). Diabetic patients also have markedly higher than normal amounts of oxidative DNA damage in terms of the 8-oxodG concentration and number of SBs and ENDOIII sites in leukocytes (54–56). These diseases therefore may serve as excellent models for oxidatively stressed subjects. Data from diabetic patients show conflicting effects of dietary vitamin E (29, 57) and a protective effect of a diet high in flavonols (58). To the best of our knowledge, there have been no antioxidant intervention trials that have investigated the effects on oxidative DNA damage in patients with chronic inflammatory diseases or infections. It is conceivable that such intervention trials will prove useful for assessing the beneficial effects of antioxidants in subjects with oxidative stress related to diseases.
A critical review of short-term studies of antioxidant effects indicates that study designs need to be stronger, particularly with respect to parallel control groups and statistical power. For instance, period effects were evident in 3 studies that reported decreased 8-oxodG concentrations during the study period (38, 43, 44). In addition, seasonal variation was observed in biomonitoring studies using the comet assay (51). Mixed results were obtained from the short-term intervention studies, whereas single-dosing studies unanimously showed protective effects of antioxidants, as detected by the comet assay and measurement of 8-oxodG, in at least one blood sample taken within 24 h. An explanation for this discrepancy may be that the effects of the antioxidants are too short-lived to be detected (by chance) in one blood sample taken during the intervention.
REFERENCES