点击显示 收起
1 From the Department of Nutrition and the Office of Educational Development, Schools of Medicine and Public Health, University of North Carolina at Chapel Hill.
2 Supported by grants from the National Cancer Institute (R25CA65474), the US Department of Agriculture (96-38411-2826), and the American Institute for Cancer Research. 3 Address reprint requests to M Kohlmeier, Department of Nutrition, CB#7400, University of North Carolina, McGavran Greenberg Hall, Chapel Hill, NC 27599-7400. E-mail: mkohlmeier{at}unc.edu.
ABSTRACT
Background: Computer-based instruction has been introduced at the University of North Carolina at Chapel Hill to augment its nutrition course for first-year medical students. Seven program modules have been completed; 2 more are planned. Each module explains the biochemistry and physiology of nutrition through interactive lessons, exercises, and a video case study.
Objective: The goal of this study was to evaluate the instructional efficacy and acceptability of the nutrition and cancer module when used by first-year medical students.
Design: The module was used by 163 first-year medical students at the university's medical school as an obligatory component of the nutrition course. Before and after using the module, students were asked to answer multiple-choice questions concerning their knowledge and attitudes; each question had 5 possible answers.
Results: On average, students spent 3 h studying the lessons. The percentage of correct responses to 20 knowledge questions increased from 22% before the module was used to 86% immediately after its use. When a randomly selected subsample of 25% of the students took the same test 3 mo later, they answered 62% of the questions correctly. The increase in the percentage of students who felt prepared to provide advice regarding nutrition's role in cancer prevention (from 5.7% to 66.9%) suggested a successful subjective learning experience. Neither the students' initial level of interest in cancer nutrition nor their acceptance of computer-based instruction was related to learning outcome.
Conclusions: The tested module is a useful and effective aid for teaching nutritional principles of cancer prevention. The evaluation strategy helped identify areas for instructional improvement.
Key Words: Computer-based instruction • cancer prevention • instructional effectiveness • nutrition education • retention testing • Nutrition and Medicine curriculum series • medical school
INTRODUCTION
To a large extent, life expectancy, the onset and severity of most chronic diseases, and expenditures for health care are determined by long-term dietary habits. Physicians are uniquely well positioned to convey to their patients the importance of prudent dietary choices. However, they rarely receive formal training in nutrition. For example, fewer than one-third of current medical students are required to complete any formal nutrition course (1, 2), and those students attending a medical school that requires nutrition education receive <6 h of instruction on average (3). A dearth of qualified instructors and the already overburdened medical school curriculum are offered as reasons for the lack of nutrition education given to medical students (2).
Convinced of the value of nutrition instruction, a group of physicians, nutritionists, and educators at the University of North Carolina is developing a self-administered, computer-based curriculum in nutrition for first- and second-year medical students (4). The purpose of this study was to determine the efficacy of the then newest module, Nutrition and Cancer, especially when used by first-year medical students.
METHODS
Study design
All incoming first-year medical students (n = 163) at the University of North Carolina School of Medicine were required to complete the Nutrition and Cancer module as part of their nutrition-course requirements. Tests consisting of multiple-choice questions were completed by the students before and after completion of the module so we could determine the extent of knowledge they acquired and changes in their attitudes related to module use.
As detailed below, 20 questions for the evaluation of module efficacy were selected on the basis of an itemized analysis of a pilot study conducted 1 y earlier. For the initial test, 40 questions of similar format and scope were added to minimize recognition of questions in the later tests. In addition, subjective attitudinal questions were used with response choices based on a 5-point Likert scale. The entire set was sent to each student by e-mail with the request to reply by e-mail within 2 d. The accompanying message informed the students that they could use the 60 questions as a study guide for later exams.
Students had 2 wk to use the Nutrition and Cancer module. At the end of this period, the 20-question test, with changes in both the sequence of questions and response choices, was administered as a written exam. The students' scores on the exam counted toward their grade for the course. Students were asked again to rate their attitudes and perceived knowledge on a 5-point scale.
Two weeks later, students took a midterm exam that contained 2 independently generated questions related to the topic of cancer and nutrition. Performance on the midterm exam was compared with performance on the initial and second tests. Three months after the second test, 40 randomly selected students were asked to participate in a retention test consisting of the same 20 questions with both the sequence of questions and the order of response choices rearranged. This retention test, like the second test, was given in a controlled, written format.
Teaching materials
The material on cancer and nutrition was taught in an interactive, computer-based format that combined lessons on biochemistry, epidemiology, and practice with the video presentation of clinical information. Major topics covered by the Nutrition and Cancer module are given in Table 1. Learning objectives and key points were provided to the students as separate printable files.
View this table:
TABLE 1.. Contents of the Nutrition and Cancer module
The presented case described how a family physician counseled the son of a patient with colon cancer about the son's dietary risk factors. Six video segments of a total duration of 37 min introduced the topic, provided pertinent clinical information, and concluded the module. Several interactions within the case presentation explored the application of nutritional concepts in clinical practice, showed how unfavorable dietary habits can be identified and modified, and explained what criteria might be useful for the critical evaluation of health claims for fad diets and nutritional supplements. In one of these interactions, for example, the user has the task of evaluating a food-frequency questionnaire completed by a patient and of identifying specific dietary habits that increase the cancer risk of this patient. If warranted on the basis of the patient's responses to the questionnaire, further explanations about desirable food choices are provided by the patient's family physician. The video segments were integrated with 5 lessons that explained molecular events in carcinogenesis, tumor initiation, tumor promotion and progression, nutritional epidemiology, and dietary guidelines. The lessons incorporated 104 screens with text, graphics, and animations that explained on a molecular and cellular level how cancer develops, how specific food components and dietary practices enhance or inhibit cancer development, the principles of epidemiologic research and their application to decision- making, and current recommendations for the reduction of cancer risk. The practical relevance of this information was highlighted by presenting 18 clinical applications of this basic scientific information. Each of the lessons also used sets of 3 short questions and text-based, triple-tiered case discussions with remedial feedback to emphasize important points. For example, carcinogen inactivation is explained as follows: one screen outlines the enzymatic conjugation of xenobiotics in the liver by a phase II enzyme, a second screen explains how isothiocyanates and other compounds in foods can reduce DNA exposure to carcinogens by inducing hepatic phase II enzyme activity, and a third screen then reinforces the concept with a simple question and remedial answer.
Generation of test items
More than 100 multiple-choice questions, each with 5 possible answers, were developed and evaluated by several internal and external reviewers. The questions were grouped into 20 sets based on topics covered by the teaching module. A preliminary 60-item test set was generated by selecting the best 3 questions from each group. Eleven volunteers (first-year dental students) answered the questions on the preliminary test, both before and after the material outlined in Table 1 was presented in a conventional lecture. On the basis of the students' responses, some of the questions and answers were modified and the revised set was tested in a group of first-year medical students at the Duke University (n = 11) and Wake Forest University (n = 6) medical schools. These students had volunteered to answer the modified test questions before and after using the Nutrition and Cancer module. The test results were used to select the best questions from each triple set of related questions for the final 20-question set used in the current study.
Statistical analysis
The data were analyzed by using SAS for PC 6.03 (SAS Institute, Cary, NC). The statistical significance of instructional effects was calculated by analysis of variance. To detect associations between answers to the questions and test outcomes, correlation coefficients were calculated.
RESULTS
Knowledge assessment
Most students (156 of 163) completed the first and second tests within the given time frame. The remainder of the students were excused because of illness during either testing period. The overall proportion of correct answers increased from a mean (±SD) of 22.3 ± 9.9% to 86 ± 10.7% on the graded second test, but to only 61.2 ± 13.1% (n = 38) on the voluntary retention test taken 3 mo after the use of the module (Table 2). The reliability of the second test and of the retention test was 0.58 and 0.47, respectively.
View this table:
TABLE 2.. Percentage of correct answers in the initial, second, and third (retention) tests
The students had to take a written biochemistry exam 2 wk after the second test. The multiple-choice questions were of the same format as those on the second test, but were formulated independently by the instructors of the course. Two questions addressed the topic of nutrition and carcinogenesis. The 115 students who answered both of these questions correctly had a significantly greater gain in score between the initial test and the second test than did the 37 students who answered only one of these questions correctly and the 2 students who answered both questions incorrectly (P < 0.002). Two students did not take the test.
The percentage of correct answers given before instruction was not better than would be expected if the students had chosen 1 of the 5 possible answers randomly, which indicates that students entered the study with minimal or no knowledge of the subject matter. Furthermore, reliability was very low (0.05) and the scores on the initial test were not related to the scores on the second test (r = 0.03). A review of the answers suggested that most students attempted to provide correct answers and selected their answers on the basis of commonly held fallacies. One question regarding the evolution of cancer cells was answered correctly by more students (39%) than would be expected (20%) by chance alone, presumably because the students knew some of the underlying concepts from a concurrent pathology course. The preinstructional knowledge manifested in the better than random (20%) scores on 4 of the 5 epidemiology questions could not be traced to medical school instruction.
About half of the lesson content addressed principles of carcinogenesis and mechanisms involving nutrients and cooking practices (Table 1). Accordingly, 10 of the test questions were designed to assess learning and retention of the involved concepts and key points. The percentage of correct answers increased from 22% on the first test to 86% on the second test (P < 0.001) and was 59% on the retention test given 3 mo later. Two of the questions uncovered an instructional weakness regarding the action of phase I and phase II enzymes. The students were clearly familiar with the concepts of enzymatic carcinogen activation and detoxification and the role of specific nutritional factors modulating these activities, but they still were confused about the respective terminologies. The results also showed that use of the module succeeded in correcting some common fallacies. For example, before instruction, more than one-third of the students thought that tocopherol accumulated to toxic concentrations in the liver and most of the remaining students thought that excess vitamin E impaired LDL clearance, damaged DNA of the intestinal mucosa, or was cleaved to generate retinol. After using the module, 91% of the students knew that ascorbic acid is required for the regeneration of tocopherol and that vitamin E can become a prooxidant in persons with poor vitamin C status. Three months later, nearly as many students (84%) had retained this knowledge.
An important objective of the module was to improve the students' ability to evaluate reports and recommendations based on epidemiologic studies. Five of the questions assessed the knowledge gained in this area. The percentage of correct answers improved from 27% to 88% from the first to the second tests and was 65% 3 mo later on the retest. The results showed that most students (95%) recognized the design used in a particular study, selected suitable methods for dietary assessment (87%), and listed criteria of causality (78%). The students performed equally (88% correct answers) well in answering questions that required more critical thinking, eg, What would be appropriate advice to give a patient who wonders about a news item on purported beneficial effects of a food? Another question asked students to identify a potential confounder of a diet-cancer association (92%).
Another 5 questions addressed the scientific foundation of current treatment regimens and cancer-prevention recommendations. The percentage of correct answers increased from 23% on the first test to 87% on the second test and was 61% on the retest 3 mo later. The results again showed that some common misconceptions were corrected by the instruction. For example, one-third of the students initially thought that carotenoids in fruit and vegetables inactivate nitrosamines through their antioxidant properties or by forming complexes; however, after instruction almost all of the students knew that ascorbic acid in these foods inhibits nitrosamine formation in the stomach.
Self-assessment of skills and attitudes
During the first and second tests, the students were asked to rate their current skills and their expectations on a scale from 1 (strongly disagree with statement) to 5 (strongly agree with statement). Most students agreed or strongly agreed that it was essential for them to know about nutritional issues involving cancer, and even more of them thought so after completing the module (89.4% compared with 78.2%; P < 0.05).
After completing the module, most of the students considered themselves knowledgeable about issues involving cancer (59.8% compared with 4.5% before completing the module; P < 0.01); even more of the students thought that they could now provide advice regarding nutrition's role in cancer prevention (66.9% compared with 5.7% before completing the module; P < 0.01). However, not as many students felt sufficiently prepared to advise cancer patients about nutrition therapy after studying the material, but the increase was still significant (35.1% compared with 5.2%; P < 0.01).
There was no significant relation between the students' level of interest in the topic and the gain in test scores. Before completing the Nutrition and Cancer module, only 16.7% of the students responded that the use of computer modules is an effective way to learn. The students' responses did not predict the actual learning effect, as measured by the improvement in scores from the first to the second test.
Usage tracking
Patterns of use and duration of computer-based instruction (CBI) were recorded on floppy disks. Of 156 students, 104 (67%) returned the disks with readable files after completing the course; 3 students returned disks with damaged files. Of those who returned disks with readable files, 81 (77.9%) reviewed the entire module. Two students did not complete any of the module's 28 segments, 8 missed between 6 and 23 segments, 4 did not complete between 3 and 5 segments, and 9 missed 1 or 2 segments. There was no clear pattern to the omissions. Segments 2–9, which covered the clonal nature of cancer and the biochemistry of carcinogenesis, were missed most often (6.7%), possibly because the students thought they already knew this basic material.
Recorded use time included all idle times and time spent reviewing lessons and video cases, using the reference materials, and completing the practice questions. Students who completed all sections but took only one or no practice exam (n = 21) spent a median of 3.3 h (range: 1.2–6.1 h) on the module compared with 5.1 h (1.2–15.5 h) for those (n = 59) who took all 3 practice exams.
DISCUSSION
CBI is becoming an increasingly attractive and viable resource for postgraduate education. Generally, CBI has been found to be as effective as the traditional lecture format (5–11). The same was reported for programs teaching principles of carcinogenesis and cancer risk factors (12, 13). Applications that incorporate video interactions, case simulations, and problem-solving formats have been found to be particularly effective.
Most of the students rated the Nutrition and Cancer module as useful or very useful; only 20.3% did not. Nonetheless, many students felt that the time they spent on the program was disproportionate to the grades derived from it. The median 3.3 h that was spent by those students who reviewed all sections but did not take all 3 practice exams may provide the best outward estimate for the time actually needed to complete the lessons, interactive exercises, and video cases. This value compares favorably with the 3–4 h that students would be expected to spend studying this material in a conventional lecture format (1–2 h of in-class instruction and 2 h of reading and exam preparation). As was observed by others (14), a negative attitude about the effectiveness of CBI did not appear to be a barrier to effective learning in this group of freshman medical students. Positive reception to the instruction was indicated by some students' comments that the material increased their awareness of the effect of diet on cancer risk and that they had modified their diets on the basis of the newly learned principles.
A potential strength of CBI lies in its standardized presentation, which provides opportunities to test instructional effectiveness and to correct detected weaknesses, thereby systematically improving the quality of the instruction. We identified sections in the module that did not convey information as well as intended. For example, we found that students had difficulty linking the cancer-preventive potential of cabbage to the induction of carcinogen-conjugating activity (phase II enzymes); many mistakenly thought that isothiocyanates inhibit phase I enzyme activity. Therefore, the most recent version of the Nutrition and Cancer module reflects modifications to several pages, which clarified the relative roles of phase I and phase II enzymes in carcinogen metabolism.
Currently, 7 instructional modules have been completed, each providing an average of 3 h of instruction; additional modules are in preparation (Table 3). Medical schools can use the modules to augment their curriculums or as a framework for new nutrition courses. At this time, more than one-third (48 of 129) of medical schools in the United States use modules of the Nutrition in Medicine curriculum series for nutrition education of first- or second-year medical students. Many of these schools have provided us their results from both formal (15) and informal testing as well as specific comments from instructors. These comments have been invaluable for the further improvement of the modules. This kind of integrated process of module modification and formal efficacy testing will be necessary to generate instruction that ensures that future generations of physicians have the knowledge for appropriate nutritional interventions and the motivation to apply what they have learned.
View this table:
TABLE 3.. Modules of the Nutrition in Medicine curriculum series1
REFERENCES