点击显示 收起
1 Dr Rajendra Prasad Centre for Ophthalmic Sciences, All India Institute of Medical Sciences, New Delhi, India
2 Sarojini Devi Eye Hospital, Hyderabad, India
3 Lions Aravind Institute of Community Ophthalmology, Madurai, India
4 London School of Hygiene and Tropical Medicine, London, UK
Correspondence to:
Professor Astrid E Fletcher
Department of Epidemiology and Population Health, Keppel Street, London School of Hygiene and Tropical Medicine, London WC1E 7HT, UK; astrid.fletcher@lshtm.ac.uk
Accepted for publication 1 July 2004
ABSTRACT
Objective: To develop and evaluate the acceptability, reliability, validity, and responsiveness of the Indian vision function questionnaire (IND-VFQ).
Methods: Problem statements from previous qualitative studies were reduced to a 45 item interviewer administered questionnaire representing three a priori domains (general functioning, psychosocial impact, and visual symptoms) which was evaluated in patients with cataract (n = 420), glaucoma (n = 120), diabetic retinopathy, or age related macular degeneration (n = 120) and normal controls (n = 120). Standard methods were used for item reduction and to evaluate psychometric properties.
Results: Psychometric item reduction produced a 33 item questionnaire. Psychometric evaluation showed that two of the three scales (psychosocial impact and visual symptoms) had good acceptability, and that all three scales showed high internal consistency (alpha >0.80; item-total correlations 0.54–0.86) and test-retest reliability (>0.89). All three scales showed moderate evidence of convergent and discriminant validity. Responsiveness, assessed in cataract patients (n = 120) before and after surgery, was good for all three scales (effect sizes >1).
Conclusions: The IND-VFQ33 is a psychometrically sound measure of vision function addressing a gap in patient defined measures of vision function developed in populations living in low income countries.
Abbreviations: IND-VFQ, Indian vision function questionnaire
Keywords: vision function questionnaire; psychometric evaluation; blind; visually impaired; low income countries
This paper describes the development and psychometric evaluation of the Indian vision function questionnaire (IND-VFQ). Our earlier paper1 presented the rationale for the questionnaire and described its content development. In brief, there is a lack of patient based measures of vision related quality of life developed for use in low income country settings and populations. The development of the IND-VFQ sought to redress this gap. With the current and predicted global burden of blindness highest in low income countries,2 it is imperative that such measures are both available and relevant to health practitioners and their patient populations. The IND-VFQ is a new, patient based measure of vision related quality of life that was developed and field tested in three eye institutions in India.
METHODS AND PARTICIPANTS
Instrument development
In a previous paper we described how patients participated in the content development of the IND-VFQ by identifying the impact their vision problems had on their daily lives and psychosocial wellbeing.1 Approximately 5000 elicited problem statements grouped into 18 domain areas were reduced to 210 statements through a process of merging statements with a similar semantic meaning and context. We undertook a further process of item reduction by eliminating statements with low frequency counts (defined as problems reported by less than 10% of the patient population). This process resulted in 103 statements for inclusion in a draft questionnaire for preliminary field testing.
Visual symptoms, psychosocial impact, and general functioning questions were assessed by a four point response scale (1 = not at all, 2 = a little, 3 = quite a bit, 4 = a lot) that was identical to that used in the Madurai Intra-Ocular Lens Study quality of life instrument.3,4 An extra response category (which scored 5) was included in the general functioning domain to reflect the respondent’s inability to carry out the activity because of vision impairment. The final field test version of the questionnaire also included one open ended question that asked patients if there were any other ways that their eye problem affected their lives. This query did not yield any new information that was not already covered in the questionnaire.
After preliminary field testing for face validity and acceptability, the 103 item IND-VFQ was administered to a total of 96 patients equally divided across the three study centres: Dr Rajendra Prasad Centre for Ophthalmic Sciences, Delhi (RPC), Sarojini Devi Eye Hospital, Hyderabad, and Aravind Eye Hospital, Madurai. Sixty cataract patients (30 recruited via community based clinics linked to each study hospital and 30 hospital inpatients); 18 hospital outpatient glaucoma patients; and 18 hospital outpatients with diabetic retinopathy or age related macular degeneration (case definitions are described elsewhere).1 Additionally, 30 people without ocular pathology (defined as visual acuity = 6/12 in the presenting better eye) who constituted a normal comparison group, were also administered the draft questionnaire. Participants in the "normals" comparison group were recruited from the patient "escort" population and were comparable with the patient groups on age, sex, and other relevant sociodemographic variables.
We examined endorsement frequencies for the 103 item questionnaire for the group as a whole (excluding people in the normal comparison group) and separately for the three conditions. Questions were eliminated from the draft questionnaire on the basis of the following criteria:
More than 10% of respondents endorsed the response category "Don’t do this for other reasons." A total of 18 questions were eliminated on the basis of this criterion.
More than 30% of respondents selected the response "Not at all." A total of 40 questions were eliminated on the basis of this criterion.
This process resulted in a 45 item questionnaire with three scales: general functioning (28 questions), psychosocial impact (nine questions), and visual symptoms (eight questions). These three scales were adopted because they captured the semantic flavour of patient identified problems of function, behaviour, feelings, and symptoms rather than a clinically derived taxonomy of problems. Each scale was scored using a simple addition of the values according to the response scales. For example, general functioning scores could range from 28 (no problems on all items in this domain) to 140 (maximum responses on all items). Before questionnaire administration, all language versions of the questionnaire satisfied independent translation and back translation from English into local mother tongue languages of Hindi, Telegu, and Tamil.
Field testing the 45 item IND-VFQ
Study population
We aimed to recruit 780 participants, including 660 patients who satisfied predefined clinical criteria (described elsewhere1) for cataract, glaucoma, and retinal conditions (diabetic retinopathy and macular degeneration) in equal numbers from the three study centres, all leading ophthalmic hospitals, located in New Delhi, Hyderabad, and Madurai, and 120 normal controls. The target number of participants were cataract (n = 420), of which 120 were to be recruited through hospital outpatient departments before surgery and 300 through community outreach camps before surgery; glaucoma (n = 120), recruited through hospital outpatient departments; retinal conditions (n = 120), recruited through hospital outpatient departments; and normal controls (n = 120). All cataract patients underwent extracapsular cataract extraction surgery plus intraocular lens implant. Patients who participated in the content development phase and cataract patients scheduled for intracapsular rather than extracapsular extraction were excluded. To evaluate the test/retest reliability of the IND-VFQ, 360 patients (120 hospital recruited cataract patients, 120 glaucoma patients, and 120 patients with retinal conditions) were scheduled for a repeat administration of the questionnaire 1–2 days after the baseline assessment. Responsiveness was evaluated only in cataract patients by administering a second questionnaire 5–8 weeks after intraocular lens surgery. Informed verbal consent was obtained for all study participants. Ethical approval was obtained from respective governing authorities at each study centre.
Questionnaire administration
Interviewers from each study centre attended a training workshop conducted by experienced interviewers from the Tamil Nadu centre. Interviewers were familiarised with the principles of questionnaire administration and supervised in the conduct of practice interviews. During data collection, principal investigators at each centre observed random interviews to ensure their correct conduct, and quality control in questionnaire completion. For glaucoma and retinal patients, baseline interviews were completed before patients had their ophthalmic consultation. Cataract patients completed baseline questionnaires after diagnosis of their eye condition but before surgery. Where possible, the same interviewer who conducted the baseline interview administered the retest and post-surgery questionnaire. All questionnaires were sent to the central coordinating office (Dr Rajendra Prasad Centre) for data entry. SPSS version 11.01 was used for data analyses.
Psychometric evaluation
Standard psychometric methods and criteria were used for item reduction and to evaluate acceptability, reliability, validity, and responsiveness (table 1).5,6,7,8,9,10,11,12 All psychometric analyses were performed on the pooled dataset from all three centres. This approach to performing analyses on pooling data across language versions has been used in previous psychometric validations of other commonly used international outcome measures.8 Item reduction analyses were performed on pooled data for the three patient groups but excluded normal controls. Psychometric analyses were carried out by SS and SG with expert advice from DL.
Table 1 Psychometric tests and criteria (adapted from Lamping et al8)
RESULTS
At baseline, the 45 item IND-VFQ was administered to all 660 patients and 120 controls by the trained interviewers at each study centre (100% response rate). Study participants tended to be elderly with chronic vision impairment, including blindness, often educated to primary/secondary school level yet inclusive of a sizeable illiterate population which is representative of the general population, and mixed in terms of caste composition (table 2). Repeat questionnaires were obtained for 97% (n = 350) of hospital recruited patients (120 cataract, 114 glaucoma, and 116 retinal). Post-surgery questionnaires were obtained for 87% of cataract patients (111 hospital recruited and 255 community recruited). The main reason for non-completion of the questionnaire was failure of the patient to attend.
Table 2 Sociodemographic characteristics of participants
Item reduction
Item analyses were performed to evaluate the IND-VFQ45 for missing data, redundancy and item convergent/discriminant validity. Four items (all from the general functioning scale) were eliminated because they had 5% missing data. A further eight items were eliminated, six because of inter-item correlations of >0.80 (three from general functioning and three from psychosocial impact) and two for failing tests of item convergent/discriminant validity (one from psychosocial impact and one from visual symptoms). Item reduction analyses thus produced a 33 item questionnaire (IND-VFQ33). All subsequent psychometric tests were performed on the IND-VFQ33 scales: general functioning (Q1–Q21), psychosocial impact (Q22–Q26), and visual symptoms (Q27–Q33).
Psychometric evaluation of the IND-VFQ33
Scores showed the full range from minimum to maximum. The proportion of missing data on each scale was acceptable for psychosocial impact and visual symptoms scales (5.3% and 1.4% respectively) but higher for general functioning (14.5%). The main reason for missing data in the general functioning scale was the respondent did not consider the question relevant. Floor/ceiling effects were low (1% or less) for general functioning and visual symptoms but 6.9% for psychosocial impact. Similarly, ceiling effects tended to be higher for the psychosocial scale (15.7%) compared to general functioning (0.2%) and visual symptoms (6.9%). Skewness values were low (range –0.16 to +0.09).
An examination of the reliability of the IND-VFQ33 showed good internal consistency and test-retest reliability (table 3). Cronbach’s alpha coefficients indicated high internal consistency for all scales; all values exceeded the minimum criterion of 0.70 for scales and every item in each scale passed the item-total criterion of 0.40. For test-retest reliability, Pearson’s correlation coefficients for all scales were 0.90 and satisfactorily passed the threshold requirement of 0.80. As correlation coefficients can be misleading when evaluating the relation between two measures, we also applied the Bland-Altman method.13 The plots showed no evidence of bias as most of the mean differences between the two measures were close to 0.
Table 3 Internal consistency and test-retest reliability of the IND-VFQ33
Construct validity was examined both within scale and in comparison with other measures. The IND-VFQ33 shows good internal consistency as demonstrated by high item-total correlations and high alpha coefficients (table 3), indicating that a single construct is being measured and that items can be combined to form summary scores. Analyses of intercorrelations between scales followed the expected pattern: all three scales are intercorrelated (>0.65). Construct validity was also assessed by comparison with other measures. The IND-VFQ33 shows moderate convergent validity when compared with visual acuity in the better eye, (correlations of between 0.50 to 0.57 for the three scales). Results for discriminant validity were mixed when judged against sociodemographic measures. Low correlations (all were |0.15|) between the IND-VFQ33 and age and sex suggest that these variables do not influence responses. However, there was evidence that responses varied according to education level and whether the participants lived in urban/rural areas (correlations ranged from |0.33| to |0.58| for education and urban/rural). These differences remained after adjustment for other variables including eye condition. Table 4 shows means and tests of differences for IND-VFQ33 scores for each of the clinical groups and the normal comparison group. Multiple regression analyses controlling for age, sex, education, urban/rural, and caste were used to test differences. For each clinical condition, all scales differentiated significantly between patients in the three clinical conditions and normal controls, providing support for construct validity.
Table 4 Between group (eye disease groups and controls) differences in the IND-VFQ33 scales*
IND-VFQ33 scores showed improvements after surgery, indicating good responsiveness (table 5). Effect sizes are large (all exceed 1.00) and are similar to those reported in the MIOLS study for the group receiving IOL surgery.4
Table 5 Responsiveness of the IND-VFQ33 in cataract surgery patients (n = 366)
DISCUSSION
The Indian vision function questionnaire 33 item questionnaire (IND-VFQ33) (table 6) directly captures patients’ experiences of their vision problems upon their daily lives. Questionnaire items are classified into one of three scales (general functioning, psychosocial impact, and visual symptoms). Developed as an interviewer administered questionnaire and taking an average 20–25 minutes to complete, the IND-VFQ33 is suitable for use in populations of mixed literacy levels and short enough to keep respondent burden to a minimum. Subsequent psychometric evaluation found the IND-VFQ33 to be an acceptable, reliable, valid, and responsive measure of vision function.
Table 6 Indian vision function questionnaire (33 item)
Although the IND-VFQ33 shows satisfactory psychometric properties, there are a number of caveats to consider when evaluating the psychometric evidence. Firstly, as the test-retest interval for determining instrument reliability was 1–2 days, respondents’ recall of their answers from the initial questionnaire administration may have led to an overestimate of test-retest reliability.
Secondly, although the IND-VFQ33 demonstrated good convergent validity when compared to measures of visual acuity and, importantly, distinguished between people with different types of eye problems including those with no eye problems, we were unable to undertake a head to head comparison with other measures of quality of life because of the limited availability of such questionnaires validated for use in India. Future evaluation should include a comparison with the vision related quality of life instrument developed for use in the MIOLS study3—bearing in mind that unlike the IND-VFQ33, the latter is based solely on expert opinion rather than new qualitative data from patients. It is also advisable to control for education and urban/rural effects in IND-VFQ33 data analyses because responses appear to be influenced by these factors.
Thirdly, the IND-VFQ33 produces separate summary scores for the three scales rather than an overall total score. This is because it is methodologically unsound to simply sum raw scores in a questionnaire such as the IND-VFQ, which includes items with a varying number of response scale options. Yet in order to make direct comparisons of the three summary scores when the IND-VFQ33 is used in future studies, and for ease of interpretation, it would be appropriate to standardise the three summary scores to a common 100 point scale to produce a summary score. Future research with IND-VFQ33 could consider the feasibility of creating an overall total summary score using formal methods of standardised scores such as z score equivalents.
Fourthly, instrument responsiveness was confined to cataract patients because cataract surgery provides a clearly defined treatment intervention of known efficacy which the instrument should and did detect. There is scope to evaluate the responsiveness of IND-VFQ33 in other clinical conditions (for example, refractive error).
Fifthly, the IND-VFQ33 is an interviewer administered questionnaire. Great care was taken in our study to train and monitor interviewers in the principles and conduct of questionnaire administration. Potential users of the questionnaire should therefore also be aware of the importance of interviewer training and quality assurance.
Finally, it is envisaged that IND-VFQ33, which has been shown to be a practical and scientifically sound measure, will be used to evaluate vision related quality of life in clinical trials, epidemiological studies, or clinical audit throughout India and possibly south Asia, where social, cultural, and lifestyle factors may be more similar than different. As with other instruments measuring vision related quality of life,14,15 it is likely that future research efforts will refine the IND-VFQ33. Any subsequent modifications to the IND-VFQ33, including changes in format, content, or translation into other languages, would need to be re-evaluated for psychometric properties in an independent sample.
ACKNOWLEDGEMENTS
We would like to thank the participants who took part in this study, and the following people and organisations.
REFERENCES
Murthy GVS, Gupta SK, Thulasiraj RD, et al. The development of the Indian vision function questionnaire: questionnaire content. Br J Ophthalmol 2005;89:498–503.
Thylefors B, Négrel A-D, Parajasegaram R, et al. Global data on blindness. Bull World Health Organ 1995;73:115–21.
Fletcher AE, Ellwein LE, Selvaraj S, et al. Measuring vision function and quality of life in patients with cataract in southern India. Report of instrument development. Arch Ophthalmol 1997;115:767–74.
Fletcher A, Ellwein LB, Vijaykumar V, et al. The Madurai Intraocular Lens Study. III. Visual functioning and quality of life outcomes. Am J Ophthalmol 1998;125:26–35.
Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002;11:193–205.
Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 2nd ed. Oxford: Oxford University Press, 1995.
McDowell I, Jenkinson C. Development standards for health measures. J Health Serv Res Policy 1996;1:238–46.
Lamping DL, Schroter S, Marquis P, et al. The Community-Acquired Pneumonia Symptom questionnaire: a new, patient-based outcome measure to evaluate symptoms in patients with community-acquired pneumonia. Chest 2002;122:920–9.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297–334.
Cohen J. Statistical power analysis for the behavioral sciences Rev, ed. New York: Academic Press, 1977.
Hays RD, Hayashi T. Beyond internal consistency: Rationale and users’ guide for Multi-Trait Analysis Program on the microcomputer. Behav Res Methods, Instruments and Computers 1990;22:167–75.
Ware JE, Harris WJ, Gandek B, et al. MAP-R for Windows: multitrait/multi-item analysis program-revised user’s guide version 1. Boston, MA: Health Assessment Lab, 1997.
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986:307–10.
Mangione CM, Lee PP, Pitts J, et al. Psychometric Properties of the National Eye Institute Visual Function Questionnaire (NEI-VFQ). Arch Ophthalmol 1998;116:1496–504.
Mangione CM, Lee PP, Gutierrez PR, et al. Development of the 25-Item National Eye Institute Visual Function Questionnaire. Arch Ophthalmol 2001;119:1050–8.