IOVS Journal of Neuroscience
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

(Investigative Ophthalmology and Visual Science. 2004;45:3919-3928.)
© 2004 by The Association for Research in Vision and Ophthalmology, Inc.
DOI:  10.1167/iovs.04-0208
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (22)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Stelmack, J. A.
Right arrow Articles by Massof, R. W.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Stelmack, J. A.
Right arrow Articles by Massof, R. W.

Psychometric Properties of the Veterans Affairs Low-Vision Visual Functioning Questionnaire

Joan A. Stelmack,1,2,3 Janet P. Szlyk,3,4 Thomas R. Stelmack,2,3,4 Paulette Demers-Turco,5,6 R. Tracy Williams,2,7,8 D’Anna Moran,1 and Robert W. Massof9

1From the Blind Rehabilitation Center, Edward Hines VA Hospital, Hines, Illinois; the 2Illinois College of Optometry, Chicago, Illinois; the 3Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago College of Medicine, Chicago, Illinois; the 4Jessie Brown VA Medical Center, Chicago, Illinois; the 5Massachusetts Eye and Ear Infirmary, Boston, Massachusetts; the 6Department of Ophthalmology, Harvard Medical School, Boston, Massachusetts; the 7Lois and Edwin Deicke Center for Vision Rehabilitation, Wheaton, Illinois; the 8Department of Ophthalmology, Loyola University School of Medicine, Maywood, Illinois; and the 9Wilmer Eye Institute, Johns Hopkins University, Baltimore, Maryland.


   Abstract
Top
 Abstract
Methods
 Results
 Discussion
 References
 
PURPOSE. To describe psychometric properties of a self-report questionnaire, the Veterans Affairs (VA) Low-Vision Visual Functioning Questionnaire (LV VFQ-48), which was designed to measure the difficulty visually impaired persons have performing daily activities and to evaluate low-vision outcomes.

METHODS. The VA LV VFQ-48 was administered by telephone interview to subjects with visual acuity ranging from near normal to total blindness at five sites in the VA and private sector. Rasch analysis with the Andrich rating scale model was applied to difficulty ratings from 367 subjects, to evaluate measurement properties of the instrument.

RESULTS. High intercenter correlations for item measure estimates (intraclass correlation coefficient [ICC] = 0.97) justified pooling the data from these sites. The person measure fit statistics (mean square residuals) confirm that the data fit the assumptions of the model. The item measure fit statistics indicate that responses to 19% of the items were confounded by factors other than visual ability. The separation reliabilities for pooled data (0.94 for persons and 0.98 for items) demonstrate that the estimated measures discriminate persons and items well along the visual ability dimension. ICCs for test–retest data (0.98 for items and 0.84 for persons) confirm temporal stability. Subjects used the rating categories in the same way at all five centers. Ratings of slight and moderate difficulty were used interchangeably, suggesting that the instrument could be modified to a 4-point scale including not difficult, slightly/moderately difficult, extremely difficult, and impossible. Fifty additional subjects were administered the questionnaire with a 4-point scale to confirm that the scale was used in the same way when there were four rather than five difficulty ratings.

CONCLUSIONS. The VA LV VFQ-48 is valid and reliable and has the range and precision necessary to measure visual ability of low-vision patients with moderate to severe vision loss across diverse clinical settings.

Low vision is defined as any chronic visual impairment that interferes with everyday function and is not correctable by ordinary spectacles or contact lenses.1 The loss of visual abilities can have a profound effect on the life of an individual, by limiting performance of even simple everyday tasks, such as dressing, eating, writing, traveling from place to place, and communicating with others.2 Persons with low vision may feel frustrated and restricted in their lifestyles when they are unable to perform these everyday activities independently. Low-vision services enhance the use of remaining vision, increase independence, and improve the quality of life of persons with chronic visual impairments.3 Low-vision service delivery begins with an extensive interview to learn about the patient’s problems, needs, and goals. To promote cost-effective low-vision service delivery and to ensure that all relevant issues are covered, a standard assessment tool is needed to capture the patient’s self-report of difficulty performing activities at the initial interview, and the change in difficulty performing these activities in the community after rehabilitation. In this article, we describe the psychometric properties of a new self-report visual functioning questionnaire that was designed to serve both as a patient evaluation tool and as a rehabilitation outcome measure.

Visual functioning questionnaires include a set of questions that assess the performance of daily living activities.4 The activities being rated are referred to as items. Items work in synchrony to create a scale. The number of response choices included on visual functioning questionnaires ranges from as few as 2 to as many as 10. Responses are usually ordered by rank. Items are usually assigned a numerical score corresponding to the patient’s rating. Most visual functioning questionnaires employ Likert scoring, which is simply a linear transformation of the average response rank across items. Although it is hoped that Likert scores are monotonic with the person trait the instrument is attempting to measure, there are no means to confirm that they are without a normative measurement model. Therefore, we used the Rasch analysis to estimate interval visual ability scales from Likert-type visual function rating scales to test construct validity.5 We have used Rasch analysis to compare the difficulty that patients in two VA low-vision programs experienced in performing activities before and after rehabilitation.6 Other investigators have used Rasch analysis to validate a questionnaire to assess mobility in patients with visual field loss,7 8 an activities of daily living (ADL) index (the Melbourne Low Vision ADL Index),9 and a questionnaire to measure functional vision performance of visually impaired children (LV Prasad-Functional Vision Questionnaire).10 Most recently, Rasch analysis has been used to revalidate or improve on existing instruments, including the National Eye Institute Visual Functioning Questionnaire (NEI VFQ),11 the Visual Function self-report (VF-14),12 13 and the Activities of Daily Vision Scale.14

Although there are currently more than a dozen visual functioning questionnaires, most of the instruments that were available 5 years ago, when we began our outcomes studies, were developed for clinical trials of eye disease treatments that restore or improve vision (such as the removal of cataracts).3 4 None of these instruments is used routinely in delivery of low-vision services. Different visual functioning questionnaires were developed with different applications in mind. For example, the NEI VFQ-25 was developed with the purpose of measuring vision-specific quality of life in clinical research,15 16 whereas the VF-14 was developed with the purpose of measuring functional limitations caused by cataract and the outcomes of cataract surgery.17 Although many items from visual functioning questionnaires, including the Activities of Daily Vision Scale, VF-14, and NEI VFQ-25, could be used to measure low-vision rehabilitation outcomes, the number of items on each instrument is not sufficient to evaluate the range of activities that are a problem for persons with vision loss.18 Based on our review of these questionnaires and their development, we determined that an additional instrument was needed to measure the outcomes of low-vision service delivery.


   Methods
Top
 Abstract
 Methods
Results
 Discussion
 References
 
The development of the 48-item Veterans Affairs (VA) Low-Vision Visual Functioning Questionnaire (LV VFQ-48) using a modified Delphi method and preliminary analysis of data (before full sample) with the Andrich Rating scale model is described in two previous publications.19 20 Clinicians, rehabilitation specialists, scientists, research staff and persons with vision loss contributed to item development and selection. Service providers were asked to draw on their many years of experience in patient care to identify patient’s needs, activities that are commonly addressed in each rehabilitation program, and expected outcomes. The preferred practice patterns and clinical guidelines for VA blind rehabilitation services, optometry, ophthalmology, and occupational therapy were examined. An extensive literature review was conducted. Development, validation, and sensitivity to change of the existing visual functioning questionnaires and quality of life instruments were reviewed.19 20 In addition to generation of new items, questions and items from existing instruments were incorporated in the VA LV VFQ-48 or modified to improve their usefulness. Additional information was available from a previously conducted study that included structured interviews with persons with low vision to determine patients’ needs for low-vision devices.21 The 48 items chosen by the research team for the initial version of the VA LV VFQ-48 are listed in Table 1 .19 20


View this table:
[in this window]
[in a new window]
  		TABLE 1. Items Included in the 48-Item Field Test Version VA LV VFQ

 
The initial version of the VA LV VFQ includes four questions. Question 1 is asked about all 48 items: "Is it difficult to... ?" Response choices include: not difficult, slightly difficult, moderately difficult, extremely difficult, impossible, and do not do it for nonvisual reasons (which is scored as missing data). Question 2, "Is it (difficult) because of your vision?", is asked if the subject responds that it is difficult to perform an activity. Response choices are yes or no. If it is difficult to perform the activity because of vision loss, question 3 is asked, "Do you want training?" Responses are also yes or no. Patients who are able to perform the activities are asked in question 4 how they perform the activity. Question 4 is "How do you usually... ?" Response choices include: own eyes or eye glasses, vision devices/techniques (e.g., magnifier) other senses/nonvisual devices (e.g., cane), someone helps me, and not applicable.

Administration of the VA LV VFQ-48
The VA LV VFQ-48 was administered by telephone before rehabilitation. Two interviewers located at the Department of Ophthalmology (University of Illinois at Chicago) conducted interviews of patients from the Visual Impairment Center to Optimize Remaining Sight (VICTORS). Three interviewers located at the Hines Blind Rehabilitation Center conducted interviews for the other four sites. Administration time for all four questions on the VA LV VFQ-48 was 25 to 35 minutes.

Subjects
English-speaking patients were recruited from five clinical sites: the Hines Blind Rehabilitation Center (Hines BRC), Hines VA Outpatient Low Vision Rehabilitation/Eye Clinic (Hines LV Clinic), Jessie Brown VA Medical Center VICTORS Program (VICTORS), Lois and Edwin Deicke Center for Visual Rehabilitation (Deicke Center) and the Vision Rehabilitation Center at Massachusetts Eye and Ear Infirmary (MEEI). These sites were chosen to include a representative sample of patients with low vision who are typically referred or self-refer to low-vision clinics. The VA sites include a suburban hospital (Hines) and an inner-city medical center (Jessie Brown VA Medical Center). Included are regional VA programs targeting services to legally blind veterans (Hines BRC) and visually impaired veterans (VICTORS), as well as a local VA hospital outpatient low vision/eye clinic (Hines LV Clinic). The private sector sites chosen include a not-for-profit agency located in the Chicago suburbs (Deicke Center) and a hospital-based rehabilitation service located in a major city (MEEI), the latter of which is affiliated with both a department of ophthalmology and a department of physical medicine and rehabilitation.

Clinical staff at each program site excluded patients who would not be able to participate in the study. Reasons for exclusion included active major depression, cognitive loss, terminal illness, or other serious health conditions. The study was conducted in compliance with the tenets of the Declaration of Helsinki for research in human subjects.22 Informed consent was received based on the requirements of the local institutional review board (IRB) protocols approved at each participating site with oversight from the IRBs at Hines Hospital and the University of Illinois at Chicago.

The recruitment goal was a convenience sample of at least 50 subjects per site. Recruitment goals were exceeded at four of the centers, so the sample size for the initial analysis was 367 patients. A description of the subjects from each site is provided in Table 2 . Distribution of subjects by visual acuity is presented in Table 3 . Primary diagnoses of subjects are listed by site in Table 4 .


View this table:
[in this window]
[in a new window]
  		TABLE 2. Description of Subjects

 

View this table:
[in this window]
[in a new window]
  		TABLE 3. Habitual Visual Acuity of Subjects

 

View this table:
[in this window]
[in a new window]
  		TABLE 4. Primary Eye Diagnosis of Subjects

 
The test was administered a second time 3 to 4 weeks later to 30 subjects from the Hines BRC and the Hines LV Clinic. These subjects included 28 men and 2 women (mean age, 73 ± 9.4 (SD) years; range, 49–88 years). Primary diagnosis of these subjects was macular degeneration (50%), diabetic retinopathy (27%), glaucoma (13%), neurologic disorders (7%), and cataracts (3%). Habitual visual acuity was near-normal impairment (13%), moderate impairment (13%), severe impairment (54%), profound impairment (13%), and nearly or absolutely no light perception (7%).

As will be described in the Results section, subject ratings of slightly and moderately difficult were used interchangeably so that the categories could be collapsed to a 4-point scale (not difficult, slightly/moderately difficult, extremely difficult, and impossible), with no loss of information. Therefore, the questionnaire was administered to an additional 50 subjects recruited from the Hines BRC, Hines LV Clinic, and Deicke Center offering only four difficulty rating categories rather than five, to determine the validity of collapsing response categories in the data analysis. The 50 subjects in the validation study are described separately (as Verification) in Tables 2 3 and 4 .

Statistical Analysis
Rasch analysis23 24 was performed using Winsteps25 on difficulty ratings for the combined data from all five sites (N = 367). Rasch analysis also was performed separately on the data sets from each of the five sites, on the set of retest data from the Hines BRC and Hines LV Clinic, and on the additional 50 subjects from the Hines BRC, Hines LV Clinic, and Deicke Center who responded to the version of the instrument having four difficulty rating categories. Rasch analysis, using the Andrich Rating scale model,24 provides estimates of the visual ability of each person ({alpha}n) the required visual ability (i.e., inherent difficulty) of each item ({rho}i) and the threshold ({tau}x) for each response category (i.e., {alpha} – {rho}, where response probabilities for neighboring response categories are equal), on an interval logit scale.

The Andrich rating scale model is a normative probabilistic conjoint measurement model.24 It is normative, because it conforms to the structural requirements of axiomatic measurement theory.26 If the VA LV VFQ is valid as a measurement instrument, the data must agree with the expectations of the model. If the VA LV VFQ is to be useful as a measurement instrument, the estimated measures must be precise and reliable. To evaluate measurement precision, the analysis includes standard errors of each estimate and measurement reliability coefficients. To test measurement validity, the analysis includes normative measurement model fit statistics and estimates of conjoint structural coherence. To evaluate test–retest reliability and to evaluate the validity of collapsing response categories, we used the intraclass correlation coefficient (ICC).27

Estimates of Response Probabilities
The Andrich rating scale model is

where {pi}nix is the probability of person n responding with difficulty rating x to item i for m + 1 response categories (x = 0 to m).23 24


   Results
Top
 Abstract
 Methods
 Results
Discussion
 References
 
The results reported in this article are limited to the difficulty ratings of question 1 obtained before rehabilitation.

Figure 1 illustrates estimates of {pi} for each difficulty-rating category as a function of functional reserve, {alpha} – {rho} (person measure –item measure), based on the response matrix for all 367 subjects from the five sites combined.



View larger version (14K):
[in this window]
[in a new window]
  		FIGURE 1. Estimates of response probabilities using five response categories. The {pi} for each difficulty rating category is estimated as a function of functional reserve, {alpha} – {rho} (person measure –item measure), based on the response matrix for all 367 subjects from the five sites combined.

 
Note that there is no value of {alpha} – {rho} for which difficulty rating 2 is the most probable response. This observation suggests that patients used response category 2 idiosyncratically, which would imply that patients cannot reliably discriminate more than four categories of difficulty. This interpretation of the probability curves is echoed in Guttman’s coefficient of reproducibility,28 or coherence, for each difficulty-rating category: 83% for category 1, 21% for category 2, 28% for category 3, 38% for category 4, and 76% for category 5 (a coherence of 100% means that there is perfect order in the response matrix, i.e., all items agree in their ordering of person ability and all persons agree in their ordering of item difficulty; a coherence of 0% means that there is no ordered pattern in the response matrix). The Guttman coefficient of reproducibility across all responses is 57%. Category 2 is the least coherent, which means it is least able to discriminate among persons of different ability and among items of different required ability.

Because of the poor performance of rating categories 2 and 3, we combined those two response categories and reanalyzed the data with four difficulty rating categories instead of five. Figure 2 illustrates response probabilities for the recoded data set as a function of {alpha} – {rho}.



View larger version (14K):
[in this window]
[in a new window]
  		FIGURE 2. Figure 2 illustrates response probabilities for the recoded data set as a function of {alpha} – {rho} when rating categories 2 and 3 are combined.

 
In this case, there is a value of {alpha} – {rho} for which each of the difficulty rating categories is the most probable response. Guttman’s coefficients of reproducibility for the recoded difficulty rating categories are 78% for category 1, 44% for category 2, 38% for category 3, and 73% for category 4. The overall Guttman coefficient of reproducibility for the recoded data set is 60%.

To test the validity of recoding the difficulty rating categories, we obtained data from an additional 50 subjects and used four difficulty response categories instead of five. Figure 3 illustrates the estimated response probability functions for each of the four rating categories as a function of {alpha} – {rho} (solid curves).



View larger version (18K):
[in this window]
[in a new window]
  		FIGURE 3. Figure 3 illustrates the estimated response probability functions for each of the four rating categories as a function of {alpha} – {rho} (solid curves) for 50 subjects. For comparison, the probability curves in Figure 2 are presented as gray curves.

 
The Guttman coefficients of reproducibility for these data are 80% for category 1, 41% for category 2, 43% for category 3, and 78% for category 4. The overall Guttman coefficient of reproducibility is 62%. The response probability functions from Figure 2 are reproduced in Figure 3 for comparison (broken curves). The agreement between the two sets of probability functions confirms the validity of combining response categories 2 and 3 in the recoded data set.

Estimates of Person and Item Measures
Winsteps estimates person and item measures, {alpha}n and {rho}i, for each person and item, by an unconditional maximum-likelihood estimation routine.25 Both person and item measures are reported as log odds ratios (logits), which are on an interval scale. Item measure estimations from the separate Rasch analyses of each subset of data agree across the five sites (ICC = 0.97). Therefore, we combined the data of all sites to estimate item measures. The origin of the logit scale is defined to be the average required ability across all items. By definition, the average estimated value of {rho}i for the 48 items in the VA LV VFQ is zero. The SD of {rho}i is 1.06 with a range of –3.07 (least difficult) to 1.72 (most difficult). The root mean square standard error of the estimate over all items is 0.09. The precision of the item measures is represented by a separation reliability coefficient, which is the ratio of the adjusted variance to the observed variance in the item measure distribution. The adjusted variance is the difference between the observed variance and the mean square SE (i.e., adjusted variance is an estimate of the actual variance in {rho} across items). Separation reliability, which ranges from 0 to 1, is similar to Cronbach’s {alpha} and is an index of precision with which differences in {rho} between items can be discriminated.13 The item measure separation reliability is 0.993.

The estimated values of {alpha} range from –1.89 (least able) to 5.21 (most able). The mean (±SD) of the person measure distribution is 0.98 ± 1.07. The root mean square SE of the estimate for all patients is 0.27. The person measure separation reliability is 0.93. Figure 4 compares cumulative distributions of {alpha} for each of the five sites.



View larger version (16K):
[in this window]
[in a new window]
  		FIGURE 4. Figure 4 compares cumulative distributions of {alpha} for each of the five sites, demonstrating that the {alpha} distributions for Deicke, MEEI, and VICTORS are the same, whereas the {alpha} distribution for the BRC is significantly different from the other distributions.

 
The {alpha} distributions for Deicke Center, MEEI, and VICTORS are the same (Kolmogorov-Smirnov [K-S] test, P = 0.88, 0.95, 0.99, respectively).29 The {alpha} distribution for the Hines BRC is significantly different from the other distributions (K-S, P < 0.001). The distribution for the Hines LV Clinic has a lower mean than the MEEI, Deicke Center, and VICTORS distributions, but the differences do not achieve statistical significance, given the multiple comparisons (K-S, P = 0.04, 0.167, 0.168, respectively).

Figure 5 illustrates the person–item map. The vertical axis of the logit scale is visual ability ranging from least able (lowest value of {alpha}; at the bottom) to most able (highest value of {alpha}; at the top). The curves on the right are the frequency distributions of person measures for each of the five sites. The horizontal lines to the left of the axis represent the distribution of item measures ranging from least difficult (lowest value of {rho}) to most difficult (highest value of {rho}). The item measure distribution is well centered on the Hines BRC person measure distribution, but the person measure distributions for the other sites extend to higher visual abilities than are covered by the items (this means that the rating scale is doing most of the work of separating persons having greater ability). The items are not evenly spaced, and many of the items have similar item measures. Therefore, from a measurement perspective, there is high redundancy in the item measures and the instrument performs less well in discriminating among people with high ability than among people with lower ability.



View larger version (19K):
[in this window]
[in a new window]
  		FIGURE 5. The Rasch Person Item Map displays a scale created from measurement of patients’ abilities to perform activities of daily living (right) and the visual ability needed to perform each activity (left).

 
Validity of Person and Item Measure Estimates
The Andrich Rating scale model is a normative measurement model. It is based on a Rasch logistic model and conforms to the tenets of axiomatic measurement theory.23 24 Consequently, if the estimated measures from the VA LV VFQ difficulty ratings are valid, the pattern of patient responses to the items must agree with the expectations of the model within the limits of statistical error.30 Goodness of fit is evaluated with weighted mean square residual errors across items for each person, across persons for each item, and across all person–item encounters for the instrument.30 Two different weighting schemes are used. First, for each person–item encounter, the squared residual error is normalized to the expected variance. This normalized squared residual, which is expected to be distributed as {chi}2, is called the outfit statistic, because it is sensitive to outlying errors.30 For the second weighting scheme, the mean square residual is normalized to the average expected variance. This normalized mean squared residual, which also is expected to be distributed as {chi}2, is called the infit statistic, because it is most representative of inlying errors.30 Because both fit statistics are expected to be distributed as {chi}2, the weighted mean squares are transformed to an expected standard normal distribution using a Wilson-Hilferty transformation.31 Thus, the transformed outfit and infit values for each item and person are presented as z-scores with an expected mean of 0 and an expected SD of 1.

Table 5 lists the estimated measures ({rho}), the SE of the estimate, the infit and outfit mean squares (MNSQ), and the z-score transformations (ZSTD) of the infit and outfit mean squares for each item.


View this table:
[in this window]
[in a new window]
  		TABLE 5. Results of Rasch Analysis Applied to Rating of Difficulty for Each Activity

 
Figure 6 illustrates a scatterplot of the infit versus outfit mean square z-scores for the 48 items.



View larger version (19K):
[in this window]
[in a new window]
  		FIGURE 6. Scatterplot of the normalized mean square fit statistics for item measures, showing visual ability needed to perform each activity. The box includes infit and outfit mean square values that are within ±2 SDs of the model’s expectations.

 
The box encloses the values that are within ±2 SD of the expected value (0). Negative z-scores below and to the left of the box indicate that the residual errors for those items are less than would be expected from normally distributed measurement error. Positive z-scores above and to the right of the box indicate that the residual errors for those items are greater than would be expected from normally distributed measurement error. Of the 48 items, 13 (27%) have outfit mean squares, 10 (21%) have infit mean squares, and 9 (19%) items have both outfit and infit means squares that are greater than the expected value by >2 SD. Two of the items, work on favorite hobby and use public transportation, have outfit and infit mean squares that exceed the expected value by >5 SD. Other items with both outfit and infit mean squares that exceed 2 SD include, identify medicine, handle finances, go out at night, get around in a crowd, avoid bumping into things, read headlines, and sign your name. At the other extreme, 8 (17%) items have outfit mean squares and 10 (21%) items have infit mean squares that are less than the expected value by >2 SD. For two of the items, get around in unfamiliar places and see photos, both the outfit and infit means squares are less than the expected value by >4 SD.

Figure 7 illustrates the distribution of z-scores for person measure infit mean squares along the x-axis and the distribution of person measures ({alpha}) along the y-axis. The dashed vertical lines enclose the region that is ±2 SD from the expected value. Eleven percent of the patients have infit mean squares greater than the expected value by ≥2 SD (compared with 2.5% for a normal distribution) and 7% of the patients have infit mean squares less than the expected value by ≥2 SD (compared with 2.5% for a normal distribution). Two patients from VICTORS and two from Hines LV Clinic are grossly misfitting with infit mean squares greater than the expected value by >4 SD. Their response patterns to the items indicate that some confounding variable(s), and not the variable of interest, is governing the responses of these patients.



View larger version (25K):
[in this window]
[in a new window]
  		FIGURE 7. Scatterplot of the normalized mean square fit statistics for person measures, showing patients’ abilities to perform activities. The box includes infit and outfit mean square values that are within ±2 SDs of the model’s expectations.

 
The average infit and outfit mean squares across persons are 1.02 and 1.04, respectively, compared with an expected value of 1.00. In terms of z-scores, the average person infit and outfit are both 0.1, compared with an expected value of 0.0, and the standard deviations are 1.7 and 1.6, respectively, compared with an expected value of 1.0. The average infit and outfit mean squares across items are 1.01 and 1.03, respectively, compared with an expected value of 1.00. In terms of z-scores, the average item infit and outfit are 0.1 and 0.3, respectively, compared with an expected value of 0.0, and the standard deviations are 2.8 and 2.6, respectively, compared with an expected value of 1.0. These observations, combined with the previously described coherence of 62%, indicate that the VA LV VFQ is a valid, but imprecise, functional assessment instrument. The fit statistics indicate that constructs other than visual ability contribute error to the measurement.

Test–Retest Reliability
Figure 8 is a scatterplot of item measures estimated from responses to the initial administration of the VA LV VFQ by 30 patients versus item measures estimated from responses to the re-administration of the VA LV VFQ to the same 30 patients 3 to 4 weeks later. The error bars represent ±1 SE of the item measure estimate, and the diagonal line is the identity line. There is strong test–retest concordance with an ICC of 0.98 (95% confidence interval [CI]: 0.96–0.99). Figure 9 is a similar test–retest scatterplot for person measures. There is good test–retest concordance, with an ICC of 0.84 (95% CI: 0.69–0.92).



View larger version (17K):
[in this window]
[in a new window]
  		FIGURE 8. Scatterplot of item measures demonstrating test–retest concordance that was estimated from 30 patients initial responses to the VA LV VFQ-48 compared with the responses of the same patients 3 to 4 weeks later. Error bars, ±1 SE of the item measure estimate; diagonal line: identity line.

 


View larger version (18K):
[in this window]
[in a new window]
  		FIGURE 9. Scatterplot of person measures demonstrating test–retest concordance that was estimated from 30 patients initial responses to the VA LV VFQ-48 compared with the responses of the same patients 3 to 4 weeks later. Error bars: ±1 SE of the item measure estimate; diagonal line: identity line.

 

   Discussion
Top
 Abstract
 Methods
 Results
 Discussion
References
 
Our goal was to develop a self-report questionnaire to measure the difficulty visually impaired persons have in performing the daily living activities. This questionnaire could be used to tailor rehabilitation programs to meet individual patient needs and to measure outcomes of treatment strategies and rehabilitation programs. The 48-item VA LV VFQ was validated in both VA and private-sector programs, to facilitate comparison of outcomes from different service delivery settings and to increase the number of visually impaired female participants. Patients from multiple VA programs were included because the VA system provides separate rehabilitation programs for legally blind and partially sighted veterans, although there is some overlap in eligibility criteria for the programs.

The precision, validity, and reliability of the initial version of the VA LV VFQ were measured. The Andrich rating scale model was used to evaluate the required visual ability for each item and the threshold for each response category.23 24 25 The VA LV VFQ-48 includes five rating categories (not difficult, slightly difficult, moderately difficult, extremely difficult, and impossible). Analysis indicated that categories 2 (slightly difficult) and 3 (moderately difficult) performed poorly, even when the data from all five sites were combined. These responses were combined and the data reanalyzed with four rating categories rather than five. Further testing of the four rating categories (not difficult, slightly/moderately difficult, extremely difficult, and impossible) with an additional 50 subjects confirmed the validity of combining categories slightly difficult and moderately difficult. The use of four rating categories to measure the difficulty patients have in performing daily activities is consistent with reports from other studies.7 8

A person–item map for the sample of 367 subjects (Fig. 5) was constructed from the measures of perceived visual ability for persons and the required visual ability to perform each of the 48 tasks. The map of persons and items for the total sample represented by all five sites demonstrates that the VA LV VFQ-48 best separates patients with visual ability in the range of moderate to severe loss (+2 to –2 logits). The persons are distributed from most to least disabled, as would be expected from a sample of subjects with visual impairment that varies from no light perception to near normal vision. We also note that some patients did not have much difficulty performing any items. The person–item map demonstrates a floor effect. There are three activities (get dressed, keep clean, and get around indoors in places you know) that few patients reported difficulty performing. These items could be eliminated without any practical effect on the measurement range of the instrument. During development of the VA LV VFQ-48, clinicians and rehabilitation specialists indicated that the activities get dressed, keep clean, and get around in places you know are difficult for patients with vision loss. It is surprising that few patients reported difficulty with these items. One explanation for this finding is that rehabilitation professionals may see evidence of stains on clothing or assess hazards in the living environment, whereas the patients themselves may be unaware of the hazards or stains on their clothing. Difficulty getting dressed can be interpreted as physically dressing or more generally as coordinating clothing choices. Items are considered difficult on this questionnaire only if they are perceived by patients as related to vision loss. Difficulty dressing due to physical disabilities from stroke, arthritis, or other medical conditions would not be scored.

The person–item map is well centered on the BRC person measure distribution. The distributions for the sites other than the BRC extend to higher visual abilities than are covered by the items, indicating a ceiling effect. Fifteen percent of the patients had visual ability greater than 2 logits. The items selected are adequate to describe the functional ability of 85% of the subjects in the study. The more mildly impaired (those with visual ability greater than 2 logits) are less well discriminated by the items.

We could consider adding more difficult items to the VA LV VFQ-48, such as reading signs at night or driving during poor weather conditions, to extend the scale and obtain more precise measurements of persons with higher levels of visual ability. However, discriminating differences among near-normal patients is not our goal in developing a low-vision visual functioning questionnaire. Our goals are to identify patients who need and will benefit from low-vision rehabilitation and to measure the outcomes of low-vision service delivery. It is a common problem that many patients with near-normal vision self-refer or are referred to low-vision clinics because they are dissatisfied with their refractive error correction or are experiencing normal changes in visual function associated with aging, such as age-related changes in the lens, vitreous, and retina, as well as cellular alterations in the visual pathway and limbic system. Because eligibility and referral criteria differ between centers, some patients with near-normal vision receive services from some of the low-vision programs participating in this study.

The VA LV VFQ-48 does not include any driving items. Based on our experience with visual functioning questionnaires, there are considerable missing data on these items. Many older patients, especially those with moderate to severe vision loss self-limit driving or they may not be eligible to drive based on the criteria for licensing in the state of residence. Many of the patients with high visual ability may have self-referred to the low-vision clinic based on difficulty driving in daylight or at night. Difficulty with these tasks would not be reflected in the VA LV VFQ-48. The VA LV VFQ-48 would not be an appropriate instrument to evaluate the outcomes of a driving rehabilitation program. Specialized questionnaires to evaluate driving habits have been developed by other investigators.32

The validity of the VA LV VFQ-48 was evaluated to determine the extent to which the pattern of patient responses agrees with the expectations of the model. Goodness of fit is represented by the transformed outfit and infit values for each item and person, presented as z-scores with an expected value of 0 and an SD of 1. Based on the scatterplot of the infit versus outfit mean square z scores for the 48 items (Fig. 6) , residual errors for two items, work on your favorite hobby and use public transportation exceed the expected value by >5 SD. The item measure infit and outfit mean squares can be thought of as estimates of the variance of inherent difficulty within the sample of patients for each item. Large mean square values indicate high variance in the inherent difficulty distribution and small mean square values indicate low variance. Disagreements between the infit and outfit mean squares reflect a departure from symmetry of the inherent difficulty distribution for that item. High mean square values indicate imprecision in the item measure across persons.

The poor fit of these items is most likely due to the variability in the visual ability necessary to perform different hobbies or to access available forms of public transportation and the inherent difficulty of the task independent of vision loss. The visual demands (near vision) for stamp or coin collecting differ greatly from the visual demands and skills needed to play golf or table games (e.g., chess). Use of public transportation can include complex mobility tasks, such as traveling independently by subway or simply calling a cab to provide transportation to a specific location. Because there is so much variability in these items, they create noise and contribute little to the measurement properties of the instrument. They probably should not be used when estimating {alpha}.

Other items with both outfit and infit mean squares that exceed 2 SD include, identify medicine, handle finances, go out at night, get around in a crowd, avoid bumping into things, read headlines, and sign your name. These items may need to be rewritten to be more specific. As an example, patients may interpret "identify medicine" as meaning identify bottles or tubes of different medicines, identify shapes and colors of pills, or read print on the medicine bottle. Items that can be interpreted in different ways are often noisy. It is important to include the same items on the questionnaires administered before and after rehabilitation to assess sensitivity of the instrument to changes brought about by the rehabilitation program. Decisions to eliminate or rewrite items to refine the questionnaire should be deferred until after the postrehabilitation data collection and analysis.

Another explanation for misfit of some of the items is that the sample of subjects is largely composed of persons with central visual acuity loss. Only 20% of the subjects in the initial analysis had diagnoses associated with peripheral visual field loss. Some of the items that misfit are those that expected to be sensitive to peripheral vision loss (e.g., get around in a crowd or avoid bumping into things). There is a basic assumption that the VA LV VFQ-48 is measuring one underlying construct, visual ability. Factor analysis may indicate that more than one construct is being measured. This possibility will be explored in future papers.

In some cases, there are multiple items representing the same level of difficulty. From a measurement perspective, some of these items could be omitted to shorten the instrument and decrease administration time and respondent burden. The prevailing practice in instrument development is to use the minimum number of items needed to measure a domain.13 14 However, when the instrument is used to measure low-vision outcomes, valuable information would be lost if clinically significant redundant items were eliminated. Our previous research with the NEI VFQ-25 indicates that item-specific changes occur as a result of rehabilitation.6 Low-vision rehabilitation enhances remaining vision for specific activities. As an example, read street signs and keep your place while reading represent the same ability level, but different low-vision devices would be used to make these activities easier to perform. Telescopic devices are indicated for distance activities, while magnifiers are used for near-reading activities. If we are measuring outcomes of service delivery, both items are useful. However, if we are measuring visual ability of patients undergoing cataract surgery, items could be based solely on the measurement properties of the scale, because cataract extraction restores visual ability. Changes on one item would reflect changes that would be expected on items of similar difficulty. Further modification of the VA LV VFQ-48 may be possible to reduce redundancy, eliminate or rewrite items with poor model fit to improve precision, and decrease the number of items on the instrument. If any items were to be dropped at this time, they would be the most misfitting ones and the ones easiest to perform by all the subjects. However, it is premature to edit the instrument before sensitivity of the items to change after rehabilitation can be evaluated. These results will be reported in future research reports.

Test–retest reliability was estimated for both persons and items, to confirm that we can discriminate people based on their estimated visual ability and items on the basis of their difficulty. Test–retest reliability yielded an ICC of 0.98 (95% CI: 0.96–0.99) for item measures and 0.84 (95% CI: 0.69–0.92) for person measures. These reliabilities can be compared with other instruments to measure visual impairment such as contrast sensitivity function tests although a true comparison should be done on the same patients. The VA LV VFQ-48 test–retest ICC for person measures is better than that reported for the VISTECH Chart with ICC ranges from 0.35 at low spatial frequencies to 0.93 at middle spatial frequencies and comparable to the Pelli-Robson CSF ICC of 0.86.33

In conclusion, we have developed a valid and reliable tool to measure visual ability of low-vision patients with moderate to severe vision loss across diverse clinical settings. The instrument has been designed to assess activities that are addressed by rehabilitation so that it may have use as an outcomes measure. The items within the instrument were constructed to be concise, easily and universally understood, and with simple and consistent response categories. The psychometric properties of the instrument were evaluated using Rasch analysis. Based on results from the Rasch analysis, the scale has been modified to four response choices rather than five. Further refinement of the instrument is indicated after the data on sensitivity to change is evaluated. Comparison of pre- and postrehabilitation VA LV VFQ-48 scores will confirm that the changes in values as a result of the intervention are resolvable with the instrument.


   Footnotes
 
Supported by the Office of Research and Development, Rehabilitation Research and Development Service of the Department of Veterans Affairs, Illinois Society for Prevention of Blindness, Massachusetts Eye and Ear Infirmary Joel A. Kraut Teaching and Research Fund (PD-T), Max Goldenberg Foundation (RTW), and Grant EY-012045 of the National Eye Institute (RWM).

Submitted for publication February 25, 2004; revised June 21, 2004; accepted June 28, 2004.

Disclosure: J.A. Stelmack, None; J.P. Szlyk, None; T.R. Stelmack, None; P. Demers-Turco, None; R.T. Williams, None; D. Moran, None; R.W. Massof, None

The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked "advertisement" in accordance with 18 U.S.C. §1734 solely to indicate this fact.

Corresponding author: Joan A. Stelmack, Blind Rehabilitation Center (124), Edward Hines Hospital, Hines, IL 60141; joan.stelmack{at}med.va.gov.


   References
Top
 Abstract
 Methods
 Results
 Discussion
 References
 

  1. U.S. Department of Health and Human Services. Vision Research A National Plan 1999–2003. 1998;117–130. National Eye Institute Bethesda, MD. NIH Publication No. 98-4120.
  2. Department of Veterans Affairs, Veterans Health Administration, Blind Rehabilitation Service. Coordinated Services for Blinded Veterans. 1996;4. Department of Veterans Affairs Washington, DC. IB 11-59 (revised) P87250
  3. Stelmack J. Quality of life of low-vision patients and outcomes of low-vision rehabilitation. Optom Vis Sci. 2001;78:335–342.[CrossRef][Web of Science][Medline][Order article via Infotrieve]
  4. Massof RW, Rubin GS. Visual function assessment questionnaires. Surv Ophthalmol. 2001;45:531–548.[CrossRef][Web of Science][Medline][Order article via Infotrieve]
  5. Massof RW. The measurement of vision disability. Optom Vis Sci. 2002;79:515–552.
  6. Stelmack JA, Stelmack TR, Massof RW. Measuring low-vision outcomes with the NEI VFQ-25. Invest Ophthalmol Vis Sci. 2002;43:2859–2868.[Abstract/Free Full Text]
  7. Turano KA, Massof RW, Quigley HA. A self-assessment instrument designed for measuring independent mobility in RP patients: generalizability to glaucoma patients. Invest Ophthalmol Vis Sci. 2002;43:2874–2881.[Abstract/Free Full Text]
  8. Turano KA, Geruschat DR, Stahl JW, Massof RW. Perceived visual ability for independent mobility in persons with retinitis pigmentosa. Invest Ophthalmol Vis Sci. 1999;40:865–877.[Abstract/Free Full Text]
  9. Haymes SA, Johnston AW, Heyes AD. The development of the Melbourne Low-Vision ADL Index: A measure of visual disability. Invest Ophthalmol Vis Sci. 2001;42:1215–1225.[Abstract/Free Full Text]
  10. Gothwal VK, Lovie-Kitchin JE, Nutheti R. The development of the LV Prasad-Functional Vision Questionnaire: a measure of performance of visually impaired children. Invest Ophthalmol Vis Sci. 2003;44:4131–4139.[Abstract/Free Full Text]
  11. Massof RW, Fletcher DC. Evaluation of the NEI visual functioning questionnaire as an interval measure of visual ability in low vision. Vision Res. 2001;41:397–413.[CrossRef][Web of Science][Medline][Order article via Infotrieve]
  12. Velozo CA, Lai JS, Mallinson T, Hauselman E. Maintaining instrument quality while reducing items: application of Rasch analysis to a self-report of visual function. J Outcome Meas. 2000;4:667–680.[Medline][Order article via Infotrieve]
  13. Mallinson T, Stelmack J, Velozo C. A comparison of the separation ratio and coefficient alphas in the creation of minimum data sets. Med Care. 2004;42(suppl)1–17.[CrossRef][Medline][Order article via Infotrieve]
  14. Pesudovs K, Garamendi E, Keeves JP, Elliott DB. The Activities of Daily Vision Scale for cataract surgery outcomes: re-evaluating validity with Rasch analysis. Invest Ophthalmol Vis Sci. 2003;44:2892–2899.[Abstract/Free Full Text]
  15. Mangione CM, Lee PP, Pitts J, Gutierrez P, Berry S, Hays RD. Psychometric properties of the National Eye Institute Visual Function Questionnaire (NEI-VFQ): NEI-VFQ Field Test Investigators. Arch Ophthalmol. 1998;116:1496–1504.[Abstract/Free Full Text]
  16. Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD. Development of the 25-item National Eye Institute Visual Function Questionnaire. Arch Ophthalmol. 2001;119:1050–1059.[Abstract/Free Full Text]
  17. Steinberg EP, Tielsch JM, Schein OD, et al. The VF-14: an index of functional impairment in patients with cataract. Arch Ophthalmol. 1994;112:630–2638.[Abstract/Free Full Text]
  18. Stelmack J, Szlyk J, Joslin C, Swetland B, Myers L. Pilot study: use of the NEI VFQ-25 to measure outcomes of low vision rehabilitation services in the VA. Stuen C Arditi A Horowitz Aet al eds. Vision Rehabilitation: Assessment, Intervention and Outcomes. 2000;774–776. Swets & Zeitlinger Lisse, The Netherlands.
  19. Stelmack J, Szlyk J, Stelmack T, et al. Use of the Rasch person item map in exploratory data analysis: a clinical perspective. J Rehabil Res Dev. 2004;41:233–242.[CrossRef][Web of Science][Medline][Order article via Infotrieve]
  20. Szlyk JP, Stelmack J, Massof RW, et al. Performance of the veterans affairs low vision visual functioning questionnaire. J Vis Impair Blind. 2004;98:261–275.
  21. Stelmack J, Rosenbloom A, Brenneman C, Stelmack T. Patients’ perceptions of the need for low vision devices. J Vis Impair Blind. 2003;97:521–535.[Web of Science]
  22. Dunn CM, Chadwick G. Code of Federal Regulations. Allen W eds. Protecting Study Volunteers in Research. 1999;203. Center Watch, Inc Boston, MA.
  23. Wright BD, Masters GN. Rating Scale Analysis. 1982; MESA Press Chicago, IL.
  24. Andrich D. A Rating formulation for ordered response categories. Psychometrika. 1978;43:561–573.[CrossRef][Web of Science]
  25. Linacre JM, Wright B. A User’s Guide to Winsteps Rasch-Model Computer Program. 2001; MESA Press Chicago, IL.
  26. Luce RD, Tukey JW. Simultaneous conjoint measurement: a new type of fundamental measurement. J Math Psychol. 1964;1:1–27.
  27. Muller R, Buttner P. A critical discussion of intraclass correlation coefficients. Stat Med. 1994;13:2465–2476.[Web of Science][Medline][Order article via Infotrieve]
  28. Menzel H. A new coefficient for scalogram analysis. Public Opin Q. 1953;17:268–280.[Abstract]
  29. Chakravarti IM, Laha RG, Roy J. Handbook of Methods of Applied Statistics. 1967;1:392–394. John Wiley & Sons New York.
  30. Bond TG, Fox CM. Applying the Rasch Model. 2001; Earlbaum Mahwah, NJ.
  31. Smith RM. The distributional properties of Rasch item fit statistics. Educ Psychol Meas. 1991;51:541–565.[Abstract]
  32. Owsley C, Stalvey B, Wells J, Sloane ME. Older drivers and cataract: driving habits and crash risk. J Gerontology A: Biological Sciences Medical Sciences. 1999;54:203–211.
  33. Rubin GS. Reliability and sensitivity of clinical contrast sensitivity tests. Clin Vision Sci. 1988;2:169–177.



This article has been cited by other articles:


Home page
 IOVSHome page
E. L. Lamoureux, K. Pesudovs, J. Thumboo, S.-M. Saw, and T. Y. Wong
An Evaluation of the Reliability and Validity of the Visual Functioning Questionnaire (VF-11) Using Rasch Analysis in an Asian Population
Invest. Ophthalmol. Vis. Sci., June 1, 2009; 50(6): 2607 - 2613.
[Abstract] [Full Text] [PDF]

Home page
 Arch OphthalmolHome page
J. A. Stelmack, X. C. Tang, D. J. Reda, S. Rinne, R. M. Mancil, R. W. Massof, and for the LOVIT Study Group
Outcomes of the Veterans Affairs Low Vision Intervention Trial (LOVIT)
Arch Ophthalmol, May 1, 2008; 126(5): 608 - 617.
[Abstract] [Full Text] [PDF]

Home page
 Clin TrialsHome page
J. A Stelmack, X. C. Tang, D. J. Reda, D. Moran, S. Rinne, R. M. Mancil, R. Cummings, G. Mancil, K. Stroupe, N. Ellis, et al.
The Veterans Affairs Low Vision Intervention Trial (LOVIT): Design and Methodology
Clinical Trials, December 1, 2007; 4(6): 650 - 660.
[Abstract] [PDF]

Home page
 IOVSHome page
F. Denny, A. H. Marshall, M. R. Stevenson, P. M. Hart, and U. Chakravarthy
Rasch Analysis of the Daily Living Tasks Dependent on Vision (DLTV)
Invest. Ophthalmol. Vis. Sci., May 1, 2007; 48(5): 1976 - 1982.
[Abstract] [Full Text] [PDF]

Home page
 IOVSHome page
E. L. Lamoureux, J. F. Pallant, K. Pesudovs, J. B. Hassell, and J. E. Keeffe
The Impact of Vision Impairment Questionnaire: An Evaluation of Its Measurement Properties using Rasch Analysis
Invest. Ophthalmol. Vis. Sci., November 1, 2006; 47(11): 4732 - 4741.
[Abstract] [Full Text] [PDF]

Home page
 IOVSHome page
A. Subramanian and C. Dickinson
Spatial Localization in Visual Impairment
Invest. Ophthalmol. Vis. Sci., January 1, 2006; 47(1): 78 - 85.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (22)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Stelmack, J. A.
Right arrow Articles by Massof, R. W.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Stelmack, J. A.
Right arrow Articles by Massof, R. W.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS