| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1From the Departments of Ophthalmology and 4Psychology, 2The Optical Sciences Center, and 3The College of Public Health, The University of Arizona, Tucson, Arizona.
 |
Abstract |
|---|
 Top Abstract MethodsResultsDiscussionReferences |
|---|
METHODS. Based on analysis of vision screening and eye examination data from a preschool population with a 33% prevalence of astigmatism, comparative costs to conduct a 1000-child screening program with a target sensitivity of 90% were estimated for photoscreening, noncycloplegic autorefraction, autokeratometry, and Lea symbols distance visual acuity testing. Results of the cost analysis and examination of sensitivity and specificity data from the preschool population led to development of a hybrid screening program of autokeratometry and visual acuity screening with referral thresholds of 2.25 D of corneal astigmatism or inability to read a 20/63 Lea symbols line on two separate attempts. The screening program was prospectively implemented in a community-based screening of a similar cohort of 167 children, and its efficiency was evaluated by comparison to results of cycloplegic refraction.
RESULTS. The community-based screening showed 96.8% sensitivity and 79.2% specificity for detecting the presence of refractive astigmatism of 1.50 D or more.
CONCLUSIONS. Referring children who have at least 2.25 D of corneal astigmatism or acuity worse than 20/63 on two attempts, provides the high sensitivity and specificity associated with automated keratometry while maintaining an acuity component that can detect other causes of reduced acuity in the absence of astigmatism.
There are two approaches that can be used to screen children for astigmatism. One is to use an instrument to detect physical characteristics of the eye that can cause a poorly focused image to form on the retina. This approach includes photoscreening7 8 and autorefraction,9 10 to screen for overall (refractive) astigmatism, and autokeratometry11 12 to screen for corneal astigmatism, which is efficient because astigmatism among Native Americans has been shown to be primarily corneal in origin (Kershner RM, et al. IOVS 1984;25:ARVO Abstract 70).11 13 14 15 16 The second, more conventional approach to vision screening is behavioral, in which the child is asked to "read" an eye chart, by naming or matching letters17 18 or by naming or matching shapes.19 20 With this method, if the child fails to identify targets with a small visual angle, he or she is presumed to have a problem with the visual system that warrants further study.21 22 23 However, failure of a young child to identify small targets may also result from inattention or shyness, resulting in acuity estimates that can be falsely poor when in fact no organic cause of decreased visual acuity (e.g., refractive error) is present.
We have recently completed a study comparing the ability of four vision-screening methods to detect significant astigmatism in a population of preschool children participating in Head Start programs on the Tohono O’Odham reservation.24 Photorefraction, autorefraction, autokeratometry, and measurement of visual acuity were compared by means of receiver operator characteristic (ROC) curves, and the screening utility of each method was rank ordered and characterized by maximum-likelihood estimates. The published ROC curves describe the expected sensitivity and specificity of each screening method when the method is used in a similar population of children. One important limitation of ROC curve analysis, however, is that, although the method allows comparison of screening methods and comparison of associated sensitivities and specificities, the optimum referral threshold is not directly specified. For specification of the optimum referral threshold, a more detailed analysis must be performed, and then the predictions of that analysis must be confirmed in a subsequent population.
The purpose of this article is to describe the methods used to design a cost-efficient, community-based vision screening protocol optimized for a preschool-aged population in which there is a high prevalence of astigmatism and then to report the results of the implementation of this protocol by Head Start program staff. First, the costs associated with implementing each of the four previously reported screening methods24 were compared, and an economic analysis was used to determine which screening method would be most appropriate for screening a specified number of subjects. Second, a screening protocol was developed for our target population, and a sensitivity analysis was performed on the dataset that led to the creation of the ROC curves,24 to determine recommended referral levels and to predict the number of children who would be either correctly or incorrectly referred were that program to be implemented. Finally, we used the protocol to screen another cohort of 167 preschool children and compared the results with data from comprehensive examinations on these children to determine the actual efficiency of the program in a community-based screening effort.
|
Methods |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
For assumption 1, the expected prevalence of astigmatism, we used 33%, which was the proportion of preschool children examined over a 2-year period (fall 1997 through fall 1998) who met the criteria for prescription of glasses for astigmatism (
2.00 D cylinder in a 3-year-old; 1.50 D cylinder in a 4- or 5-year-old).24 During 4 years of screening children in the tribal Head Start program (fall 1997 through fall 2000), we tested 541 children, and so a reasonable assumption in an expanded screening program that accessed all 3- to 5-year-old children of the tribe over a period of approximately 5 years would be 1000 children screened, the number applied in assumption 2. After reviewing the results of our study24 with the study’s National Eye Institute–appointed Data Monitoring and Oversight Committee that included vision scientists, clinicians, and community representation, we selected a target sensitivity of 90% for each of the selected screening methods (albeit at very different specificities), and this percentage was used for assumption 3. For assumption 4, we set the cost of an eye examination at $50, and did not adjust for inflation over time. The $50 estimate of the cost of an eye examination is based on an informal review of the cost in a retail setting and was also used in a preliminary report of data from this study.9 For assumption 5, the acquisition cost of the Photoscreener (Medical Technology, Inc. [MTI], Riviera Beach, FL) was assumed to be $2000, the autorefractor (Retinomax K-Plus; Nikon, Inc., Melville, NY) to be $15,000, the autokeratometer (Nidek KM500; Marco Ophthalmic Instruments, Inc., Jacksonville, FL) to be $4000, and the Lea symbols distance visual acuity chart (Precision Vision, LaSalle, IL) to be $50. Although there are other photorefractors, autorefractors, autokeratometers, and preschool visual acuity charts that could be used in a screening environment and their acquisition costs may differ considerably from those of the instruments used in our analysis, we do not have performance data for these instruments and therefore cannot generate ROC curves for the target population. For assumption 6, estimates of instrument performance (specificity at the selected level of sensitivity) were based on analysis of ROC curves derived in our previous work.24 These assumptions are summarized in Table 1 , and are the basis of the economic model.
|
|
2.00 D for children younger than 48 months of age and 1.50 D for children 48 months of age or older26 ). Thus, the ROC curves essentially reflect the ability of each instrument to identify correctly the preschool children with astigmatism requiring spectacle correction. As approximately equal numbers of children were present in each age group (45% were younger than 48 months age), the ROC curves should be viewed as describing the ability to detect between 1.50 and 2.00 D of refractive astigmatism in a population of 3- to five-year-old children with astigmatism, 33% of whom needed spectacle correction.
Cost-Efficiency of Each Screening Instrument as Predicted by the Economic Model.
In a screening program that is tightly coupled with a referral program, there is a direct correlation between false-positive referrals and associated costs, because each unnecessary referral either results in a charge (in a system in which patients seek and pay their own eye care provider) or loss of an appointment slot for another individual (in a capitated system, in which the costs are shared by all program participants). If level of sensitivity is specified (e.g., 90% in Table 1 ) and the prevalence of disease is known for the population (e.g., 33% in Table 1 ), it becomes possible to compare screening techniques on the basis of costs associated with these unwarranted examinations arising from referrals in which no disease is found. Until a sufficient number of children are screened to "pay for" a more expensive screening instrument that has a higher sensitivity (i.e., lower number of false referrals), a less expensive screening method is preferred.
Table 2 was constructed by rank ordering the assumed acquisition costs of each screening method and then comparing these acquisition costs to the specificity for each screening instrument that would be expected at the desired sensitivity. The number of false-positive referrals was calculated as the number of children screened x (1 - prevalence) x (1 - specificity). In all instances, the sensitivity was estimated to be 90%, and the prevalence was estimated to be 0.33.
|
Table 1 shows that the most expensive instrument (NCARS) also has the highest expected specificity (96%), and the second most expensive instrument (KERS) has the second highest expected specificity (80%). However, the third most expensive instrument (MTIPS) has a lower expected specificity (36%) than LSVAS (50%). Because MTIPS is more expensive to acquire than an eye chart, has associated recurring expenses in film (ignored in the present analysis), and has not been demonstrated to be less labor intensive than visual acuity screening, in the time required to photograph a child and interpret the image, it was immediately eliminated from further consideration for this specific population of Native American preschool children with astigmatism.
We then calculated the numbers of false-positive referrals that would occur when different numbers of children were screened with each of the three remaining screening instruments (NCARS, KERS, and LSVAS; Table 2 ). When the difference in acquisition cost between two instruments (in units of eye examinations, shown in Table 1 ) becomes less than the difference in the number of false-positive referrals between the two instruments, the more expensive screening modality becomes cost-efficient at that screening volume, compared with the next less-expensive instrument (represented by gray shading in Table 2 ). For example, screening of 2052 children would result in 55 false-positive referrals with NCARS, compared with 275 with KERS—a difference of 220 examinations at $50, or a total of $11,000. This is equal to the difference in cost ($11,000) between the NCARS and KERS instruments and is therefore the threshold number of screenings at which NCARS becomes more efficient than KERS. This threshold number is derived by dividing the incremental number of false-positive examination results to be eliminated (220 in this example) by the difference in specificity (0.96–0.80) x 1 - specificity (0.67, in this example). In the present example 2052 = 220/(0.16 x 0.67), after rounding. This screening volume is inversely proportional to the dollar amount assigned for an eye examination. For example, if the amount assigned to an eye examination is doubled to $100 from $50, then the number of subjects at which the autorefractor becomes cost-efficient is halved to 1026 from 2052.
It is interesting to note, as shown in Table 2 , that for extremely small screening volumes (less than five children), it is most cost- efficient simply to send the children for an eye examination. For a very small screening program (fewer than 400 children to be screened), it is most cost-efficient to screen the children by LSVAS. For a screening program volume of at least 400 children, screening with an autokeratometer (KERS) produces fewer unnecessary eye examinations than the number associated with LSVAS, and this smaller number is sufficient to offset the higher capital acquisition cost of the autokeratometer. For a large screening program of at least 2052 children, the autorefractor screening method (NCARS) is financially more advantageous than the autokeratometer (KERS) in the number of unneeded referrals generated compared with the higher acquisition cost.
Based on the economic analysis summarized in Table 2 and on the characteristics and number of children to be screened (Table 1) , we selected the autokeratometer (KERS) as the primary instrument in our preschool screening protocol.
Design of a Hybrid Screening Program
The Need to Incorporate a Visual Acuity Test into an Autokeratometry Screening Program.
In our target population, autokeratometry screening is very efficient in predicting the need for eyeglasses, because the predominant cause of refractive error is astigmatism, and most of the astigmatism is corneal in origin.24 However, it is possible that other significant disease may exist that would be completely missed by keratometry screening, because a keratometer measures only the shape of the anterior surface of the cornea for the presence of corneal astigmatism. Because visual acuity testing is recommended as a preschool vision screening tool by pediatric ophthalmologists, optometrists, and pediatricians,21 22 23 we decided to add visual acuity screening with a single line of letters to our keratometry screening protocol to detect more rare (in this population) defects of the visual system, such as cataract and strabismic amblyopia.
Hybrid Screening Program Procedure.
The hybrid screening procedure is based on an initial screening using autokeratometry, followed by visual acuity screening of children who pass autokeratometry screening. Only children who pass autokeratometry screening participate in visual acuity screening. Children who fail autokeratometry screening are referred for a comprehensive eye examination, and if problems with acuity exist, it is assumed that they will be found in the course of the eye examination. If a child passes autokeratometry screening and the initial visual acuity screening, no more testing is needed and the child’s screening result is "pass." However, a child who passes autokeratometry screening but fails the initial acuity screening has acuity rescreened on a separate day. Only those who fail both acuity screenings are referred for a comprehensive eye examination. Thus, initial screening with autokeratometry reduces the pool of children who must have acuity screening performed, whereas the requirement for retesting children who pass autokeratometry screening and fail the first acuity screening reduces the number of children who are referred as a result of an acuity deficit related to poor performance rather than to an ocular problem.
Specification of Referral Criteria for the Hybrid Screening Program.
To determine referral thresholds for the hybrid vision screening program, an analysis was conducted on data from the 379 children, 36 to 63 months of age, who participated in the study comparing photorefraction, autorefraction, autokeratometry, and visual acuity screening.24 None of the children had special needs as defined by the Head Start program and none had evidence of ocular disease, based on a comprehensive eye examination. As indicated in the previous publication,24 the research adhered to the tenets of the Declaration of Helsinki, parents provided informed consent before testing, and the research was approved by the institutional review board of the University of Arizona.
Testing was conducted at Head Start sites on the Tohono O’Odham reservation. Each child was tested with each of the four screening procedures, followed by a comprehensive eye examination including cycloplegic refraction, during the fall of the 1997/1998 or 1998/1999 academic year. Among the 379 children, 1 had incomplete autorefraction data, 1 had incomplete autokeratometry data, 30 were unable to perform visual acuity testing, and 3 were wearing spectacles during visual acuity testing, which rendered their acuity noninformative for screening purposes.
Three hundred eight of the 379 children participated in follow-up testing in the spring of the same academic year. The follow-up session included measurement of visual acuity, followed by a comprehensive eye examination, including cycloplegic refraction. Of the 308 children who participated in spring testing, 4 were unable to perform visual acuity testing, and 50 were wearing spectacles during visual acuity testing, which rendered their acuity noninformative for screening purposes.
The first step in the determination of referral criteria was to define disease-positive status (significant refractive astigmatism warranting spectacle correction). A single criterion of astigmatism 1.50 D or more in one or both eyes, based on the results of the cycloplegic refraction, was used. The definition of disease-positive status had to include only refractive error, because subjects having other disease were excluded from the dataset used for analysis. This definition of disease-positive status differs from that in our earlier work, in which age-dependent criteria for prescribing glasses were used.5 24 26 The effect of this change is to provide a slightly more conservative definition for the younger children, to provide a single criterion regardless of age. Of the 379 subjects, 142 (37.5%) met this criterion for disease-positive status. No children had significant hyperopia (>4.50 D) or significant myopia (>2.50 D if <48 months of age; >1.50 D if >48 months) in the absence of the specified amount of astigmatism.
The next step in determining referral thresholds for the hybrid screening program was to write a computer program that could analyze the dataset under a variety of scenarios. The input to the computer program was the amount of refractive astigmatism in the eye with the higher amount of astigmatism (from cycloplegic refraction), the amount of corneal astigmatism in the eye with the higher amount of corneal astigmatism (from KERS), and the visual acuity that was obtained from the eye with the worse acuity, as measured by trained testers using a Lea symbols chart mounted in a retroilluminated cabinet at a 3-m test distance, on up to two occasions (LSVAS). Children who were present during both the fall and spring testing sessions contributed both initial and follow-up visual acuity results for each eye to the data analysis, whereas children who were absent in either the spring or the fall contributed only a single visual acuity result.
We assumed in this analysis that a community-based measurement of corneal astigmatism using KERS would not differ from that we obtained as part of our testing program, because the autokeratometer does not seem to require a great deal of practice or skill on the part of the operator to obtain reliable readings. For analysis of visual acuity results, we assumed that a child whose visual acuity was measured to be a certain value (e.g., 20/32) under the test conditions of our program would be able to read a single line of letters administered by a community tester if that line were of the same size or larger, after warm-up testing with a single line of larger (20/80) letters. We recognize that there are differences in performance when a child has more extensive practice with larger letters as in a full-threshold measurement, and that there may be additional differences that arise from using a single-line presentation. This difference, if present, would become apparent when the protocol was prospectively evaluated.
The computer program varied referral thresholds for both KERS and LSVAS. If the child passed KERS at the target referral level, the threshold visual acuity for the worse eye from the first visual acuity test session in which the child produced uncorrected acuity results was evaluated to determine whether it was equal to or better than the target LSVAS referral level. If the first visual acuity measured in a child who had passed autokeratometry screening was worse than the target screening acuity threshold or the child was unable to perform acuity testing, the computer program would then check for the presence of a second acuity score, which would have been collected at the spring follow-up session. If the child’s second acuity score was equal to or better than the target screening acuity threshold, the computer program scored the child’s acuity results as a "pass." If the child had a spring acuity score that was worse than the target screening acuity threshold, or the child was unable to perform acuity testing, the computer scored the child’s acuity results as a "fail." If the child was absent for the spring follow-up session or was tested in spectacles at the spring follow-up session, then the child was assigned the status of having provided incomplete data for the analysis. Also assigned to "incomplete data" status was the one child in whom no keratometry measurement was obtained. All three children who wore spectacles for acuity testing in the fall had sufficient corneal astigmatism to fail autokeratometry screening at all threshold levels tested and therefore they were not assigned to incomplete data status, based on visual acuity results.
Table 3 lists the observed attributes of the children by retrospective application of the hybrid screening program. Children who had significant refractive error, herein defined as 1.50 D or more refractive astigmatism, were classified as either having a true-positive (failed keratometry, or passed keratometry and failed two acuity tests), false-negative (passed keratometry and at least one acuity test), or incomplete-positive (passed keratometry and failed the fall acuity test, but did not have results from the spring acuity test) screening classification. Children without significant refractive error were classified as true-negative (passed keratometry and at least one acuity test), false-positive (failed keratometry, or passed keratometry and failed two acuity tests), or incomplete-negative (passed keratometry and failed the fall acuity test, but did not have results from the spring acuity test) screening classification.
|
Table 4 shows the range of sensitivities and specificities that result from analysis of the data set at various referral levels at or above the target sensitivity of 90%. Lower- and upper-bound estimates are based on exclusion and inclusion, respectively, of the incomplete subjects in the numerator for the calculation (see Table 4 for formulas).
|
In screening for a 20/80 acuity referral threshold, a child would be presented with a 20/63 line. If the child could see that line, it would imply that he/she could see the 20/80 line as well. If the child could not see the 20/63 line, then the acuity would be presumed to be 20/80 or worse, warranting referral.
Finally, although the design target sensitivity was 90%, the set of referral criteria selected were anticipated to produce 91% to 92% sensitivity. This was to allow for any performance changes that might exist when screening tests were administered by a community-based tester rather than by a member of the research team and when both acuities were performed within a few days or weeks, as opposed to being separated by months. Thus, data from our original subject population suggest that the hybrid screening protocol would be expected to perform in this population with a sensitivity between 91% and 92% and specificity between 73% and 78%.
Prospective Evaluation of the Hybrid Screening Protocol
To evaluate the efficiency of the hybrid screening protocol and the selected referral thresholds, a manual of procedures was developed, and the Head Start Health Services Coordinator was trained in the use of the Nidek KM-500 autokeratometer (Marco Ophthalmic, Inc.), the Massachusetts Visual Acuity Test (MASSVAT) Lea symbols (Precision Vision, Inc., La Salle, IL), which is a flip chart with one line of symbols per page,27 and the hybrid screening protocol. The Health Services Coordinator then independently implemented the hybrid screening program in a cohort of 167 Head Start program participants during the fall of 2000. On a separate day, each child also underwent a complete eye examination including cycloplegic refraction by the research team.5 24 The research adhered to the tenets of the Declaration of Helsinki, informed consent was obtained from parents or guardians of all subjects, and the research was approved by the institutional review board of the University of Arizona.
|
Results |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
1.50 D) in this population.
|
To evaluate the potential negative effect on specificity that occurred by including the visual acuity screening component in the hybrid screening program, we compared the performance of the hybrid screening program with the performance that would have been obtained if autokeratometry screening alone had been used. Sensitivity remained at 96.8%, whereas specificity increased to 80.2% because there was one child without astigmatism who passed autokeratometry screening but failed visual acuity screening twice. Thus, it appears that the hybrid screening program, which combines autokeratometry and visual acuity screening does so with no significant loss of specificity in this population (compared with autokeratometry screening alone).
|
Discussion |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
The hybrid screening program that was implemented was able to incorporate visual acuity screening without significant degradation of specificity, because the purpose of the acuity screen is not to detect the need for spectacles, but rather the presence of significant eye problems that may arise from reasons other than astigmatism. How useful is it to detect acuity worse than 20/63, the level at which referral is instituted in the hybrid screening program? In the Amblyopia Treatment Study, which included children between 3 and 7 years of age with moderate strabismic or anisometropic amblyopia (acuity of 20/100 or better in the amblyopic eye),28 the amblyopic eye had acuity of 20/80 or worse at the time of enrollment in 65% of children less than 4 years of age, in 49% of children from 4 to 5 years of age, and in 45% of children from 5 to 6 years of age. Thus, a referral acuity threshold of vision worse than 20/63 might be expected to have about a 50% chance of detecting moderate amblyopia arising from either strabismus or anisometropia.
The decision to screen preschool-aged children for a visual acuity value other than the commonly recommended value of 20/40 was made with some trepidation. The ROC curve shows that visual acuity screening alone, when performed with 90% sensitivity, has only 50% specificity. We have shown in this population9 that when the conventionally recommended referral criterion of worse than 20/40 acuity in either eye or a two-line difference between eyes is applied, sensitivity of 91% and specificity of 44% was found. We modified the conventional recommendation in two ways. First, we changed the criterion to visual acuity worse than 20/63 in either eye and eliminated the intereye difference as a referral criterion. Second, we gave each child a second chance to recognize three of five 20/63 Lea symbols in a line. Preschool children frequently (>30% of time) improve a line or more on the second test of acuity,18 and so a second attempt to reach the minimum visual acuity threshold improves the specificity, but not to the degree that use of an automated instrument does. The hybrid screening method presented allows visual acuity testing to be performed for the detection of significant visual acuity decrease that occurs because of a mechanism that cannot be detected by the automated keratometry screener. We believe that as new modalities develop for detecting risk factors for amblyopia, there remains a role for visual acuity testing as part of the screening process. Only visual acuity measurement tests the global visual performance of the child, but the variability in test performance both within a child and between children makes acuity screening at 20/40 inherently unreliable, compared with other automated methods of screening. We have demonstrated a hybrid screening protocol that effectively incorporates both behavioral and structural testing.
In this unique population of children with astigmatism, the Photoscreener (MTI, Riviera Beach, FL) was not cost-efficient at any volume for the target sensitivity of 90%. It appears that this is a consequence of the relatively low specificity achieved for a comparably high sensitivity, and the tendency for trained raters to demonstrate significantly higher specificities (in this population, 88%–89%) than sensitivities (in this population, 57%–58%).8 24 Our ROC analysis suggested that at 90% sensitivity, only 35% specificity would be expected. Examination of the ROC curve (Fig. 1) demonstrates how the Photoscreener becomes more efficient than LSVAS for sensitivities lower than approximately 80%. It is not apparent how to recalibrate the photograph interpretation process, for there is no ready method to adjust referral criteria in photoscreening, given the nonlinear interaction between pupil size, magnitude of refractive error, and crescent size.29
In summary, we have designed and validated a hybrid screening program to detect the need for spectacle wear in a population of Native American children who have a high prevalence of astigmatism (approximately 33%). The hybrid screening program detects the need for spectacle correction through the use of automated instrumentation, while retaining the ability to detect other visual pathway conditions that result in reduced acuity that are not detected by keratometry. Although the screening protocol introduced here is appropriate only for a preschool population with a high prevalence of astigmatism, the methodology used to design the protocol can be applied in other populations with problems that warrant screening.
|
Acknowledgements |
|---|
|
Footnotes |
|---|
Supported by National Eye Institute Grant EY11155 (JMM).
Submitted for publication September 20, 2002; revised January 7, 2003; accepted January 30, 2003.
Disclosure: J.M. Miller, None; V. Dobson, None; E.M. Harvey, None; D.L. Sherrill, None
The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked "advertisement" in accordance with 18 U.S.C. 
1734 solely to indicate this fact.
Corresponding author: Joseph M. Miller, Department of Ophthalmology, University of Arizona, 655 N. Alvernon Suite 108, Tucson AZ 85711; jmiller{at}eyes.arizona.edu.
|
References |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  D. H. Tingley Vision Screening Essentials: Screening Today for Eye Disorders in the Pediatric Patient Pediatr. Rev., February 1, 2007; 28(2): 54 - 61. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
