| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH |
|
|
|
|||||||
|
|||||||||
Electronic Letters to:
|
|
Electronic letters published:
![[Read eLetter]](/icons/misc/sectionDOWN.gif){kind=icon}
Erratum
A Comparison of Raman Spectroscopy and Heterochromatic Flicker Photometry
Author Response: A Comparison of Raman Spectroscopy and Heterochromatic Flicker Photometry
|
|
|||
|
Kumari Neelam
Send letter to journal:
neelamk{at}sehb.ie Kumari Neelam
|
Erratum in: Neelam et al., Author Response: A Comparison of Raman Spectroscopy and Heterochromatic Flicker Photometry, http://www.iovs.org/cgi/eletters/46/3/1023#270. Revised Scatter Plot Attached is a revised scatter plot of that which was published in our previous letter, consisting of MP measurements from 118 subjects (n = 120 subjects). Of note, MP was not recorded in the right eye of two subjects because of the presence of amblyopia. However, the missing data from these two eyes were inadvertently included, and attributed values of zero (value), during the importation of data from Excel to SPSS for the purposes of generating a scatter plot of the square root values, thus resulting in an erroneous Pearson correlation of 0.399 (original response). Please note that once these two points are excluded manually, the strength of the correlation is substantially attenuated (Pearson correlation = 0.294); however, the positive relationship between the two techniques remains statistically significant (p = 0.001).
CAREDS Protocol In our original response to Wooten and Hammond's letter, we cited the median time for the completion of the CAREDS protocol as 45 minutes (range: 29-60 minutes). However, this time referred to the inclusion of multiple measurement points, and a standard measurement takes about 20 to 30 minutes only; on this point, we stand corrected. |
|||
|
|
|||
|
Billy R. Hammond
Send letter to journal:
bhammond{at}egon.psy.uga.edu Billy R. Hammond
|
We appreciated the article by Neelam et al.1 comparing heterochromatic flicker photometry (HFP) and Raman spectroscopy (RS). Like the authors, we feel that there is a need for valid non-invasive measurement of macular pigment (MP). The authors discuss a number of issues in their comparison of HFP and RS: reliability, time needed for measurement, etc., and especially whether the two methods agree. They conclude that the two methods "demonstrated good correlation . . . thus authenticating RS against a validated psychophysical technique of measuring MP." Using the same data and analyses, however, we came to the exact opposite conclusion. Because of the importance of the topic, we feel it is imperative to outline the basis for our different interpretations. With regards to evaluating reliability, the stimulus conditions used by Neelam et al. for assessing HFP were inadequate. Like all methods, the results achieved with HFP rely on an optimal optical apparatus and an optimal procedure. Concerning the latter, the authors note, "Perfectionist adjustment of the control was strongly discouraged, as the point of no flicker cannot be achieved."1 In our studies using HFP, subjects can easily reach a point of no flicker because we optimize the flicker rate for each subject. Neelam et al. used a fixed flicker rate for all subjects (11-12 and 6-7 Hz in the fovea and parafovea, respectively). Flicker rate determines the characteristics of perceived flicker. At an ideal physical flicker rate, perceived flicker will disappear at a point value of matching radiance. In contrast, if the physical flicker rate is too high (compared with the optimal), then there would be a zone (not a point) where perceived flicker disappears. If, on the other hand, the physical flicker rate is too low, then flicker could be minimized but not be made to disappear. Thus, depending upon flicker rate, a given observer would judge minimum flicker, no flicker, or the end-points of a relatively wide no-flicker zone. Since there is considerable individual variation in temporal vision across subjects, across the retina, and particularly across age, failure to optimize flicker rates almost certainly would yield less reliable results. Procedures have been developed (based on using CFF thresholds) that easily and quickly allow optimization of the null zone and yield much higher reliability estimates as a result.2 For example, if optimal HFP procedures are used, the coefficient of variation appears to be similar between HFP and the Raman methods when tested on the same subjects (12% and 15%, respectively) (Hogg RE, et al. IOVS 2004;45:ARVO E-Abstract 1287). The authors state that HFP is time-consuming and associated with high variability in subjects with low MP optical density, citing Snodderly and Hammond.3 Snodderly and Hammond did not make this claim. Rather, they note that the average absolute change across testing sessions ranges between about 0.05-0.08 OD for practiced subjects. Furthermore, these values are independent of MP density. Neelam et al. apparently inferred that this amount of error would disproportionately impact subjects with low MP density. For every method of measuring MP, however, very low true values are large with respect to a constant error of estimate that is independent of the absolute value (e.g., even if the measurement error were only 0.01, this would still represent 100% error for someone with a true MP density of 0.01). It is not, however, the ratio of errors to the true value that is critical, but rather the absolute magnitude of the difference between the estimate and the true value (e.g., with a 0.01 error one would still be very close to a true MPOD value of 0.01). A further consideration is that the difference should be small in some defensible sense. Small in this sense can be interpreted with respect to the range in values both between subjects and across the retina. For example, if MP ranges from about 0-1.2, an absolute error of about 0.07 is not excessive. Regarding HFP being time-consuming, a standard measurement (i.e., 1-degree stimulus) using the CAREDS protocol in elderly participants takes only about 20-30 minutes2; this time includes instruction, practice, determination of optimal flicker rates, and experimental measurement. The question of whether the Raman method is faster than HFP procedures has not actually been tested (e.g., dilation, alignment, and time for after-image recovery between flashes all must be accounted for when using RS). In discussing their RS measurements (Raman counts, RCs), the authors note that their average RCs (1123) are almost twice as high as those reported from a Utah sample (685) of similar age.4 The authors speculate that this difference "may reflect differences between the two populations studied (American versus European) in terms of dietary and ethnic factors."1 The authors, however, did not identify the ethnic composition of their sample, but we assume that, like the Utah sample, it is largely Caucasian. In any event, ethnicity has not been shown to influence MP levels.5,6 Diet seems an unlikely explanation since the average serum lutein and zeaxanthin levels and MP OD (measured using HFP) of their sample are similar to those that have been reported in the Midwestern United States (Monterosso G, et al. IOVS 2005;46:ARVO E-Abstract 1758).7 The authors were apparently motivated to believe that the RCs were correct in an absolute sense and preferred a sampling explanation (contraindicated by both their HFP and serum data). The problem with this view is that, in the absence of a valid calibration procedure (such as the use of a known standard that is directly comparable to the unknown, e.g., HPLC procedures), the RC is an arbitrary number that can vary across studies due to many factors such as the specific arrangement of stimulating and collecting optics, laser intensity and duration, head stabilization, etc. Furthermore, a recent RS study on a different European population provided an average RC of 2136 (age range = 25-55 years).10 A RC range of 3 to 1 in comparing similar European and Utah populations is likely accounted for by factors other than MPOD. As we have noted in our past discussion of this issue,9,10 one highly variable feature that will also influence both the Raman input and output beams (488 and 527 nm, respectively) is absorbance and scatter by the crystalline lens (a factor not considered by Neelam et al.). Obviously, since RS relies on measuring an absolute signal, any factor that attenuates the input or output beams will directly confound Raman results. One such factor that the authors did consider was occlusion by the iris (although how pupil size was measured is not mentioned). Since the exit pupil of the Raman system is 7 mm, precise alignment with the pupil is necessary. Pupils smaller than 7 mm will cause occlusion of the incident beam by the iris. Pupils exactly 7 mm will be associated with iridial occlusion if there are even slight misalignments. Any eye displacement in excess of 1 mm will be associated with occlusion for subjects with pupils dilated to 8 mm. Such occlusions are apparently why 40% of the RS determinations are routinely discarded. It is also clear that removing two out of the five data points because they may be confounded, as is standard in RS, does not ensure that the remaining three are not confounded. Based on the authors' Figure 2, the majority of the RCs in their study were confounded with pupil size. In fact, when pupil size was accounted for, the large age-decline in RCs was "dramatically attenuated" and statistically nonsignificant. Although the authors showed that the age-decline in RCs was due to the confounding influence of the pupil, they nonetheless discuss the age-decline as if it were due to a decline in MP. Throughout the manuscript, the authors appear to discount statistical evaluation in the discussion of their results. For instance, neither method, RS or HFP, provided results that correlated with serum L and Z (RS, p = 0.36, HFP, p = 0.54) or gender. Nonetheless, the authors conclude that there was a "good degree of correlation" between the two methods and that there was a "good correlation between RS and HFP readings. . . and relevant variables such as age, gender, and serum concentrations of L and Z."1 The authors assert that these results, none of which were statistically significant (except for RC versus age for the whole sample before the confounding effect of pupil size is accounted for), validate the RS method. Inferential statistics and statistical conventions are designed to provide the basis for scientific conclusions. One cannot make meaningful conclusions based on insignificant results. The authors also make some common errors in the interpretation of their Bland-Altman analysis. The Bland-Altman (B-A) analysis is an appropriate test for evaluating the consistency of two measurement techniques. It is a useful addition to the standard regressional comparison in that it more easily reveals systematic bias between two methods (sometimes not evident in a pure correlational plot). This does not mean, however, that a correlational analysis is irrelevant (e.g., the National Committee for Clinical Laboratory Standards recommends the B-A analysis as an adjunct to regressional analysis and the scatter plot).11 Clearly, two methods that are poorly correlated do not compare well (conversely, a high r does not necessarily mean that two methods are interchangeable). Furthermore, correlations use standardized values allowing direct covariation to be evaluated. The B-A analysis requires that the measures have the same units. Since RCs are themselves arbitrary numbers, the authors converted them to OD units by assuming that the RC range of -500 to 3500 matches an OD range of 0 to 1.0. This procedure is, as the authors state, "arbitrary." We feel that we must also add that it is invalid. The authors did not use the published calibration curves. As opposed to the linear optical density conversions that the authors assumed, the true relation between RC and MPOD is strongly nonlinear. This is shown by direct in vitro measurements. Although not a valid model of the real eye, external calibrations can show how Raman signals are returned from carotenoids of varying density embedded within some material. For example, as shown by the in vitro calibration curve in Bernstein et al.,12 RCs and ODs are effectively linear only up to about 0.30 OD. Using a least squares linear fit to the lowest five data points (if the sixth point is included, the value is clearly below the best-fitting linear curve) in Bernstein's calibration curve (and correcting the offset so that the origin of both variables are zero), RCs are underestimated by about 23% at 0.50 OD and about 36% at 1.0 OD. This lack of linearity is due to the decreasing ability of the stimulating laser to penetrate the MP carotenoids at higher densities. Based on these arbitrary conversions, the authors nonetheless generated a standard B-A plot (see their Fig. 3). This plot forms the basis of their conclusion that RS and HFP yield similar values. The authors consider that, since almost 95% of the values fall between +/- 2 standard deviations of the no-difference line, the Raman and HFP measurements are in acceptable agreement. As noted by Altman and Bland,13 however, "sometimes the method has not been adopted with full understanding. For example, we have seen it suggested that two methods agree well because most of the observations lie within the 95% limits of agreement. The limits are calculated so that this will always be the case." The important point is that the difference plot should be interpreted according to its judged clinical and/or experimental significance. As emphasized by Bland and Altman,16 the 95% limits of agreement have to be interpreted using a pre-determined criteria defining what is meant by meaningful differences. Although the authors seem to recognize this requirement, they erroneously conclude that the differences between the methods are acceptable. We feel that the differences are so large as to preclude even categorical comparisons (e.g., high versus low). For example, using the criterion limits in their Figure 3, a subject could have a RS value of 0.20 and a HFP value of 0.60 and still be within the defined limits of agreement. We cannot agree that the two methods, differing by such magnitude, are interchangeable for the measurement of MP. In sum, the authors argue that there is an "acceptable correlation" between their results obtained with HFP and RS. Reanalyzing the data in their Figure 4, we found that the relation was quite weak, namely r2 = 0.10. In other words, one method explains only about 10% of the variance in the other method measuring the same variable in the same subjects. Based upon the poor r2 and the large values of the B-A limits of agreement, we conclude that the two methods are in poor agreement. However, we applaud their efforts to validate these important methods. More study is needed to determine what factors underlie the large differences. Billy R. Hammond1 1Vision Science Laboratory, University of Georgia, Athens, GA References 1. Neelam, K, O'Gorman N, Nolan J, et al. Measurement of macular pigment: Raman spectroscopy versus heterochromatic flicker photometry. Invest Ophthalmol Vis Sci. 2005;46:1023-1032. |
|||
|
|
|||
|
Kumari Neelam
Send letter to journal:
neelamk{at}sehb.ie Kumari Neelam
|
An erratum has been published. We welcome the interest that Hammond and Wooten have shown in our recent article.1 The authors address, point by point, each concern that they have raised, as follows: Heterochromatic Flicker Photometry We used a fixed frequency rate of 18 Hz at the fovea and 13 Hz at the parafovea, not the frequencies mistakenly given in our article (11-12 Hz: fovea; 6-7 Hz: parafovea), which represent those used by other instruments. Further, our instrument has been validated against motion photometry in normal subjects.2 Undoubtedly, measuring critical flicker fusion frequency for each subject would be ideal; however, it is not practical under all circumstances. Consequently, frequencies were chosen partly from what had been reported in literature by previous authors and partly from what early subjects reported as satisfactory.3,4 When macular pigment (MP) is measured using heterochromatic flicker photometry (HFP), most subjects report that some flicker is perceived at all settings and that at no point does the flicker disappear completely. This is to be expected because, even at isoluminance, the target fields will change from blue to green. In addition, for the foveal test field, the MP is not evenly distributed across the 1-degree area that it subtends on the retina, and hence no setting can be perfect for the whole area at once. Indeed, the inability to achieve zero flicker is not unique to this instrument, as shown by the description of the end-point as "zero or minimum flicker" in prior published studies by other authors.5 Also, Wooten et al.6 have used the phrase "null zone of no flicker or . . . minimize the flicker" repeatedly in the Methods section of the Carotenoids and Age-Related Eye Disease Study (CAREDS) protocol. Furthermore, 5% of subjects with normal vision are unable to perform minimum match settings due to difficulty in grasping the concept of minimum flicker. These subjects try to make minute adjustments of the blue luminance so as to achieve a perfect setting and are reluctant to stop fixation and relax by looking around the room. It is important to note that the subjects spend no more than 15-20 seconds fixating the test fields before relaxing: Troxler phenomena eventually bother subjects who continuously fixate, and this process of relaxing and looking away from the test fields is written into the SOPs (Standard Operating Procedures) for our instrument. It is for these reasons that "perfectionist" adjustment is strongly discouraged, and for most subjects, after a few non-recorded trials, the real measurements are easily made. According to the CAREDS protocol, "HFP requires . . . a significant portion of the time devoted to a typical clinic visit or research study appointment."6 Also, the median time for the completion of the CAREDS protocol was 45 minutes (range: 29-60 minutes). In the same study, the authors have reported, "When the subject has a low MPOD [macular pigment optical density], the measurement of density at wavelengths other than the peak have a large uncertainty because of the inherent variability in the measurements."6 Raman Spectroscopy To date, ethnicity has not been investigated in a controlled fashion, and the impact of ethnicity on the MPOD has not been the primary outcome measure in either of the articles cited by Hammond and Wooten. Also, according to the article by Tang et al.,7 the measurement of MPOD was done by an invalidated custom-built HFP device. Furthermore, Hammond et al.8 have observed MPOD to be related to iris color, which is inherently associated with ethnicity. In other words, the matter of ethnicity and MPOD is unresolved and remains a possible explanation for the discrepancy between those and our readings. However, we agree with Hammond and Wooten that a possible explanation for our high Raman count (RC) values may be the sensitivity of our machine. For this reason, we have generated our own calibration curves to convert our RC measurements using a model eye from the University of Utah, Salt Lake City.9 We have described the age-related decline in the manuscript because our HFP values, a validated technique, indicate that there may be an age-related decline in MPOD. Certainly, the relationship between age and MP is one that still remains a matter of debate. Although we cannot draw conclusions based on insignificant relationships, parallel and insignificant relationships recorded on the two instruments are more desirable than a situation where observed relationships were significant with one instrument and insignificant with the other instrument. Bland-Altman Plot Hammond and Wooten are mistaken when they say that Bland-Altman (BA) plots are a useful addition to the standard regression comparison. In the analysis of the measurement method comparison data, neither the correlation coefficient nor the techniques such as regression analysis are appropriate and may be misleading. The reason is that the correlation coefficient (r) measures the strength of a relation between the two variables, not the agreement between them, and therefore the high correlation cannot be equated to a good degree of agreement. Furthermore, a high correlation may in fact conceal considerable lack of agreement between the two instruments. Therefore, a much better technique for the analysis of comparison data is the use of BA plots.10 However, we respect Hammond and Wooten's opinion, and therefore we have now correlated Raman counts and HFP MP readings. The results suggest an acceptable and inescapable positive relationship between the two techniques of measuring MP (Pearson correlation = 0.399; p = 0.000; Fig. 1). Of note, we have used the square root values of the original MP readings measured with both instruments to eliminate the skewness existing in the data.
We have, now, converted our RC values into MPOD values using calibration curves generated with the Raman spectroscope present in our laboratory. For this analysis, we limited the comparison of Raman and HFP values to subjects with RC values below 1500 because the model eye calibration curve was strictly linear for values below 1500 (r2 = 0.916). Our calibration curve leveled off at values below 2000, and this may be due to inherent optical limitations of the model eye system because considerably higher counts are achievable in human subjects. We have accordingly reanalyzed our data, and this resulted in no significant change to our BA plots. When comparing two instruments, the agreement between readings taken on two different instruments provides information as to whether the two instruments are interchangeable. We have clearly stated in our concluding paragraph, "investigators should use only one of these instruments for the duration of any given study because of differences in the scientific rationale, and the factors that influence RS and HFP measurements of MP." For example, HFP measures MP at a single point relative to a peripheral zero point, while Raman measures a spatially integrated level of MP in the region sampled. Thus, the two MP measurement techniques should not be considered to be equivalent, although reasonable correlation/agreement is to be expected. Furthermore, we agree that there may be many optical effects that will affect RS readings but that these influences will be minimized where the data relate to measurements within subject over time. We commend Hammond and Wooten on their vigilance with respect to the studies where MP levels have been measured using RS11-13 and welcome such debate. Kumari Neelam1,2 1Waterford Institute of Technology, Waterford, Republic of Ireland References 1. Neelam K, O'Gorman N, Nolan J, et al. Measurement of macular pigment: Raman spectroscopy versus heterochromatic flicker photometry. Invest Ophthalmol Vis Sci. 2005;46:1023-1032. |
|||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH |
eLetters: 
