| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH |
|
|
|
|||||||
|
|||||||||
Electronic Letters to:
|
|
Electronic letters published:
![[Read eLetter]](/icons/misc/sectionDOWN.gif){kind=icon}
Kappa Misleading
Author Response: Kappa Misleading
|
|
|||
|
Aljoscha S Neubauer, MD Dept. of Ophthalmology, Ludwig-Maximilians University Muenchen, Germany
Send letter to journal:
aljoscha.neubauer{at}med.uni-muenchen.de Aljoscha S Neubauer
|
We read with great interest the article published by Strøm and Sander in a recent issue of IOVS.1 Having researched objective retinal imaging techniques such as the retinal thickness analyzer (RTA) and optical coherence tomograph (OCT) for several years we find the results not at all astonishing and certainly "technically true." However, the study suffers from a major structural problem making its results trivial: it compares a new technology known to be more exact (RTA and in a previous similar paper OCT) to a not so exact existing one (clincial assessment). This means that the correlation of those two (here "kappa"-coefficient) must be low. Indeed, a low kappa has to be postulated if the new technology really measures more exactly than the current gold standard. This makes this approach very problematic. In glaucoma research this issue is well known for many years: the correlation/kappa of Goldmann tonometry compared with the old standard, the Schiötz showed no good correlation. Still the Goldmann method is the more exact one.2 Therefore a comparison of kappa to assess and judge new technology such as RTA or OCT does not appear appropriate to us. Moreover, in the paper stereo photographs - instead of indirect ophthalmoscopy - are used as a reference. It is well known that ophthalmoscopy is more sensitive to detect retinal thickening than are stereo photographs,3 a problem which is addressed in the paper but further biases the results. For the appropriate standard method, stereoscopic biomicroscopy, recent studies could prove that only relatively large retinal thickening can be detected clinically.4,5 Small amounts of thickening are not detected clinically but measured on RTA and OCT, of which the latter is believed by Strøm and Sander to correlate better in terms of "kappa" (see discussion section and Ref. 6). When looking at the methods section and Table 3 of their current paper indeed a high percentage of eyes with such a low amount of retinal thickening is found, which fully explains why their study design cannnot give high kappas. This is nicely reflected in a study recently published in Ophthalmology4 comparing OCT and clinical assessment, where relatively low kappa values (ranging from 0.36-0.63) are found for OCT and discussed in detail. When faced the same problem with kappa we finally decided to compare OCT to RTA instead of to clinical assessment or stereo photography.7 A good correlation makes sense in this situation. In agreement with the data published by Strøm and Sander, we showed in a recently published study8 that the RTA maps detected more cases of retinal thickening than stereoscopic fundus examination (100% sensitivity) but did not correlate well (low specificity) with clinical assessment by funduscopy. However, such results do not mean that RTA (or OCT) is less valuable than clinical assessment but on the contrary prove that they are more sensitive. In summary: for new technology sometimes low kappas are better than high ones! Aljoscha S. Neubauer Ludwig-Maximilians-University, Dept. of Ophthalmology, Muenchen, Germany References 1. Strøm S, Sander B. Comparison of objective retinal thickness analysis and subjective stereo fundus photography in diabetic macular edema. Invest Ophthalmol Vis Sci. 2004;45:1450-1455. |
|||
|
|
|||
|
Charlotte Strom
Send letter to journal:
charlotte_strom{at}dadlnet.dk Charlotte Strom
|
We appreciate the opportunity to respond to the comments of Neubauer et al. to our paper that appeared recently in IOVS.1 Neubauer and co-workers argue that it is problematic to compare a presumably sensitive and exact method like the RTA with a more imprecise method like stereo fundus photos, and that small focal retinal thickening as seen in the study population will not be found by subjective assessment of stereo fundus photos – and thus a low kappa value will be the result. We agree that slit lamp biomicroscopy is the better way of assessing retinal thickening. For study purposes it is, however, not a feasible method to use. The method of subjective assessment of stereo fundus photos of diabetic retinopathy has throughout the years proven its predictive value.2 Important studies like the ETDRS have – apart from the valid input on when to conduct timely laser photocoagulation treatment in eyes with diabetic macular edema3 – also given us a substantial amount of information on the methodology used during these studies. Moreover stereo fundus photography has proven to be a reliable and reproducible method in large scale prospective clinical trials.4 Indeed, manual assessment of stereo fundus photos is a subjective method, however it has been accepted by clinicians worldwide and is also used to grade non-clinically significant macular edema. RTA and OCT are introducing new perspectives in that they provide objective information. Even if a comparison between RTA and OCT is more satisfying from a theoretical standpoint, new methods, in any form and with whatever improvements over the standard method they may offer, must be validated in order for us to understand exactly where, and why, they differ from the standard method. Using the OCT and the RTA in a standardized manner in prospective studies of macular edema, and side by side stereo fundus photography, will answer many more questions, and to our knowledge these comparisons are now being implemented. While awaiting the results of these prospective studies, our cross-sectional studies on RTA, OCT,5 and fundus photos merely suggest what we may expect from the prospective studies. Neubauer and co-workers state, "Small amounts of thickening are not detected clinically but measured on RTA and OCT" and that the RTA is more sensitive in measuring retinal thickening. Table 3 in the paper clearly indicates that it is a misunderstanding that the RTA measures, e.g., small amounts of retinal thickening more exactly, in that it shows 18 eyes in which the RTA detects no retinal thickening where retinal thickening is detected by subjective evaluation of stereo fundus photos. Four eyes are found to have retinal thickening with the RTA and not with the stereo fundus photos. So this hardly supports the RTA to be a more sensitive tool even though photographic grading is not perfect. As stated in the Discussion section of our paper we strongly believe that as a minimum the objective tool should be capable of detecting retinal thickening where it supposedly is present and detected by the means that are currently considered "gold standard," (fundus photos). In our opinion the best explanation for the lack of agreement between the RTA and fundus photos is due to scatter problems, while the comparison between OCT and fundus photographs5 shows substantially greater agreement. Charlotte Strøm and Birgit Sander Department of Ophthalmology, Herlev Hospital, University of Copenhagen, Denmark References 1. Strøm C, Sander S. Comparison of objective retinal thickness analysis and subjective stereo fundus photography in diabetic macular edema. Invest Ophthalmol Vis Sci. 2004;45:1450-1455. |
|||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH |
eLetters: