|
|
||||||||
1 From the Hamilton Glaucoma Center and the 2 Institute for Neural Computation, University of California, San Diego, La Jolla, California; and the 3 Computational Neurobiology Laboratories, The Salk Institute, La Jolla, California.
| Abstract |
|---|
|
|
|---|
METHODS. With the HRT, one eye was imaged from each of 108 patients with glaucoma (defined as having repeatable visual field defects with standard automated perimetry) and 189 subjects without glaucoma (no visual field defects with healthy-appearing optic disc and retinal nerve fiber layer on clinical examination) and the optic nerve topography was defined by 17 global and 66 regional HRT parameters. With all the HRT parameters used as input, receiver operating characteristic (ROC) curves were generated for the classification of eyes, by three neural network techniques: linear and Gaussian support vector machines (SVM linear and SVM Gaussian, respectively) and a multilayer perceptron (MLP), as well as four previously proposed linear discriminant functions (LDFs) and one LDF developed on the current data with all HRT parameters used as input.
RESULTS. The areas under the ROC curves for SVM linear and SVM Gaussian were 0.938 and 0.945, respectively; for MLP, 0.941; for the current LDF, 0.906; and for the best previously proposed LDF, 0.890. With the use of forward selection and backward elimination optimization techniques, the areas under the ROC curves for SVM Gaussian and the current LDF were increased to approximately 0.96.
CONCLUSIONS. Trained neural networks, with global and regional HRT parameters used as input, improve on previously proposed HRT parameter-based LDFs for discriminating between glaucomatous and nonglaucomatous eyes. The performance of both neural networks and LDFs can be improved with optimization of the features in the input. Neural network analyses show promise for increasing diagnostic accuracy of tests for glaucoma.
| Introduction |
|---|
|
|
|---|
In an attempt to classify eyes effectively as glaucomatous or healthy, analysis strategies have been developed that use as input different CSLO optic disc topography measurement parameters, by using statistical methods such as linear discriminant function (LDF) analyses.8 10 11 12 13 14 LDF analysis assumes that data representing different groups are linearly separable. If this assumption is not well met, the classifiers performance is degraded. Other investigators have used artificial neural networks (specifically, multilayer perceptrons [MLPs] with back-propagated learning) trained on CSLO parameters to classify eyes as glaucomatous or healthy.8 15 Using this method, the neural network classifier is trained to detect a relationship between input (CSLO parameters) and a predefined gold-standard diagnosis by comparing its prediction with the labeled diagnosis and by learning from its mistakes. In general, neural network techniques differ from basic statistical techniques such as LDFs, because they can adapt to the distribution of the data rather than assume a predefined distribution. The success of statistical or neural network classification methods is most often measured by reporting areas under the receiver operating characteristic (ROC) curve or by reporting sensitivity at different specificities.
The purpose of the current study was to compare the performance of previously proposed HRT parameter-based LDFs with three artificial neural network methods in a single sample. Comparing different classification methods in a single sample reduces the effects of confounding variables, such as subject demographics and severity of glaucoma. Because of their adaptability, we hypothesized that neural network techniques would perform as well as or better than LDF classifiers in discriminating between glaucomatous and healthy eyes.
| Methods |
|---|
|
|
|---|
Because CSLO-measured optic disc topography was being evaluated, we chose the best indicator of glaucoma that is not dependent on optic disc appearance for training and evaluating the neural network techniques. Patients with open-angle glaucoma were defined as those with at least two consecutive SAP fields with either a corrected pattern standard deviation (CPSD) outside the 95% normal limits or a glaucoma hemifield test (GHT) result outside the 99% normal limits. At least one of the abnormal fields was obtained on or before the date of CSLO imaging. Mean deviation (±SD) of the SAP closest to the CSLO imaging date was -6.08 ± 5.77 dB, indicating mild to moderate visual field damage. Patients with glaucoma had no history of diabetes and no apparent cataracts and were not using medication known to affect visual sensitivity at the time of visual field testing. Best corrected visual acuity at the time of SAP and CSLO testing was 20/40 or better. The average age (±SD) of patients with glaucoma was 65.2 ± 13.6 years.
Healthy eyes had a measured IOP of 22 mm Hg or more with no history of elevated IOP. These eyes had intact rims, no evidence of hemorrhage, notching, glaucomatous excavation, or RNFL defect and had symmetrical optic discs (asymmetry of vertical cup-disc ratio < 0.2) based on clinical examination. SAP results were within normal limits. Healthy patients had no history of diabetes or other systemic disease and no ophthalmic or neurologic surgery or disease. Best corrected visual acuity at the time of testing was 20/40 or better. The average age (±SD) of healthy subjects was 54.2 ± 16.3 years, significantly younger than patients with glaucoma (t-test; P < 0.05).
Confocal Scanning Laser Ophthalmoscope
The Heidelberg Retina Tomograph (HRT-1, Heidelberg Engineering, Heidelberg, Germany) provides topographical measures of the optic disc and parapapillary retina, with confocal scanning laser technology. The topographical image is derived from 32 optical sections at consecutive focal depth planes. Each image consists of 256 x 256 pixels with each pixel corresponding to retinal height at its location. This instrument has been discussed in detail elsewhere.1
16
Procedure.
Three 15° field-of-view scans centered on the optic disc and judged to be of acceptable quality were obtained for each test eye. A mean topography image of these three scans was created with the HRT. The optic disc margin was outlined on the mean topography image by a trained technician using information obtained by viewing stereoscopic photographs of the optic disc.
HRT Parameters.
Eighty-three topographic parameters (automatically provided by HRT software, ver. 2.01) were used in this study (Table 1)
. We used global (360-degree) measures for each parameter and for some parameters, also used regional measures. Regions were defined as temporal superior (4690° unit circle), nasal superior (91135°), nasal (136225°), nasal inferior (226270°), temporal inferior (271315°), and temporal (31645°). Regional parameters were not evaluated for height variation of contour, mean cup depth, RNFL thickness, RNFL cross-sectional area, reference height, or rim area. All these parameters have been discussed in more detail elsewhere.1
16
17
18
|
We also developed and evaluated an LDF (called "current" LDF) that used all 83 parameters as input. This LDF was developed and tested, with 10-fold cross-validation used to reduce bias in developing and testing on the same samples (described later).
Neural Network Techniques
We evaluated the performance of three artificial neural network techniques for classifying eyes as glaucomatous or healthy. For all neural network techniques, all HRT parameters described earlier were included initially in the training set. Details and mathematical descriptions of the neural network techniques used have been described elsewhere by us and by others.19
20
21
22
23
24
Multilayer Perceptron.
The MLP, a feed-forward back-propagation network, is the most frequently used neural network technique in glaucoma research. Researchers have used this method to assess optic disc topography,8
15
to interpret and classify visual fields19
25
26
27
28
and to detect visual field progression.29
Briefly, MLPs are supervised learning classifiers that consist of an input layer (multiple HRT parameters, in our case), an output layer (glaucoma or not glaucoma, in our case), and one or more hidden layers that extract useful information during learning and assign modifiable weighting coefficients to components of the input layer. In the first (forward) pass, weights assigned to the input units and the nodes in the hidden layers and between the nodes in the hidden layer and the output, determine the output. The output is compared with the target output (binary glaucoma or no glaucoma). An error signal is then back propagated and the connection weights are adjusted correspondingly. During training, MLPs construct a multidimensional space, defined by activation of the hidden nodes, so that the two classes (glaucoma, not glaucoma) are as separable as possible. The separating surface adapts to the data.
We used a 10-unit MLP in the present study constructed in a commercial software program (Neural Network toolbox ver. 3.0 of Matlab; The MathWorks, Inc., Natick, MA). Input nodes fed into a 10-node hidden layer activated by hyperbolic tangent functions. Output was a single node with a logistic function for glaucoma (1) and healthy (0) eyes. Training was accomplished with the Levenberg-Marquant enhancement of back propagation. We evaluated MLPs with different numbers of units and found the 10-unit MLP performed best, as measured by performance of cross-validation.
Linear Kernel (SVM linear) and Gaussian Kernel Support Vector Machine (SVM Gaussian).
SVMs are newly developed techniques used for solving classification and regression problems. SVM architecture resembles the architecture of MLPs (input layer, hidden layer, output layer). During training, the SVM nonlinearly maps the training data to a high dimensional space where a hyper plane is fit that maximizes the margin of separation between classes while minimizing the generalization error (ability to generalize results from finite training set to data set), with the use of statistical learning theory. Constraints imposed on the construction of the separating surface result in a subset of training data that is involved in the decision function (called support vectors). The SVM attempts to split the positive and negative vectors to optimize the distance between the hyperplane and the nearest of the positive and negative examples. SVM linear and SVM Gaussian differ because they assume different distributions of input data. SVM linear uses linear mapping, resulting in a "dot product kernel" and SVM Gaussian uses unknown nonlinear mapping, resulting in a Gaussian kernel. Both SVM linear and SVM Gaussian have been used to classify eyes as glaucomatous or nonglaucomatous, based on visual field data.19
The SVM was programmed using the software program (Matlab, ver. 5.0; the MathWorks) and trained using Platts sequential minimal optimization algorithm. The programmer chose the parameters for penalty and the kernel by trial and error. The penalty used was C = 1.0.
Analysis
ROC curves for classifying eyes as glaucomatous or healthy were determined for all techniques. These curves describe the continuous relationship between sensitivity and specificity at specificities ranging from 0% to 100% and quantify the diagnostic accuracy of a test in a single number. An area under the ROC curve of 0.50 is equivalent to chance discrimination, and an area of 1.00 is equivalent to perfect discrimination. For SVMs and current LDF, 10-fold cross-validation was used to evaluate the classifiers. The glaucomatous and healthy eyes were each divided randomly into 10 approximately equal subsets. Ten mutually exclusive partitions were formed for cross validation (to measure the true rather than the estimated error rate) by combining one of the 10 healthy subsets with one of the 10 glaucoma subsets. One partition was used as the test set and the remaining nine partitions were combined to form the training set. The process was iterated, with each partition serving once as the test set. The results obtained for the 10 test sets were combined to generate a single ROC curve for each classification method. For MLP, cross-validation was similar, except eight partitions were used for training, one was used as a test set, and one was used as a stopping set to avoid overtraining. We provided sensitivities at specificities of 75% (representing moderate specificity) and 90% (representing high specificity), although this information is available in the graphic representations of ROC curves also presented. Finally, we reported the area under the ROC curve when specificity was 90% or more for the different techniques. These areas are bound by the ROC curve, the point at 100% specificity, and the line that passes through the point at 90% specificity and is perpendicular to the diagonal that represents chance discrimination. This information was provided to examine differences between techniques when specificity was high. The 90% specificity level was chosen because it theoretically forces the cases presumed to be the most difficult into the disease group by allowing only 10% of these cases into the healthy group.
We used the method of DeLong et al.30 to determine statistically significant differences in overall areas under the ROC curves.
| Results |
|---|
|
|
|---|
|
|
Comparing Neural Network Techniques
For MLP, the area under the ROC curve was 0.941 ± 0.01; for SVM linear, 0.938 ± 0.01; and for SVM Gaussian, 0.945 ± 0.01. No statistically significant differences between areas under the curves of neural network techniques were observed. ROC curves for the three neural network techniques are shown in Figure 2
. Areas under the curves when specificity was constrained from 90% to 100% were 0.178, 0.182, and 0.203, for MLP, SVM linear, and SVM Gaussian, respectively.
|
Comparing Linear Discriminant Functions with Neural Network Techniques
Areas under the ROC curves were significantly higher for MLP, SVM linear, and SVM Gaussian than with all previously proposed LDFs (all P < 0.01), and with the current LDF (all P < 0.03). ROC curves for the best neural network (SVM Gaussian), the current LDF, and the best previously proposed LDF (LDF Bathija et al.) are shown in Figure 3
.
|
Figures 4 and 5 show that we achieved the optimal area under the ROC curve with either forward selection or backward elimination when we were using approximately 40% of the input parameters. These figures show areas under the ROC curve (y-axis) as a function of the number of HRT parameters in the training set (x-axis). The areas were maximized with a reduced dimension data set (subset of available input parameters) that contained an optimal combination of features determined by each optimization method, compared with using the full-dimensional feature set (all available input parameters). Using forward selection, the area under the ROC curve (± SE) increased from 0.945 (± 0.01) with all input parameters, to a maximum of 0.967 (± 0.01) with 31 input parameters. When the optimal feature set was analyzed at specificities constrained from 90% to 100%, the area under the ROC curve increased from 0.203 to 0.236. Sensitivity at 75% specificity increased from 92% to 97%, and sensitivity at 90% increased from 83% to 91%. When backward elimination was used, the area under the ROC curve increased to 0.965 ± 0.01 and reached its maximum with 32 input parameters. When specificity was constrained from 90% to 100%, the area was 0.213. Sensitivity at 75% specificity was 98% and sensitivity at 90% specificity was 85%. HRT parameters included in the optimized SVM Gaussian training set with both methods are shown in Table 3 .
|
|
|
|
|
|
| Discussion |
|---|
|
|
|---|
In the present study, the nonoptimized technique that resulted in the largest area under the ROC curve for discriminating between glaucomatous and healthy eyes (area under ROC curve = 0.945 for SVM Gaussian) yielded a sensitivity of 83% at 90% specificity. In previous work, Uchida et al.8 reported an area under the ROC curve of 0.94 and sensitivity and specificity of 92% and 91%, respectively, when using a back-propagation multilayer perceptron trained with nine global HRT parameters. These results are similar to the best optimized results from the present study (optimized SVM Gaussian: 91% sensitivity at 90% specificity). Severity of glaucoma in the patients in the present study was slightly higher than that of Uchida et al. (SAP mean deviation of -6.1 and -4.8 dB, respectively). Our study is the first to investigate the performance of SVMs trained on optical imaging data for discriminating between glaucomatous and healthy eyes.
Other studies have examined the success of individual HRT parameters, linear discriminant analyses, and specially developed parameters for classifying eyes as glaucomatous or healthy. For example, Iester et al.6 and Uchida et al.8 found the HRT cup shape measure to be the best individual parameter for identifying glaucoma. These authors reported areas under the ROC curve for glaucoma detection of 0.81 and 0.84, respectively, using this parameter. These areas are slightly higher than that reported by Zangwill et al.9 (0.78) for the same parameter.
The most frequently investigated HRT parameter-based LDF is that developed by Mikelberg et al.10 Using this model, reported sensitivity for detecting glaucomatous eyes (defined by abnormal visual fields and/or abnormal appearing optic discs) ranges from 42% to 92%, and reported specificity for detecting healthy eyes ranges from 84% to 96%.9 10 12 33 In the current study, sensitivity was 81% and 64% at 75% and 90% specificity, respectively. Using other HRT parameter-based LDFs for discriminating between glaucomatous and healthy eyes, Mardin et al.11 and Iester et al.,14 reported sensitivities of 84% and 70%, respectively, and specificities of 95% and 92%, respectively. In the present study, at a set specificity of 90%, we found a sensitivity of 70% with the Mardin et al. LDF and a sensitivity of 69%, with the Iester et al. LDFs. Finally, Bathija et al.12 reported a sensitivity and specificity of 62% and 94%, respectively, within a demographically similar sample with similar inclusion criteria and severity of glaucoma as in the current study.
Using non-standard HRT parameters to discriminate between glaucomatous and healthy eyes, Caprioli et al.34 reported a sensitivity of 83% and specificity of 85% (parapapillary slope derived from radial height measures around the disc), Iester et al.14 reported a sensitivity of 65% and specificity of 100% (measurement of retinal height differential), and Wollstein et al.35 reported a sensitivity of 84% and specificity of 96% (rim area adjusted for disc area). The maximum sensitivity and specificity reported in the present study were 91% and 90%, respectively (for optimized SVM Gaussian). Comparisons across studies are difficult, however, because of differences in population demographics, definition and severity of glaucoma, and differences in sensitivity and specificity at chosen cutoff values. We include this information to provide a context for our results.
In the current study, we identified HRT parameters that most affected the area under the ROC curve for discriminating between glaucomatous and healthy eyes with forward selection and backward elimination used in the SVM Gaussian and current LDF techniques. For both SVM and LDF techniques, global cup shape was an important parameter, identified by both forward selection and backward elimination. This finding is interesting, because other research has shown that cup shape is among the best individual parameters for discriminating between glaucomatous and healthy eyes.6 8 9 Other parameters identified by our techniques are novel. For instance, with the SVM Gaussian technique, peak height contour in the temporal inferior region and disc area in the nasal region were identified. It is possible that techniques without clinical bias may identify unexpected parameters that are important for discriminating glaucomatous from healthy eyes. The discordance in the most effective parameters with different classifiers may be explained by differences in the classifier reasoning process.
Two possible limitations of the current study were the lack of independent samples on which to develop and test current LDF and neural network techniques and the significant difference in age between healthy subjects and patients. Although we used cross-validation to train the neural network classifiers and the current LDF, part of the reason for the improved performance of these techniques compared with previously proposed LDFs might be that our techniques were trained and tested on groups with similar demographics and severity of glaucoma. We plan to test these techniques on outside populations. Because of the age difference between healthy subjects and patients, we did not include age as input when training the neural networks or developing the current LDF. The inclusion of age might allow the neural networks to classify eyes as glaucomatous or healthy based on age alone. This classification is clearly not practical from a diagnostic standpoint. When age was included in the training set, areas under the ROC curves for all neural network techniques and current LDF increased by 0.01 or less. These increases were not statistically significant. Sensitivities at the chosen specificity cutoffs increased by less than 5%. We also trained and tested the neural network techniques on a subset of our data in which age in both the healthy subjects and patients was constrained to between 40 years and 81 years. This subset was composed of 133 healthy subjects (mean age, 65.10 years) and 90 patients with glaucoma (mean age, 66.78 years). Age was not significantly different between groups (t-test, P > 0.10). Areas under the ROC curve for all neural network techniques and current LDF changed by 0.01 or less and no changes were statistically significant. Sensitivities at the chosen specificity cutoffs changed by less than 5%.
Although neural networks successfully discriminated between healthy and glaucomatous eyes in this study, these techniques incur one general criticism. Due to the complexity of the classifiers, they do not allow the interaction of important variables to be identified and measured. Other classification techniques, such as Bayesian networks, allow better assessment of the relative contribution of features.
In summary, neural network techniques were more successful at discriminating between glaucomatous and healthy eyes than previously proposed LDFs. This improvement suggests that neural network techniques show at least as much potential for use in diagnosis of glaucoma as linear discriminant techniques. In addition, support vector machines demonstrated better generalization performance, and therefore better classification performance than MLPs (see also Refs. 19 ,34 ,35 ). This result, coupled with the fact that SVMs are faster to train than MLPs, suggests that SVMs show superior potential for use in diagnosis of glaucoma when compared with MLPs.
| Footnotes |
|---|
Submitted for publication February 4, 2002; revised June 5, 2002; accepted June 17, 2002.
Commercial relationships policy: N.
The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked "advertisement" in accordance with 18 U.S.C.
1734 solely to indicate this fact.
Corresponding author: Christopher Bowd, Hamilton Glaucoma Center, Department of Ophthalmology, University of California at San Diego, La Jolla, CA 92093-0946; cbowd{at}eyecenter.ucsd.edu.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
K A Townsend, G Wollstein, D Danks, K R Sung, H Ishikawa, L Kagemann, M L Gabriele, and J S Schuman Heidelberg Retina Tomograph 3 machine learning classifiers for glaucoma detection Br. J. Ophthalmol., June 1, 2008; 92(6): 814 - 818. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Bowd, J. Hao, I. M. Tavares, F. A. Medeiros, L. M. Zangwill, T.-W. Lee, P. A. Sample, R. N. Weinreb, and M. H. Goldbaum Bayesian Machine Learning Classifiers for Combining Structural and Functional Measurements to Classify Healthy and Glaucomatous Eyes Invest. Ophthalmol. Vis. Sci., March 1, 2008; 49(3): 945 - 953. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Boden, K. Chan, P. A. Sample, J. Hao, T.-W. Lee, L. M. Zangwill, R. N. Weinreb, and M. H. Goldbaum Assessing Visual Field Clustering Schemes Using Machine Learning Classifiers in Standard Perimetry Invest. Ophthalmol. Vis. Sci., December 1, 2007; 48(12): 5582 - 5590. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Naithani, R. Sihota, P. Sony, T. Dada, V. Gupta, D. Kondal, and R. M. Pandey Evaluation of Optical Coherence Tomography and Heidelberg Retinal Tomography Parameters in Detecting Early and Moderate Glaucoma Invest. Ophthalmol. Vis. Sci., July 1, 2007; 48(7): 3138 - 3145. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Coops, D. B. Henson, A. J. Kwartz, and P. H. Artes Automated Analysis of Heidelberg Retina Tomograph Optic Disc Images by Glaucoma Probability Score Invest. Ophthalmol. Vis. Sci., December 1, 2006; 47(12): 5348 - 5355. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Shunmugam and A. Azuara-Blanco The quality of reporting of diagnostic accuracy studies in glaucoma using the heidelberg retina tomograph. Invest. Ophthalmol. Vis. Sci., June 1, 2006; 47(6): 2317 - 2323. [Abstract] [Full Text] [PDF] |
||||
![]() |
M.-L. Huang and H.-Y. Chen Development and Comparison of Automated Classifiers for Glaucoma Diagnosis Using Stratus Optical Coherence Tomography Invest. Ophthalmol. Vis. Sci., November 1, 2005; 46(11): 4121 - 4129. [Abstract] [Full Text] [PDF] |
||||
![]() |
Z. Burgansky-Eliash, G. Wollstein, T. Chu, J. D. Ramsey, C. Glymour, R. J. Noecker, H. Ishikawa, and J. S. Schuman Optical Coherence Tomography Machine Learning Classifiers for Glaucoma Detection: A Preliminary Study Invest. Ophthalmol. Vis. Sci., November 1, 2005; 46(11): 4147 - 4152. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. H. Goldbaum, P. A. Sample, Z. Zhang, K. Chan, J. Hao, T.-W. Lee, C. Boden, C. Bowd, R. Bourne, L. Zangwill, et al. Using Unsupervised Learning with Independent Component Analysis to Identify Patterns of Glaucomatous Visual Field Defects Invest. Ophthalmol. Vis. Sci., October 1, 2005; 46(10): 3676 - 3683. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Bowd, F. A. Medeiros, Z. Zhang, L. M. Zangwill, J. Hao, T.-W. Lee, T. J. Sejnowski, R. N. Weinreb, and M. H. Goldbaum Relevance Vector Machine and Support Vector Machine Classifier Analysis of Scanning Laser Polarimetry Retinal Nerve Fiber Layer Measurements Invest. Ophthalmol. Vis. Sci., April 1, 2005; 46(4): 1322 - 1329. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. M. Zangwill, K. Chan, C. Bowd, J. Hao, T.-W. Lee, R. N. Weinreb, T. J. Sejnowski, and M. H. Goldbaum Heidelberg Retina Tomograph Measurements of the Optic Disc and Parapapillary Retina for Detecting Glaucoma Analyzed by Machine Learning Classifiers Invest. Ophthalmol. Vis. Sci., September 1, 2004; 45(9): 3144 - 3151. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Bowd, L. M. Zangwill, F. A. Medeiros, J. Hao, K. Chan, T.-W. Lee, T. J. Sejnowski, M. H. Goldbaum, P. A. Sample, J. G. Crowston, et al. Confocal Scanning Laser Ophthalmoscopy Classifiers and Stereophotograph Evaluation for Prediction of Visual Field Abnormalities in Glaucoma-Suspect Eyes Invest. Ophthalmol. Vis. Sci., July 1, 2004; 45(7): 2255 - 2262. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. A. Medeiros, L. M. Zangwill, C. Bowd, and R. N. Weinreb Comparison of the GDx VCC Scanning Laser Polarimeter, HRT II Confocal Scanning Laser Ophthalmoscope, and Stratus OCT Optical Coherence Tomograph for the Detection of Glaucoma Arch Ophthalmol, June 1, 2004; 122(6): 827 - 837. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |