Validity and Acceptance of Color Vision Testing on Smartphones

Background: Ishihara color plates (ICP) are the most commonly used color vision test (CVT) worldwide. With the advent of new technologies, attempts have been made to streamline the process of CVT. As hardware and software evolve, smartphone-based testing modalities may aid ophthalmologists in performing more efficient ophthalmic examinations. We assess the validity of smartphone color vision testing (CVT) by comparing results using the Eye Handbook (EHB) CVT application with standard Ishihara color plates (ICP). Methods: Prospective case-control study of subjects 18 years and older with visual acuity of 20/100 or better at 14 inches. The study group included patients with any ocular pathology. The color vision deficient (CVD) group was patients who failed more than 2 plates. The control group had no known ocular pathology. CVT was performed with both ICP and EHB under standardized background illuminance. Eleven plates were tested with each modality. Validity of EHB CVT and acceptance of EHB CVT were analyzed. Statistical analyses were performed using Bland-Altman plot with limits of agreement (LOA) at the 95th percentile of differences in score, independent samples t tests with 95% confidence interval (CI), and Pearson χ tests. Results: The Bland-Altman plot showed agreement between correct number of plates in EHB and ICP for the study subjects (bias, -0.25; LOA, -1.92 to 1.42). Agreement was also observed between the correct number of plates in EHB and ICP for the controls (bias, -0.01; LOA, -0.61 to 0.59) and CVD (bias, -0.50; LOA, -4.64 to 3.64) subjects. The sensitivity of EHB was 0.92 (95% CI 0.76-1.07) and the specificity of EHB was 1.00 (95% CI 1.00-1.00). Fifty-nine percent preferred EHB, 12% preferred ICP, and 29% had no preference. Conclusions: In healthy controls and patients with ocular pathology, there was an agreement of CVT results comparing EHB with ICP. Overall, the majority preferred EHB to ICP. These findings demonstrate that further testing is required to understand and improve the validity of smartphone CVT in subjects with ocular pathology.

Original Contribution Validity and Acceptance of Color Vision Testing on Smartphones Omar K. Ozgur, MD, Trisha S. Emborgo, BA, Mark B. Vieyra, BA, Rebecca F. Huselid, PhD, Rudrani Banik, MD Background: Ishihara color plates (ICP) are the most commonly used color vision test (CVT) worldwide. With the advent of new technologies, attempts have been made to streamline the process of CVT. As hardware and software evolve, smartphone-based testing modalities may aid ophthalmologists in performing more efficient ophthalmic examinations. We assess the validity of smartphone color vision testing (CVT) by comparing results using the Eye Handbook (EHB) CVT application with standard Ishihara color plates (ICP). Methods: Prospective case-control study of subjects 18 years and older with visual acuity of 20/100 or better at 14 inches. The study group included patients with any ocular pathology. The color vision deficient (CVD) group was patients who failed more than 2 plates. The control group had no known ocular pathology. CVT was performed with both ICP and EHB under standardized background illuminance. Eleven plates were tested with each modality. Validity of EHB CVT and acceptance of EHB CVT were analyzed. Statistical analyses were performed using Bland-Altman plot with limits of agreement (LOA) at the 95th percentile of differences in score, independent samples t tests with 95% confidence interval (CI), and Pearson x 2 tests. Results: The Bland-Altman plot showed agreement between correct number of plates in EHB and ICP for the study subjects (bias, 20.25; LOA, 21.92 to 1.42). Agreement was also observed between the correct number of plates in EHB and ICP for the controls (bias, 20.01; LOA, 20.61 to 0.59) and CVD (bias, 20.50; LOA, 24.64 to 3.64) subjects. The sensitivity of EHB was 0.92 (95% CI Ophthalmic Plastic and Reconstructive Surgery (OKO), Advanced Eye Medical Group, Mission Viejo, California; Lake Erie College of Osteopathic Medicine at Seton Hill (TSE), Greensburg, Pennsylvania; Sackler Faculty of Medicine (MBV), Tel Aviv University, Tel Aviv, Israel; Hunter College (RFH), the City University of New York, New York, New York; and New York Eye and Ear Infirmary of Mount Sinai (RB), New York, New York. The authors report no conflicts of interest. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the full text and PDF versions of this article on the journal's Web site (www. jneuro-ophthalmology.com). Address correspondence to Rudrani Banik, MD, New York Eye and Ear Infirmary of Mount Sinai, 950 Fifth Avenue New York, NY 10075; E-mail: [e-mail] Ozgur et al: J Neuro-Ophthalmol 2018; 38: 13-16 0.76-1.07) and the specificity of EHB was 1.00 (95% CI 1.00-1.00). Fifty-nine percent preferred EHB, 12% preferred ICP, and 29% had no preference. Conclusions: In healthy controls and patients with ocular pathology, there was an agreement of CVT results comparing EHB with ICP. Overall, the majority preferred EHB to ICP. These findings demonstrate that further testing is required to understand and improve the validity of smartphone CVT in subjects with ocular pathology. Journal of Neuro-Ophthalmology 2018;38:13-16 doi: 10.1097/WNO.0000000000000637 © 2018 by North American Neuro-Ophthalmology Society C olor vision testing (CVT) is an important part of the standard ophthalmic examination. It is necessary for evaluating and monitoring progression of ophthalmic disease. Ishihara color plates (ICP) were first published in 1906 and are the most commonly used color vision test worldwide (1). Several properties may affect the validity of CVT including the quality of the printed or displayed image, the viewing distance of the image, background illumination, and an individual's visual acuity. With the advent of new technologies, attempts have been made to streamline the process of CVT. iPhone- and Android-based smartphones currently are in use by most physicians in the United States (2). In addition to being communication devices, smartphones provide applications that may help with the care and management of patients. As hardware and software evolve, smartphone-based testing modalities may aid ophthalmologists in performing more efficient ophthalmic examinations (2). Smartphone applications such as the Eye Handbook (EHB) (http://www.eyehandbook. com/) (3) provides CVT and other ophthalmic tools for iPhone- and Android-based smartphones. However, there is minimal evidence to support the validity of these nonstandardized tools. Campbell et al (4) found a strong correlation between testing color vision with ICP and a CVT application on an iPad. Shah et al (5) presented data on 100 normal subjects tested with 13 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Original Contribution iPhone EHB and with standard ICP. They found no significant difference between the tests, although 89 of those tested had no color vision deficits on either test. Our study sought to compare the results of CVT on smartphones using EHB and ICP in patients with a range of ocular pathology and healthy controls. METHODS An institutional review board-approved prospective study was performed on individuals between the ages of 18-85 years with Snellen near visual acuity of 20/100 or better at 14 inches, the minimum resolution required for accurate testing with ICP (6). Subjects were recruited from the comprehensive and subspecialty eye clinics at a tertiary eye hospital. Patients 50 years and younger were considered in the younger age range, whereas those older than 50 years were considered in the older age range. All individuals underwent a complete ophthalmic examination including dilated funduscopic examination. The study group included patients with ocular pathology and was grouped by disorders of the retina, optic nerve (nonglaucomatous optic neuropathy and glaucoma were analyzed as separate categories), cornea and/or ocular surface disease lens, orbit, or known congenital color deficiency. If a subject had more than 1 ocular pathology, that person was assigned to the category believed to most compromise visual function. The control group included those with no known pathology by history and examination. Patients with a history of previous ocular surgery were excluded from the study. CVT was performed with both EHB and ICP under a standardized background illuminance, as measured with Sper Scientific Light Meter FC-840021 (Sper Scientific Direct, Scottsdale, AZ) before turning on the phone and performing any study testing. Color vision was assessed using the Ishihara color vision test (The Series of Plates Designed as a Test for ColourDeficiency: Concise Edition Book, Kanehara & Co, Lt, Tokyo, Japan). In the 14-plate edition, a total of 11 plates were used: plates 1-8, 10, 12, and 13. An Apple iPhone 4 with retina display and maximal brightness of 500 cd/m2 (7) and a Samsung Infuse 4G with maximal brightness of 300 cd/m2 (personal communication with Samsung Technical Support, October 2014) were used to conduct the EHB color vision tests using full brightness screen illumination. The study was randomized by order of testing and phone model. Each eye was tested independently. The eye with pathology was considered the study eye. If both eyes were affected, the eye with the worse vision was considered the study eye. Each subject was presented with eleven color plates; each modality scored by number of correct plates of eleven test plates. Subjects, who failed more than 2 plates, were considered color vision deficient (CVD) (8). 14 Statistical analyses were performed using IBM SPSS version 22 (SPSS, Inc, Armonk, NY) software. The difference between the 2 methods was plotted against the mean of the 2 scores using the Bland-Altman plot with limits of agreement (LOA) at the 95th percentile of differences in score (9). The difference between ages of the control and study groups was calculated and tested using an independent samples t test with 95% confidence interval (CI). Pearson x2 tests were used to analyze the subject's method preference. RESULTS Of 206 subjects enrolled, 10 did not complete all required study testing, and 7 withdrew from the study after enrollment. Of the 189 remaining patients, 113 with ocular pathology were categorized in the study group and 76 with no ocular pathology were assigned to the control group. Mean background illuminance for CVT was 727.7 cd/ m2. Of 113 study subjects, 41 (36%) had a primary diagnosis of nonglaucomatous optic neuropathy, 22 (20%) had glaucoma or suspicion of glaucoma, 26 (23%) had cornea and/or ocular surface pathology, 11 (10%) had retinal pathology, 10 (9%) had cataracts, and 3 (3%) had thyroid eye disease. Thirty-five percent of study patients (39/113) had more than 1 diagnosis. Study subjects had a mean age of 50.8 ± 17.5 years (range: 18-82 years) and 64% were female. Control individuals had a mean age of 38.3 ± 14.8 years (range: 18-74 years) and 63% were female. Using an independent samples t test, we found a significant difference in age for the control and study groups; t (187) = 25.123, P = 0.001, 95% CI 217.31 to 27.68, although not for sex (P = 0.343). The Bland-Altman plot showed agreement between correct number of plates in EHB and ICP for the study subjects (bias, 20.24; LOA 21.92 to 1.42) (See Supplemental Digital Content, Figure E1, http://links.lww.com/WNO/A296). Agreement also was observed between the correct number of plates in EHB and ICP for the controls (bias, 20.01; LOA, 20.61 to 0.59) and CVD (bias, 20.50; LOA, 24.64 to 3.65) patients (See Supplemental Digital Content, Figures E2 and E3, http://links.lww.com/WNO/A297, http://links.lww.com/WNO/A298). The Bland-Altman LOA data are found in Supplemental Digital Content 1, (see Table E1, http://links.lww.com/WNO/A295). Of the 113 study subjects, 12 (11%) were characterized as CVD, missing more than 2 of eleven plates on either the EHB and/or ICP. Study group individuals found to be CVD had the following diagnoses: optic neuropathy (10/ 12; 83%) and cornea/other ocular surfaces (2/12; 17%). Eleven of 12 CVD subjects were found to be CVD on both EHB and ICP testing. One CVD patient with a primary diagnosis of cornea/other ocular surfaces, combined with a diagnosis of cataracts, was found to be color deficient with 11 correct plates on the EHB, but 8 correct plates on the Ozgur et al: J Neuro-Ophthalmol 2018; 38: 13-16 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Original Contribution ICP. Two CVD subjects had an underlying diagnosis of congenital color deficiency. Both of these recognized 3 correct plates on the EHB; on ICP testing, 1 identified 3 correct plates and the other only 1 correct plate. The number of plate errors and discordances for study subjects without CVD and control subjects is shown in Supplemental Digital Content 2, (see Table E2, http://links.lww.com/WNO/A299). Using ICP as the "gold standard" for CVT, the sensitivity of EHB was 0.92 (95% CI 0.76-1.07) and the specificity of EHB was 1.00 (95% CI 1.00-1.00). After CVT with both ICP and EHB, patients were asked which modality they preferred, as well as for additional comments. The overall testing preference was 59% for the EHB were as follows: "no comment" (43%), "clearer" (24%), "brighter" (18%), "sharper" (5%), "easier" (4%), and "too bright" (2%). Of this group, 3% preferred the large plate size of the ICP. Of study subjects in the older group, 34 (57%) preferred the EHB, 7 (12%) preferred the ICP, and 19 (32%) had no preference. In the younger group, 31 (59%) preferred the EHB, 7 (13%) preferred the ICP, and 15 (28%) had no preference. The controls comments regarding EHB were as follows: "no comment" (41%), "clearer" (24%), "easier" (11%), "sharper" (8%), "brighter" (7%), and "too bright" (3%). Of the controls, 1% preferred the larger plate size of the ICP. Of control subjects, the older 12 (63%) preferred the EHB, 1 (5%) preferred the ICP, and 6 (32%) had no preference. In the younger group, 35 (61%) preferred the EHB, 7 (12%) preferred the ICP, and 15 (26%) had no preference. The mean age for individuals who preferred the ICP and EHB was 43.4 ± 19.5 years and 45.7 ± 19.5 years and 45.7 ± 16.5 years, respectively. Using Pearson x2 test, there was no significant difference between the control and study groups vs EHB and ICP (P = 0.63). The mean age for patients with no preference was 46.9 ± 18.8 years. DISCUSSION Previous studies have reported that approximately 60% of normal subjects identify all color plates correctly on ICP testing (8). In our study, 70 (92%) and 71 (93%) of control patients were able to identify all plates correctly on ICP and EHB, respectively. In our study group with ocular pathology, 78 (69%) identified 11 color vision plates correctly with ICP and 96 (85%) identified all 11 color vision plates correctly with EHB. These results suggest that CVT with the EHB may be easier to perform, thus the potential of missing color vision deficiencies. We showed that there were no discrepancies in CVT between ICP and EHB in control subjects. Our results are in agreement with the study by Shah et al (5) in which there was no difference in ICP and EHB CVT testing in control patients. We found that individuals with ocular pathologies also had an agreement between ICP and EHB results on Ozgur et al: J Neuro-Ophthalmol 2018; 38: 13-16 CVT. The Bland-Altman plot visually demonstrated that there were no positive nor negative trends for the differences between the ICP and EHB CVT testing. For the CVD group, our study had only 1 discordance in which the patient read all plates correctly on the EHB and missed 3 plates on the ICP; the CVD subject was not considered CVD with the EHB but considered CVD on the ICP. It is important to note that the error made by the CVD patient on EHB was either a misreading or an atypical error. There are numerous factors which may potentially lead to discordant results between ICP and EHB CVT. Screen size, resolution, brightness, and color saturation of each smartphone model likely impacts the accuracy of EHB CVT. Smartphones also have differences in lighting uniformity on screen, which may lead to variability in colors seen on EHB. Lighting also may play a role; ICP testing is performed with front-light, whereas EHB on smartphones uses back light. Similarly, de Fez et al (10) suggested that mobile device applications should specify colorimetric characterizations, visualization conditions, illumination conditions, screen brightness, and/or distance for specific devices to ensure more reliable measurements. Limitations of our study include the fact that the study and control groups were not age-matched; notably, the mean age of the study group was older than that of the control group. There was also a female preponderance in both the study and control groups. The pathologies studied were diverse, with small sample sizes for each category, including only 12 CVD individuals, which made it impossible to draw meaningful conclusions regarding expected EHB CVT results based on specific diagnoses. Although CVT on smartphones may be a convenient bedside tool, there are disadvantages to such testing. One is the rapidly changing smartphone industry. New smartphone models are developed every 1 to 2 years and upgrades in screen displays may require frequent validity testing of applications such as EHB. In addition, applications which allow for digital CVT such as EHB are freely accessible to download. Individuals eager to score well on CVT may download the application and memorize the correct responses. Random order of administration of the plates or the institution of a passcode for eye care providers to allow a download of the application may help address these issues. Despite the limited sensitivity of EHB in CVD individuals, most subjects preferred the EHB to the ICP because of the EHB's perceived clearer, easier, brighter, and sharper image. Patients who preferred the ICP cited the harsh lighting on the EHB and larger plate sizes on the ICP as reasons. Further studies evaluating CVD individuals are needed. CVT on smartphones using EHB may be useful as a screening test; however, the authors still 15 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Original Contribution recommended ICP testing in patients with suspected ocular pathology. STATEMENT OF AUTHORSHIP Category 1: a. Conception and design: O. K. Ozgur and R. Banik; b. Acquisition of data: O. K. Ozgur, T. S. Emborgo, M. B. Vieyra, and R. Banik; c. Analysis and interpretation of data: O. K. Ozgur, T. S. Emborgo, M. B. Vieyra, R. F. Huselid, and R. Banik. Category 2: a. Drafting the manuscript: O. K. Ozgur, T. S. Emborgo, R. F. Huselid, and R. Banik; b. Revising it for intellectual content: O. K. Ozgur, T. S Emborgo, R. F. Huselid, and R. Banik. Category 3: a. Final approval of the completed manuscript: O. K. Ozgur, T. S. Emborgo, M. B. Vieyra, R. F. Huselid, and R. Banik. REFERENCES 1. Dain SJ. Clinical colour vision tests. Clin Exp Optom. 2004;87:276-293. 2. Chhablani J, Kaja S, Shah VA. Smartphones in ophthalmology. Indian J Ophthalmol. 2012;60:127-131. 16 3. Eye Handbook. Available at: http://www.eyehandbook.com. Accessed February 16, 2016. 4. Campbell TG, Lehn A, Blum S, Airey C, Brown H. iPad colour vision apps for dyschromatopsia screening. J Clin Neurosci. 2016;20:92-94. 5. Shah AA, Pasadhika S, Kim J, Wang M. Pseudoisochromatic color vision testing on an iPhone. Invest Ophthalmol Vis Sci. 2012;53:6399. 6. McCulley TJ, Golnik KC, Lam BL, Feuer WJ. The effect of decreased visual acuity on clinical color vision testing. Am J Ophthalmol. 2006;141:194-196. 7. Apple, iPhone 4-Technical Specifications. Available at: https://support.apple.com/kb/sp587?locale=en_US. Accessed February 16, 2016. 8. Birch J. Efficiency of the Ishihara test for identifying red-green colour deficiency. Ophthalmic Phsyiol Opt. 1997;17:403- 408. 9. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;327:307-310. ero D. 10. de Fez D, Luque MJ, Garcia-Domene MC, Camps V, Pin Colorimetric characterization of mobile devices for vision applications. Optom Vis Sci. 2016;93:85-93. Ozgur et al: J Neuro-Ophthalmol 2018; 38: 13-16 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited.