One Eye or Two: Statistical Considerations in Ophthalmology With a Focus on Interventional Clinical Trials

Research in ophthalmology is unusual in comparison with other many medical specialties by having 2 targets where frequent and accessible data can be collected and analyzed. The major factor is that 2 eyes are not independent as they belong to the same individual. As many statistical methods assume independence of observations, historically, many researchers have arbitrarily chosen one eye for analysis. Another approach to maintain independence of observations is to generate subject level index variables using information from both eyes. This was the approach taken by Anagnostou et al in this issue of Journal of Neuro-ophthalmology. Specifically, for measurements taken in both eyes of subjects with 3 possible grades (i.e., 0, 1, 2 in the right eye and 0, 1, 2 in the left eye), each combination was considered to yield 9 potential subject level outcomes for 2 analyses.

Editorial One Eye or Two: Statistical Considerations in Ophthalmology With a Focus on Interventional Clinical Trials Susan P. Mollan, FRCOphth, Victoria Homer, MsC, Simon Gates, PhD, Kristian Brock, PhD, Alex J. Sinclair, PhD R esearch in ophthalmology is unusual in comparison with other many medical specialties by having 2 targets where frequent and accessible data can be collected and analyzed. The major factor is that 2 eyes are not independent as they belong to the same individual. As many statistical methods assume independence of observations, historically, many researchers have arbitrarily chosen one eye for analysis. Another approach to maintain independence of observations is to generate subject level index variables using information from both eyes. This was the approach taken by Anagnostou et al (1) in this issue of Journal of Neuro-ophthalmology. Specifically, for measurements taken in both eyes of subjects with 3 possible grades (i.e., 0, 1, 2 in the right eye and 0, 1, 2 in the left eye), each combination was considered to yield 9 potential subject level outcomes for 2 analyses. Statistical methods exist which allow for the analysis of both eyes; however, these require acknowledging and accounting for the intraparticipant correlation between eyes. Although the potential for leveraging data is promising, ensuring appropriate application of statistical methods is critical to ensure sound analysis and appropriate conclusions. The purpose of this article is to provide a statistical overview for researchers and readers of clinical studies to allow for critical appraisal of the statistical methods chosen, for using both or either eye in analysis with an emphasis on interventional clinical trials. Benefits of Using One Eye In a review considering the one-eyed vs two- eyed problem, the ophthalmic literature between 2009 and 2012 was collated, and the authors found that 64% of studies only obtained data from one eye, with the most common methods of eye selection being the right (35%) followed by the worst (23%) (2). Therefore, by only analyzing one eye and keeping the methodology used consistent, there is an opportunity for direct comparison with historical studies. The requisite analysis of one eye may be more straightforward because independence of observations (one observation per subject) is maintained. Thus, more traditional statistical methods are being used (e.g., t tests, analysis of variance, and linear regression). There are certain study settings where it is appropriate to analyze only one eye.1 For example, in the instance where only one eye is affected by disease at the time of the study (such as occurs in acute optic Birmingham Neuro-Ophthalmology (SPM), Queen Elizabeth Hospital, Birmingham, United Kingdom; Cancer Research UK Clinical Trials Unit (CRCTU) (VH, SG, KB), University of Birmingham, Birmingham, United Kingdom; Metabolic Neurology (AJS), Institute of Metabolism and Systems Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom; Centre for Endocrinology (AJS), Diabetes and Metabolism, Birmingham Health Partners, Birmingham, United Kingdom; and Department of Neurology (AJS), Queen Elizabeth Hospital, University Hospitals Birmingham NHS Foundation Trust, Birmingham, United Kingdom. A. J. Sinclair is funded by a Sir Jules Thorn Award for Biomedical Science. The view expressed is those of the authors and not necessarily those of the UK National Health Service. S. P. Mollan reports other from Invex Therapeutics, other from Heidelberg engineering, other from Chugai-Roche Ltd, other from Janssen, other from Santhera, other from Allergan, other from Santen, other from Roche, and other from Neurodiem, outside the submitted work. K. Brock reports other from Invex Therapeutics; other from AstraZeneca, other from GlaxoSmithKline, other from Eli Lilly, and other from Merck, outside the submitted work. Professor Sinclair reports other from Novartis; other from Allergan, and personal fees from Invex therapeutics as well as share option and shareholdings. All other authors declare no competing interests. S. P. Mollan and V. Homer joint first. Address correspondence to Alex J. Sinclair, PhD, Institute of Metabolism and Systems Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, B15 2TT, United Kingdom; E-mail: [e-mail]. Mollan et al: J Neuro-Ophthalmol 2021; 41: 421-423 421 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Editorial neuritis or nonarteritis anterior ischemic optic neuropathy) or if the intervention only affects one eye (e.g., certain topical applications); in these cases, analyses using both eyes would not be appropriate. Here, it would not be appropriate to assume that any intervention will affect each eye analogously, and analyses should instead focus on only the affected eye or treated eye. In studies of diseases where both eyes are affected and analyses only focuses on one eye, there are multiple choices regarding the selection of the eye. These include arbitrarily selecting the left or right eye, randomly selecting the left or right eye, using the worst (most disease affected) eye, or using the best (least disease affected eye). Each of these selections have their own advantages, for example, arbitrarily choosing an eye a priori or having an eye selected by a randomized process mitigates against selection bias, whereas focusing analyses around the worst eye allows for the evaluation of the intervention in the most serious scenario. However, all the selections have a disadvantage because there is potential for selection bias as to which eye shou11ld be chosen and which method of selecting the eye should be used. Drawbacks of Using One Eye Analysis of one eye may make implicit assumptions regarding generalizability. These may be that both eyes will act similarly (in the case where one eye is randomly chosen) or that if an intervention works well in the most serious scenario then conclusions will be generalizable to the less severe scenario (in the case where the worst eye is used). As data are not collected or analyzed in such a way to address this, such assumptions remain unverifiable. The key limitation to only using one eye in analysis of clinical trials is that such methods are not information efficient. Data are usually available from both eyes (even if not explicitly collected as part of the trial), yet half of it is somewhat arbitrarily discarded before analysis. This may mean that a larger sample size is needed to ascertain the same precision as in the case where both eyes are analyzed. Benefit of Using Both Eyes Research data, that is, generously provided by patients and carefully collected should have the benefit of not being cut in half at analysis where possible. By analyzing both eyes, more data are contributed by the same number of participants, making better scientific use of a precious resource. The increase in data by using both eyes reduces uncertainty by providing more precise effect estimates and reducing uncertainty (such as narrowing of the confidence interval). Using all the data contributed per participant in any study may be more ethical for several factors. First, as each participant is contributing more data, it is possible to 422 ascertain the same (or greater) statistical power with fewer participants. This could mean that fewer participants would be needed over all to deliver the results of any given trial and therefore allow for faster recruitment and overall shorter trial length. This ultimately would result in quicker ascertainment regarding the efficacy of an intervention by analyzing both eyes. This reduction in the sample size without sacrificing statistical power would facilitate trials in rare diseases, where single eye sample sizes may be infeasible because of the low prevalence in the general population. Furthermore, the arbitrary choice of which eye to analyze is negated, reducing any potential selection bias. Two examples of specific study designs that use data from both eyes in subjects are discussed in detail below. Studies Where Both Eyes Are Affected by Disease of Interest and a Subject Level Intervention is Administered In this instance, one of the following situations will hold: Both eyes will be affected equally (such as often occurs in toxic optic neuropathy); the disease will affect each eye differentially, and this difference will be deterministic (such as occurs in glaucoma or Leber hereditary optic neuropathy); or the disease will affect each eye differentially, and this difference will be random (such as occurs in multiple sclerosis and acute optic neuritis). Provided the mechanism of action of the intervention can be assumed to act equally over both eyes, analysis using both eyes will lead to an increase in the effective sample size (equivalently, an increase in statistical power). The extent to which effective sample size is increased depends on the intraocular correlation. If outcomes are perfectly correlated, the effective sample size does not increase at all, that is, the two-eye analysis will be no more efficient than a single eye analysis (3). The theoretical maximal increase in effective sample size is 100% (i.e., effective sample size is doubled), occurring when outcomes in the eyes are completely independent. However, in reality, outcomes from different eyes within a patient will always be correlated to some extent, and thus, the expected increase in effective sample size is somewhere between these 2 extremes (4). To perform 2 eye analyses, the data for each eye needs to be paired and come from a common source (e.g., each participant contributing both a left and right eye). This creates a nested data structure within participants thus requiring more complex methods of analysis. To properly account for this nesting structure, hierarchical extensions to well-known regression modeling methods (also referred to as multilevel, random effects, or mixed-effect models) that account for the nesting of data through random effects should be used. Many common statistical tests (e.g., linear regression, t test, etc.) can be written as linear models, and hierarchical extensions to linear models can be used (5). In instances where linear models are not appropriate, Mollan et al: J Neuro-Ophthalmol 2021; 41: 421-423 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Editorial hierarchical extensions exist to other types of modeling such as repeated measures analysis, logistic regression models, and time-to-event models, all of which can be performed using either frequentist or Bayesian methodology (6–8). Owing to the increased computational intensity of the planned analysis when using hierarchical models for 2 eyes, ascertainment of characteristics of trials designs (including sample size calculations and derivation of type I and II errors) can be more complex. Existing sample size formulas rarely have extensions to allow for hierarchical models, and instead, simulations are often warranted. These simulations can be multifaceted and time consuming. Studies Where Both Eyes Are Affected by the Disease of Interest and an Eye Level Intervention Is Administered. Dependent on the mechanism of action of the intervention, it may here be appropriate to use intraparticipant controls. This is feasible where it can be assumed that the intervention will only affect one eye (such for some topical treatments) and not for systemic drug interventions. By using each participant’s untreated eye as their own control, the intraparticipant variation will be reduced, and so, it may be possible to again ascertain more precise effect estimates. In this situation, analogous to the analyses of both eyes, more involved analysis methods, such as hierarchical models would need to be used. The assumption that a local treatment will only affect one eye can, however, be compromised, as was demonstrated in “REVERSE”. REVERSE was a randomized, double-masked, sham-controlled, multicenter, phase 3 clinical trial that evaluated the efficacy of a single unilateral intravitreal injection of rAAV2/2-ND4 in subjects with visual loss from Leber hereditary optic neuropathy that reported an unexpected bilateral improvement in visual function after unilateral injection suggesting bilateral effect (9). This effect has been noted in other ocular diseases such as uniocular intravitreal injections in bilateral diabetic macular edema (10) and topical treatments to lower intraocular pressure (11). Other Study Designs Two-eye data can be analyzed for other study designs as well. For example, generalized estimating equation or mixed effect models and bootstrapping approaches can generate point estimates and confidence intervals of sensitivity, specificity, and predicative value studies of diagnostic accuracy using 2 eye data that may be congruent between Mollan et al: J Neuro-Ophthalmol 2021; 41: 421-423 eyes in some subjects and incongruent between eyes in different subjects (12). These methods have broader application to other observational studies. CONCLUSION Statistical analysis plans in ophthalmic disease need to carefully consider the rational for analyzing one eye or 2. Authors should clearly describe the methods used and the rationale for their statistical choices. Where the disease affects both eyes, generally both eyes should be analyzed as this will allow a more precise estimate of effects, consequently increased power and reduced bias from selecting a single eye; however, statistical tools must be used to avoid bias. Using a single-eye analysis because it is seemingly the default option may not be in patients’ best interest who dedicate their time and energy to participate in clinical studies. REFERENCES 1. Anagnostou E, Koutsoudaki P, Tountopoulou A, Spengos K, Vassilopoulou S. Bedside assessment of vergence in stroke patients. J Neuroophthalmol. 2021;41:424–430. 2. Armstrong RA. Statistical guidelines for the analysis of data obtained from one or both eyes. Ophthalmic Physiol Opt. 2013;33:7–14. 3. Maguire MG. Assessing intereye symmetry and its implications for study design. Invest Ophthalmol Vis Sci. 2020;61:27. 4. Karakosta A, Vassilaki M, Plainis S, Elfadl NH, Tsilimbaris M, Moschandreas J. Choice of analytic approach for eye-specific outcomes: one eye or two?. Am J Ophthalmol. 2012;153:571– 579. 5. Gelman A, Hill J. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge, New York: Cambridge University Press, 2006. 6. Diggle PJ, Liang KY, Zeger SL. Oxford Statistical Science Series, 13. Analysis of Longitudinal Data. Oxford New York, Clarendon Press: Oxford University Press, 1994. 7. Raudenbush S, Byrk A. Hierarchical Linear Models: Applications and Data AnalysisNo Title, 1992. 8. Shenyang G. Multilevel analysis of time-to-event data. In: Survival Analysis: Oxford Scholarship Online: January 2010, 2009. 9. Yu-Wai-Man P, Newman NJ, Carelli V, Moster ML, Biousse V, Sadun AA, Klopstock T, Vignal-Clermont C, Sergott RC, Rudolph G, La Morgia C, Karanjia R, Taiel M, Blouin L, Burguière P, Smits G, Chevalier C, Masonson H, Salermo Y, Katz B, Picaud S, Calkins DJ, Sahel JA. Bilateral visual improvement with unilateral gene therapy injection for Leber hereditary optic neuropathy. Sci Transl Med. 2020;12:eaaz7423. 10. Hanhart J, Tiosano L, Averbukh E, Banin E, Hemo I, Chowers I Fellow eye effect of unilateral intravitreal bevacizumab injection in eyes with diabetic macular edema. Eye (Lond) 2014; 28: 646–653. 11. Kwitko GM, Shin DH, Ahn BH, Hong YJ. Bilateral effects of long-term monocular timolol therapy. Am J Ophthalmol. 1987;104:591–594. 12. Ying GS, Maguire MG, Glynn RJ, Rosner B. Calculating sensitivity, specificity, and predictive values for correlated eye data. Invest Ophthalmol Vis Sci. 2020;61:29. 423 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited.