Journal of Neuro-Ophthalmology, March 2015, Volume 35, Issue 1

Transparency, Reproducibility, and Validation: Raising the Quality of Reporting in the Journal of Neuro-Ophthalmology

Transparency, Reproducibility, and Validation: Raising the Quality of Reporting in the Journal of Neuro-Ophthalmology Jason L. Roberts, PhD, Larissa Shamseer, MSc For readers to assess the validity of research and replicate its results or for editors to effectively judge the accuracy and true potential impact of an article, authors must carefully-and comprehensively- document their study methodology and findings. Sadly, as has been repeatedly recorded, failure to report essential methodological elements is prevalent across the biomedical literature (1-3), including neurol-ogy (4,5) and ophthalmology (6-9). At the level of an individual article, lack of description as to how a study was conducted, which criteria were used to select patients, or, for research syntheses, how the literature search was structured and executed are inconvenient. Amplified across an entire field of study, poor reporting can elevate weak, unsound, or simply specious evidence, with the undesirable potential to misdirect future research up scientific cul-de-sacs. Complete descriptions of the purpose and results in a research study should never be an afterthought. Indeed, every author should consider the issues of reporting research methods and findings from the moment the article is conceived. Fortunately, an array of tools to assist authors exists in the form of reporting guidelines. The most well known of these are CONSORT 2010 (Consolidated Standards of Reporting Trials) (10,11), STROBE (Strengthening the Reporting of Observational studies in Epidemiology) (12,13), and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) (14,15). Reporting guidelines, which are evidence- and consensus-based, and usually designed, tested, and validated by world-renowned methodology experts, pinpoint core com-ponents for reporting a particular type of study. They might quite simply be defined as an approach to encourage the provision of carefully structured minimum elements of information within a research report/article. Such reporting guidelines are commonly known in their distilled form: checklists and, in some cases, algorithms. It is these checklists that the Journal of Neuro-Ophthalmology has decided to use in an effort to raise its reporting standards. Presently, close to 600 biomedical and health research journals have endorsed CONSORT for authors who submit the results of clinical trials for publication (16). Evidence shows that trials pub-lished in journals following these guidelines are more completely reported (17). Similarly, data are emerging demonstrating a positive effect of PRISMA endorsement on systematic review articles (18). Starting in the spring of 2015, the Journal of Neuro-Ophthalmology will strongly encourage authors to include reporting criteria within their articles. The journal will request that authors adhere to the appropriate reporting guideline checklist and indicate where specific reporting elements can be found in their submissions. Checking for adherence to these reporting standards will become a feature of the Journal of Neuro-Ophthalmology peer review and editorial decision-making process. The Journal of Neuro-Ophthalmology is not striking out alone on this matter. Nine of the top 10 general medical journals (by impact factor) endorse the use of relevant reporting guidelines. In 2014, Neurology declared that all submissions would be required to use a reporting guideline checklist and demonstrate adherence as part of its peer review process (19). The American Journal of Ophthalmology, in its Instructions for Authors, insists that all randomized controlled trials must show adherence to the CONSORT statement and provide an algorithm of the flow of participants throughout the trial (20). Editorial Office (JR), Journal of Neuro-Ophthalmology, Plymouth, Massachusetts; Department of Neurology (JR), Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire; Ottawa Hospital Research Institute (LS), Ottawa, Ontario, Canada; and University of Ottawa (LS), Ottawa, Ontario, Canada. The authors report no conflicts of interest. Address correspondence to the Journal of Neuro-Ophthalmology Editorial Office, 36 Old Mill Lane, Plymouth, MA, 02360. E-mail: [e-mail] Roberts and Shamseer: J Neuro-Ophthalmol 2015; 35: 3-5 3 Editorial Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. However, as of December 2014, a quick investigation showed that only 4 of the top 10 journals (by impact factor) in the clinical neurosciences and 3 of the top 10 in oph-thalmology endorse the use of any reporting guideline. The movement to raise reporting standards is not solely a journal-led exercise. In October 2014, the National Institutes of Health, along with Nature Publishing Group and Science, promoted a core set of reporting standards in preclinical work to ensure the reproducibility of results, build-ing on an earlier call for action on the issue (21,22). There is a growing consensus that key funding agencies will eventually take the next step and insist on compliance to these validated measures of reporting for the research they fund. So, what does this mean for authors? First and foremost, reporting guidelines, although sometimes perceived as such, are not barriers to submission (23). Although it may be frustrating after composing an article to have to go back to include missing information, it must be remembered that an article is only strengthened as a result of such an exercise. Ultimately, we want authors to compose their articles using reporting guidelines as a writing aid, rather than an after-thought. Furthermore, rare is the article that completes peer review without some form of amendment and revision. Indeed, rather than necessarily reviewing for adherence to reporting standards in a gatekeeper capacity (i.e., should a manuscript be accepted or rejected?), the Journal of Neuro-Ophthalmology intends to use the reporting guidelines to work with authors to polish their articles. Authors should also be cognizant that these guidelines are designed to be portable across journals and even fields of study. Should a submission be rejected by the Journal of Neuro-Ophthalmology, having followed the reporting guide-lines will only serve the article well at the next journal to which it is submitted. With the launch of a new campaign to raise reporting standards, the Journal of Neuro-Ophthalmology will now pro-vide reporting guideline templates both from within its on-line Instructions for Authors and from within the Editorial Manager submission system. The checklists can also be downloaded from the EQUATOR Network site (www. equator-network.org). The hope is that authors will provide a completed reporting guideline checklist. The reporting guideline template will, in turn, then be provided to the peer reviewers. Where gaps in the depth of reporting are found, the journal will expect authors to address these alongside other requests for revision. In many professions, checklists are used to ensure accuracy, completeness, and transparency. Biomedical reports, especially when considering what is at stake, should similarly adopt such practices. The Journal of Neuro-Ophthalmology believes that by requesting adherence to higher reporting standards, authors will benefit by creating a better manuscript. Readers of the journal similarly gain by reading articles that have been vetted using the best tools available to ensure high reporting standards. It should be expected that the field of the neuro-ophthalmology will be better served, and it is hoped that all stakeholders embrace a commitment to a higher level of academic scholarship. REFERENCES 1. Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ. 2010;340:c723. 2. Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions of treatment in trials and reviews? BMJ. 2008;336:1472-1474. 3. Pino C, Boutron I, Ravaud P. Inadequate description of educational interventions in ongoing randomized controlled trials. Trials. 2012;13:63. 4. Shukralla AA, Tudur-Smith C, Powell GA, Williamson PR, Marson AG. Reporting of adverse events in randomised controlled trials of antiepileptic drugs using the CONSORT criteria for reporting harms. Epilepsy Res. 2011;97:20-29. 5. Carp J. The secret lives of experiments: methods reporting in the fMRI literature. Neuroimage. 2012;63:289-300. 6. O'Day R, Walton R, Blennerhassett R, Gillies MC, Barthelmes D. Reporting of harms by randomised controlled trials in ophthalmology. Br J Ophthalmol. 2014;98:1003- 1008. 7. Eyawo O, Lee CW, Rachlis B, Mills EJ. Reporting of noninferiority and equivalence randomized trials for major prostaglandins: a systematic survey of the ophthalmology literature. Trials. 2008;9:69. 8. Borg Debono V, Zhang S, Ye C, Paul J, Arya A, Hurlburt L, Murthy Y, Thabane L. The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement. BMC Anesthesiol. 2012;12:13. 9. Siddiqui MA, Azuara-Blanco A, Burr J. The quality of reporting of diagnostic accuracy studies published in ophthalmic journals. Br J Ophthalmol. 2005;89:261-265. 10. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. 11. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340: c869. 12. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M; STROBE Initiative. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4:e297. 13. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; for the STROBE Initiative. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335:806. 14. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. 15. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. 16. The CONSORT statement Website [Internet]. 2014. Available at: www.consort-statement.org. Accessed December 3, 2014. 17. Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT statement impact the completeness of reporting of randomised controlled trials published in medical journals? A cochrane review. Syst Rev. 2012;1:60. 18. Stevens A, Shamseer L, Weinstein E, Yazdi F, Turner L, Thielman J, et al. Relation of completeness of reporting of health research to journals' endorsement of reporting guidelines: systematic review. BMJ. 2014;348:g3804. 4 Roberts and Shamseer: J Neuro-Ophthalmol 2015; 35: 3-5 Editorial Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. 19. Baskin PK, Gross RA. Publishing rigor: an extension of scientific rigor: Minding the business of scientific reporting. Neurology. 2014;83:2-3. 20. American Journal of Ophthalmology Instructions for Authors [Internet]. Available at: http://www.ajo.com/content/authorinfo. Accessed December 3, 2014. 21. Principles and guidelines for reporting preclinical research-about NIH-National Institutes of Health (NIH) [Internet]. Available at: http://www.nih.gov/about/reporting-preclinical-research.htm. Accessed December 3, 2014. 22. Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, et al. A call for transparent reporting to optimize the predictive value of preclinical research. Nature. 2012;490:187-191. 23. Roberts JL, Houle TH, Loder EW, Penzien DP, Turner DP, Rothrock JF. Establishing a coherent reporting guidelines policy in health journals. In: Moher D, Altman DG, Schulz KF, Simera I, Wager E, eds. Guidelines for Reporting Health Research. Chichester, United Kingdom: Wiley-Blackwell, 2014:309-317. Roberts and Shamseer: J Neuro-Ophthalmol 2015; 35: 3-5 5 Editorial Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited.