Predictive Value of International Classification of Diseases Codes for Idiopathic Intracranial Hypertension in a University Health System

Misclassification bias is introduced into medical claims-based research because of reliance on diagnostic coding rather than full medical record review. We sought to characterize this bias for idiopathic intracranial hypertension (IIH) and evaluate strategies to reduce it.

Original Contribution Section Editors: Clare Fraser, MD Susan Mollan, MD Predictive Value of International Classification of Diseases Codes for Idiopathic Intracranial Hypertension in a University Health System Fareshta Khushzad, BS, Riya Kumar, Irma Muminovic, MD, Heather E. Moss, MD, PhD Background: Misclassification bias is introduced into medical claims–based research because of reliance on diagnostic coding rather than full medical record review. We sought to characterize this bias for idiopathic intracranial hypertension (IIH) and evaluate strategies to reduce it. Methods: A retrospective review of medical records was conducted using a clinical data warehouse containing medical records and administrative data from an academic medical center. Patients with 1 or more instances of International Classification of Diseases (ICD)-9 or -10 codes for IIH (348.2 or G93.2) between 1989 and 2017 and original results of neuroimaging (head CT or MRI), lumbar puncture, and optic nerve examination were included in the study. Diagnosis of IIH was classified as definite, probable, possible, or inaccurate based on review of medical records. The positive predictive value (PPV) for IIH ICD codes was calculated for all subjects, subjects with an IIH code after all testing was completed, subjects with high numbers of IIH ICD codes and codes spanning longer periods, subjects with IIH ICD codes associated with expert encounters (ophthalmology, neurology, or neurosurgery), and subjects with acetazolamide treatment. Results: Of 1,005 patients with ICD codes for IIH, 103 patients had complete testing results and were included in the study. PPV of ICD-9/-10 codes for IIH was 0.63. PPV in restricted samples was 0.82 (code by an ophthalmologist n = 57), 0.70 (acetazolamide treatment n = 87), and 0.72 (code after all testing, n = 78). High numbers of code instances and longer duration between the first and last code instance also increased the PPV. Conclusions: An ICD-9 or -10 code for IIH had a PPV of 63% for probable or definite IIH in patients with necessary diagnostic testing performed at a single institution. Coding accuracy was improved in patients with an IIH ICD code assigned by an ophthalmologist. Use of coding algorithms considerDepartment of Ophthalmology (FK, RK, IM, HEM), Stanford University, Palo Alto, California; and Department of Neurology & Neurological Sciences (HEM), Stanford University, Palo Alto, California NIH K23 EY 024345, NIH P30 026877, Research to prevent blindness unrestricted grant to Stanford Department of Ophthalmology. The authors report no conflicts of interest. Address correspondence to Heather E. Moss, MD, PhD, Spencer Center for Vision Research at Stanford MC 5353, 2370 Watson Court, Suite 200, Palo Alto, CA; E-mail: [e-mail] Khushzad et al: J Neuro-Ophthalmol 2021; 41: e679-e683 ing treatment providers, number of codes, and treatment is a potential strategy to reduce misclassification bias in medical claims–based research on IIH. However, these are associated with a reduced sample size. Journal of Neuro-Ophthalmology 2021;41:e679–e683 doi: 10.1097/WNO.0000000000000992 © 2020 by North American Neuro-Ophthalmology Society I diopathic intracranial hypertension (IIH) causes disability and reduced quality of life because of headaches and vision loss, which is permanent and at the level of blindness in a small but significant number of affected individuals (1). The prevalence is increasing with associated increased burden on health care systems around the world (2). Proposed risk factors identified based on associations with IIH in case reports, case series, and case–control studies, include female sex, obesity, and certain medications (3). However, many of these have not been confirmed in a population sample. A big data approach using medical claims data has potential applications in supporting these as causal associations because they capture real-world experience and have relatively large sample sizes compared with traditional epidemiologic study approaches (4). An important part of any medical study is identification of cases. Medical claims data contain information about medical encounters including type, diagnostic codes (e.g., International Classification of Diseases [ICD] versions 9 and 10), and procedure codes (e.g., Current Procedural Terminology [CPT]) that can be leveraged to identify cases. For example, IIH cases might be identified as those with an encounter including the ICD-9 diagnostic code 348.2 or ICD-10 diagnostic code G93.2. However, claims data lack the detailed medical records necessary to do this based on diagnostic test and examination results and therefore may lead to misclassification bias. This was demonstrated in a study of IIH patients receiving care in the emergency room at an academic medical center where only 55% of e679 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Original Contribution charts with the ICD-9 code for IIH met diagnostic criteria on full chart review (5). More detailed algorithms have applied additional inclusion criteria such as requiring a diagnostic code for IIH being filed after completion of necessary testing (6). However, the accuracy of such algorithms has not been evaluated. The objective of this study is to compare the accuracy (positive predictive value [PPV]) of IIH case identification algorithms for patients receiving care in a university health system using medical claims data. METHODS This is a retrospective study performed using the Stanford Research Repository (STARR), which is a clinical data warehouse containing medical records and administrative data for patients receiving care at Stanford Health Care from 1995 to present. Approval for this study was granted by the Stanford Office of Research with a waiver of informed consent. Subjects for this study were those who had an ICD code for IIH and records of completing necessary testing to confirm the diagnosis within the health system. Potential subjects were those with one or more encounters associated with ICD codes for IIH (ICD-9 348.2 or ICD-10 G93.2) before July 2017, which was the date when we commenced this retrospective study. Inclusion criteria were completion of lumbar puncture (LP) with available report (CPT codes 62270, 62272, ICD-9 03.31, or ICD-10 009U3*), completion of neuroimaging (CT head [CPT 70450, 70460, or 70470] or MRI brain [70551 or 70553] based on manual review of radiology records) with available report from within 12 months of LP, and documentation of optic nerve examination (search for “fundus examination, eye examination, or optic nerve examination”) within 12 months of LP. Medical records of the included subjects were used as the basis for diagnosis classification. We used the modified Dandy criteria as used in the IIH Treatment Trial (IIHTT), stratified based on the revised pseudotumor cerebri diagnostic criteria into definite, probable, and possible (7,8). All IIH diagnoses required lack of secondary causes of high ICP on neuroimaging, CSF analysis, and medication history. Definite IIH was diagnosed for an LP opening pressure .25 cm H2O and papilledema or optic atrophy on fundus examination. Probable IIH was diagnosed for LP opening pressure 20–25 cm H2O with optic nerve edema on examination or findings of high ICP noted in neuroimaging reports (e.g., empty sella, globe flattening, or increased CSF in optic nerve sheath) or if opening pressure was not recorded, optic nerve edema on examination responsive to therapy. Possible IIH was diagnosed in subjects with an LP opening pressure .25 cm H2O without papilledema, optic atrophy, or findings of high ICP noted in neuroimaging reports. Subjects on treatment for IIH at the time of normal e680 eye examination were classified as possible (on treatment). Subjects not meeting criteria for IIH were classified as not having IIH, and alternative diagnoses for their symptoms were recorded. Accuracy of case identification for the entire sample was calculated as the PPV (# IIH/total # subjects). Accuracy of case identification for patients seen by expert specialists was calculated using the subgroups of subjects with an IIH code associated with an ophthalmology, neurology, or neurosurgery encounter. Accuracy of case identification for patients with the appropriate diagnostic sequence was calculated using the subgroups of subjects with at least 1 diagnostic code for IIH either on the same day of completion of necessary testing (i.e., neuroimaging and LP) or on the subsequent day. Accuracy of case identification for patients treated with acetazolamide was calculated using the subgroup of subjects with an acetazolamide prescription in the medical record. The impact on continuity of care for IIH on accuracy of case identification was studied using duration (in years) between initial and most recent IIH-associated encounters, the total number of IIH code instances, and the number of unique days with IIH codes. Case identification accuracy was calculated within subject groups defined by quartiles of these continuous variables. RESULTS A total of 1005 potential subjects with 1 or more instances of ICD-9/-10 codes for IIH (348.2 or G93.2) between 1995 and 2017 were identified. Three hundred of these subjects had LPs with reports. Of them, 55 lacked neuroimaging within the necessary time frame, 105 lacked optic nerve examinations, and 37 lacked both. One hundred three subjects were included in the study (Table 1). Of the 65 subjects with IIH, all definite cases (n = 39) and 12/15 probable cases had papilledema. Three probable cases had an unknown papilledema status because eye examination was documented after treatment, but they were included as probable due to neuroimaging findings. Three of 12 possible cases lacked papilledema at the time of diagnosis, and 9/12 did not have an eye examination documented until after treatment. There were no patients with papilledema with ICP ,20 cm H2O. Subjects without IIH had primary headache syndrome (12), metastatic or primary brain tumors (9), meningitis (5), venous sinus thrombosis (3), intracerebral hemorrhage (3), inflammation (2), and 1 each of hydrocephalus, medication induced IH, CSF leak, and encephalocele. Overall case identification accuracy for subjects with ICD-9 or -10 codes for IIH with necessary testing was 63.1%. This was increased to 71%–82% when only subjects with codes associated with expert specialists (neurosurgery, neurology, and ophthalmology) were considered (Table 2). Consideration of medical treatment with acetazolamide or appropriate diagnostic sequence with an IIH Khushzad et al: J Neuro-Ophthalmol 2021; 41: e679-e683 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Original Contribution TABLE 1. Subjects with ICD-9 or -10 coding for IIH, lumbar puncture, neuroimaging, and optic nerve examination at an academic medical center (n = 103) Variable Age Sex LP opening pressure Elevated (.25 cm H2O) Borderline (20–25 cm H2O) Normal (,20 cm H2O) Missing CSF analysis Normal Likely normal* Abnormal Missing Neuroimaging Normal ICP-associated findings Abnormal—unrelated findings Abnormal—secondary cause Optic disk examination Edema Atrophy Other optic disc findings Normal Diagnosis IIH Definite Probable Possible (no papilledema on treatment) Possible (no papilledema before treatment) Not IIH Distribution 36.6 ± 13.9 years 76 (74%) female 63 14 13 13 (61%) (14%) (13%) (13%) 54 (52%) 18 (17%) 22 (21%) 9 (9%) 49 19 17 18 (48%) (18%) (16%) (17%) 54 (52%) 5 (5%) 1 (1%) 43 (42%) 65 (63%) 39 (38%) 15 (15%) 8 (8%) 3 (3%) 38 (37%) *Isolated elevation in RBC or protein. IIH, idiopathic intracranial hypertension; LP, lumbar puncture. code given simultaneously with or after necessary diagnostic testing slightly improved case identification accuracy (69%– 72%, Table 3). The amount of IIH codes either by time or number improved case identification accuracy to 83%–92% for the upper quartiles of these variables (Table 4). DISCUSSION Medical coding using ICD and CPT codes, along with pharmacy records, constitutes a map of health care delivery. The entries associated with individuals track their route through this map. In addition to applications for medical billing, analysis of this data can provide important insights into disease risk factors and delivery of care. Understanding the accuracy of coding is critical to development and interpretation of such studies. In the current study, we assessed the accuracy of ICD-9 and -10 coding for IIH at an academic medical center and evaluated strategies to improve accuracy through sample selection criteria. We find that the overall accuracy is 63% for 1 or more instances of ICD codes for IIH in patients with neuroimaging, LP, and fundus examination. Tightening criteria to include coding by Khushzad et al: J Neuro-Ophthalmol 2021; 41: e679-e683 a specialist, diagnostic sequence, medical treatment, number of codes, or duration of codes all increased the accuracy of ICD codes for prediction of IIH. These also reduced the sample size of the collected cases. These results have relevance for design and interpretation of medical claims–based research on IIH. Diagnostic accuracy of IIH codes in our study is similar to that reported in 2 previous studies. A study of ER and inpatient utilization by IIH patient visits at a single institution reported an accuracy of the IIH ICD-9 code as 55% (5). Similarities between this study and ours include location (single US academic medical centers). Differences include inclusion of patients without ER or inpatient care in the current study and consideration of external health records in diagnostic categorization of the previous study. A study using the national patient register in a Swedish county reported an accuracy of the IIH ICD-10 code as 65% (9). In contrast to our study, this was a non-American population–based study not limited to a single institution. Multiple factors likely contribute to this low accuracy including interprovider variability in assigning codes, assignment of codes by administrative staff, and e681 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Original Contribution TABLE 2. Coding accuracy for IIH codes associated with encounters with relevant medical subspecialties Specialist Neurology Ophthalmology Neurosurgery n Accuracy (PPV) 52 57 17 0.75 0.82 0.71 IIH, idiopathic intracranial hypertension; PPV, positive predictive value. transcription errors (10). The errors may be administrative (11) or may reflect true diagnostic errors on behalf of the treatment providers (12). A strategy to improve case identification from medical claims data is to modify the inclusion and exclusion criteria for case selection beyond a single ICD code. We found that tightening the sample selection–based ICD code associated with a specialist encounter (neurology, ophthalmology, or neurosurgery), ICD code on the same or a later day as diagnostic test completion, acetazolamide treatment, more instances of ICD codes (exclude ,3 and include .21), more unique days with ICD codes (.5 days), and longer duration between the initial and most recent code (.1 year) each improved classification accuracy. However, each of these also reduced the sample size of included cases, and the tradeoff between accuracy and sample size needs to be considered in applications of these findings. Duration of codes, coding by an ophthalmologist, and diagnostic sequence are factors that improved IIH coding accuracy in our study that have not been previously considered. Sundholm et al considered age, sex, number of times the IIH code was recorded, visit with a neurologist, and acetazolamide prescription to develop a coding algorithm (13). An algorithm considering age and 3 or more instances of the IIH ICD-10 code increased accuracy to greater than 80% from 65% in a development sample and was confirmed in a validation sample (13). The other variables were found not to be helpful in case identification. Sohdi et al used an inclusion criteria of CPT codes for neuroimaging and LP within 15 days of the IIH code to identify IIH cases (6). Mollan et al used the exclusion criteria of ICD-10 codes for secondary causes of high ICP (e.g., hydrocephalus, cerebral venous sinus thrombosis, TABLE 3. Coding accuracy for IIH codes associated with appropriate medical management Patient Management Diagnostic sequence Same day Separate day Acetazolamide treatment n Accuracy (PPV) 87 78 87 0.69 0.72 0.70 IIH, idiopathic intracranial hypertension, PPV, positive predictive value. e682 brain cancer, and hypertensive encephalopathy) (2), a strategy in line with validated case identification strategies for other rare diseases (14). Beyond true diagnostic errors, a challenge in both clinical care for and research on IIH is debate regarding the spectrum of this entity. Per its title, IIH requires intracranial hypertension that is idiopathic. However, the diagnostic criteria move beyond these 2 basic criteria to address the challenges in accurately measuring intracranial pressure and to select people with pathophysiological effects from high ICP. This is because 2.5% of normal adults have ICP .25 cm H2O (15) whereas 10% of those with clear pathological effects of elevated ICP (i.e., papilledema) have ICP #25 cm H2O (16). Diagnostic criteria considering papilledema, sixth nerve palsy, and radiologic features narrow the diagnosis to those with measurable effects of high ICP. Classification schema that allow for definite, probable, and possible cases, such as the one applied in this article, allow for consideration of the broader spectrum of disease (8). A limitation of these is likely false positive diagnoses, particularly in the possible category. In our study, the narrow inclusion criteria, requiring primary documentation of necessary testing available in the medical record of the study center, limited the sample size but accurately reflect the approach taken in claims-based IIH analysis, which is case selection based on diagnosis (ICD) and testing (CPT) codes (6). Although our sample size is on the same order of magnitude as previous studies (5,13), it is discouraging that only 103 subjects were identified from over 1,000 patients with an IIH code in the medical record of the study center. The approximately 900 excluded patients likely fall into 3 categories: inaccurate diagnosis, incomplete diagnosis, or accurate diagnosis with testing completed elsewhere. To classify these patients TABLE 4. Association between duration and volume of ICD-9/-10 codes for IIH and coding accuracy Instances of code 1–2 codes 3–8 8–21 .21 Unique days with code 1 days 2–5 6–15 .15 Duration with diagnosis codes 1 year 2 3 .3 n PPV 28 24 26 25 0.36 0.63 0.65 0.92 23 28 26 22 0.30 0.61 0.73 0.85 48 21 10 24 0.43 0.76 0.80 0.83 IIH, idiopathic intracranial hypertension, PPV, positive predictive value. Khushzad et al: J Neuro-Ophthalmol 2021; 41: e679-e683 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited. Original Contribution would require collection and review of medical records from outside institutions with patient permission, which is beyond the scope of this chart-based study but a future research opportunity. CONCLUSION The current investigation determined that the PPV for ICD coding of IIH is 63% among patients with necessary testing performed. The findings are similar to previously reported estimates for IIH ICD code accuracy. This low accuracy supports the concern for misclassification bias in studies identifying IIH cases based on ICD coding alone and needs to be considered in the interpretation of these studies. We offer strategies for improving accuracy of case identification. Further research is needed to validate these strategies in other populations and to determine the root of coding error. STATEMENT OF AUTHORSHIP Category 1: a. Conception and design: FK, RK, IM, and HM; b. Acquisition of data: RK and IM; c. Analysis and interpretation of data: FK, RK, IM, and HM. Category 2: a. Drafting the manuscript: FK and RK; b. Revising it for intellectual content: HM. Category 3: a. Final approval of the completed manuscript: FK, RK, IM, and HM. REFERENCES 1. Corbett JJ, Savino PJ, Thompson HS, Kansu T, Schatz NJ, Orr LS, Hopson D. Visual loss in pseudotumor cerebri. Follow-up of 57 patients from five to 41 years and a profile of 14 patients with permanent severe visual loss. Arch Neurol. 1982;39:461– 474. 2. Mollan SP, Aguiar M, Evison F, Frew E, Sinclair AJ. The expanding burden of idiopathic intracranial hypertension. Eye (Lond). 2018;33:478–485. 3. Ahmad SR, Moss HE. Update on the diagnosis and treatment of idiopathic intracranial hypertension. Semin Neurol. 2019;39:682–691. Khushzad et al: J Neuro-Ophthalmol 2021; 41: e679-e683 4. Moss HE, Joslin CE, Rubin DS, Roth S. Big data research in neuro-ophthalmology: promises and pitfalls. J Neuroophthalmol. 2019;39:480–486. 5. Koerner JC, Friedman DI. Inpatient and emergency service utilization in patients with idiopathic intracranial hypertension. J Neuroophthalmol. 2014;34:229–232. 6. Sodhi M, Sheldon CA, Carleton B, Etminan M. Oral fluoroquinolones and risk of secondary pseudotumor cerebri syndrome: nested case-control study. Neurology. 2017;89:792–795. 7. Wall M, Kupersmith MJ, Kieburtz KD, Corbett JJ, Feldon SE, Friedman DI, Katz DM, Keltner JL, Schron EB, McDermott MP. The idiopathic intracranial hypertension treatment trial: clinical profile at baseline. JAMA Neurol. 2014;71:693–701. 8. Friedman DI, Liu GT, Digre KB. Revised diagnostic criteria for the pseudotumor cerebri syndrome in adults and children. Neurology. 2013;81:1159–1165. 9. Sundholm A, Burkill S, Sveinsson O, Piehl F, Bahmanyar S, Nilsson Remahl AIM. Population-based incidence and clinical characteristics of idiopathic intracranial hypertension. Acta Neurol Scand. 2017;136:427–433. 10. Lee CD, Williams SE, Sathe NA, McPheeters ML. A systematic review of validated methods to capture several rare conditions using administrative or claims data. Vaccine. 2013;31(suppl 10):K21–K27. 11. Lau M, Prenner JL, Brucker AJ, VanderBeek BL. Accuracy of billing codes used in the therapeutic care of diabetic retinopathy. JAMA Ophthalmol. 2017;135:791–794. 12. Stunkel L, Kung NH, Wilson B, McClelland CM, Van Stavern GP. Incidence and causes of overdiagnosis of optic neuritis. JAMA Ophthalmol. 2018;136:76–81. 13. Sundholm A, Burkill S, Bahmanyar S, Nilsson Remahl AIM. Improving identification of idiopathic intracranial hypertension patients in Swedish patient register. Acta Neurol Scand. 2018;137:341–346. 14. Mezaache S, Derumeaux H, Ferraro P, Capdepon P, Steinbach JC, Abballe X, Palas D, Saichi N, Desboeuf K, Lapeyre-Mestre M, Sailler L, Moulis G. Validation of an algorithm identifying incident primary immune thrombocytopenia in the French national health insurance database. Eur J Haematol. 2017;99:344–349. 15. Whiteley W, Al-Shahi R, Warlow CP, Zeidler M, Lueck CJ. CSF opening pressure: reference interval and the effect of body mass index. Neurology. 2006;67:1690–1691. 16. Corbett JJ, Mehta MP. Cerebrospinal fluid pressure in normal obese subjects and patients with pseudotumor cerebri. Neurology. 1983;33:1386–1388. e683 Copyright © North American Neuro-Ophthalmology Society. Unauthorized reproduction of this article is prohibited.