Post-acute COVID-19 Syndrome (PCS) related symptoms in primary care

Background COVID-19 survivors report that symptoms flare up unpredictably, that can be debilitating, even 3 months after a positive COVID-19 test. In the multi-project study of symptoms of ‘Long-Haul COVID-19', there is a need to account for clustering at the clinical practice level. This study aims to analyze the prevalence of long-haul COVID-19 symptoms in primary care patients, to compare the burden of symptoms between patients with a positive COVID-19 test result and patients with a negative COVID-19 test result, and investigate possible unmeasured symptoms and latent classes of Long-Haul COVID-19 symptoms. U of U Health Electronic Health record data was combined with Long-haul COVID survey data from the DFPM study with team Primary Investigator: Bernadette Kiraly, with Co-Investigators: Jennifer Leiser, Kirsten Stoesser, Camie Schaefer, and Dominik Ose. This study project is accordant to and exempted by the UofU Institutional Review Board (IRB # 139714). Method For this MSTAT Research Project, multilevel structural equation modeling (MSEM) with bivariate correlations and regression was utilized with real data. This multilevel approach was able to put into consideration potential variances by Health Center, and within-person variances by linking patients to physicians. There were 13 health centers to consider. This approach was important because it brings new focus to the study, answering questions regarding the impact of Health Care interventions. Clinicodemographic breakdowns with data quality reports were created. A Directed Acyclic Graph was created to prepare and address any potential confounders or colliders. Candidate models were assessed with a training data set, and test data set. Additional exploratory analysis was conducted in analyzing the relationship between patient characteristics and hospital visit characteristics. Latent class regression (LCR) analysis was performed to discriminate between untested reinfection symptoms and Long-Haul COVID-19 symptoms. The data uses the UofU Data Science Services (Enterprise Data Warehouse), where there is record data from inpatient and outpatient Electronic Health Records across 13 health centers. Analyses were conducted with respect to the clinic, Covid-19 hospitalization dx, if they died of covid, covid vaccination, primary health insurance, patient county of residence, age, ethnicity/race, Sex/Gender, Smoking Status, Obesity, BMI, or multicomorbidity (Charlson Comorbidity Index).

Post-acute COVID-19 Syndrome (PCS) related symptoms in primary care. Statistician: Andrew Curtin, MSTAT Candidate Datasets: REDCap DFPM Long-HAUL COVID-CCTS 4486 and DFPM Long-Haul COVID SpanishCTSI 4486 survey data. U of U Health Electronic Data Warehouse Health Records. Key Words: COVID-19, Post COVID-19 Syndrome (PCS), SARS-CoV-2, Multilevel Structural Equation Modeling (MSEM), Latent Class Analysis (LCA) ABSTRACT: Background COVID-19 survivors report that symptoms flare up unpredictably, that can be debilitating, even 3 months after a positive COVID-19 test. In the multi-project study of symptoms of ‘Long-Haul COVID-19’, there is a need to account for clustering at the clinical practice level. This study aims to analyze the prevalence of long-haul COVID-19 symptoms in primary care patients, to compare the burden of symptoms between patients with a positive COVID-19 test result and patients with a negative COVID-19 test result, and investigate possible unmeasured symptoms and latent classes of Long-Haul COVID-19 symptoms. U of U Health Electronic Health record data was combined with Long-haul COVID survey data from the DFPM study with team Primary Investigator: Bernadette Kiraly, with Co-Investigators: Jennifer Leiser, Kirsten Stoesser, Camie Schaefer, and Dominik Ose. This study project is accordant to and exempted by the UofU Institutional Review Board (IRB # 139714). Method For this MSTAT Research Project, multilevel structural equation modeling (MSEM) with bivariate correlations and regression was utilized with real data. This multilevel approach was able to put into consideration potential variances by Health Center, and within-person variances by linking patients to physicians. There were 13 health centers to consider. This approach was important because it brings new focus to the study, answering questions regarding the impact of Health Care interventions. Clinicodemographic breakdowns with data quality reports were created. A Directed Acyclic Graph was created to prepare and address any potential confounders or colliders. Candidate models were assessed with a training data set, and test data set. Additional exploratory analysis was conducted in analyzing the relationship between patient characteristics and hospital visit characteristics. Latent class regression (LCR) analysis was performed to discriminate between untested reinfection symptoms and Long-Haul COVID-19 symptoms. The data uses the UofU Data Science Services (Enterprise Data Warehouse), where there is record data from inpatient and outpatient Electronic Health Records across 13 health centers. Analyses were conducted with respect to the clinic, Covid-19 hospitalization dx, if they died of covid, covid vaccination, primary health insurance, patient county of residence, age, ethnicity/race, Sex/Gender, Smoking Status, Obesity, BMI, or multicomorbidity (Charlson Comorbidity Index). Page 1 of 22 Anticipations We anticipated that higher symptom presence and severity would be associated with a previous positive COVID-19 test at 3 or more months, with key symptoms persisting among patients even after a year since the positive COVID-19 test and would not return to a baseline health as the negative COVID-19 test group. We anticipated the presence of latent classes to help describe Positive COVID-19 test patient’s symptoms. We anticipated the need to account for clustering in the hierarchical interventions for Clinics and Primary Care Providers. METHODS Key terminology (Adeloye 2021): -COVID-19 A mild to severe respiratory illness caused by sars-cov-2 coronavirus. -Post COVID-19 Syndrome (PCS) Persistent ill health after 3 months of acute COVID-19. Often also referred to as LongCOVID, Post-acute COVID-19 syndrome, or post-COVID-19 Condition. Sample Inclusions and Exclusions for the Study Population 22,248 invitations were delivered to non-hospitalized UofU health primary care patients with preferred language English or Spanish at 18 years or older of age at the visit. There were 19,321 non-responders and 2,927 responders. 2,539 patients had valid and nonmissing values and participated in an English or Spanish survey regarding COVID-19 history and symptoms. The group of interest was Positive COVID-19 Test patients (n=1,410) with a reference group of negative COVID-19 patients (n=1,129), after 3 or more months since the last COVID-19 test. (Disclaimer: data update is currently ongoing. Final study population n does not necessarily represent final publishable paper results) Operationalization and Classifications of Key Variables Charlson Comorbidity Index (CCI): score of multicomorbidities with a patient range scoring of 0-15 Smoking Status: categories as those that never smoked, quit smoking, or currently smokes. BMI: measured as kg/m2, only accepting valid values within 10-200 BMI COVID-19 Vaccine Status: Categorized as None, or Yes (any) Race: American Indian/Alaskan Native, Asian, Black/African American, Hispanic/Latino, Native Hawaiian/Other Pacific Islander, White, Other, Unknown Page 2 of 22 Sex: Male, Female Age: Age at visit. Also categorized as 18 to 34 years, 35 to 49 years, 50 years and older Time since COVID-19 test result and survey: 3-9 months, 10-12 months, more than 12 months Primary Care Provider: One of 428 providers Clinic: One of 13 named Health Clinics across Utah. Symptom: Self-reported symptom present: Yes, No Self-Reported Symptom Severity: 0=None, 1=mild, 2=moderate, 3=severe N=54 symptoms Page 3 of 22 Statistical Logic Unadjusted P-Value Tests In assessing clinicodemographics, and comparison to adjusted p-values in symptom results, various unadjusted p-value tests were acquired using distribution-free nonparametric tests. The chi-square test and Wilcoxon rank sum test were used in assessing the relationship between two categorical variables in positive and negative test groups. The Kruskal-Wallis test was used for assessing relationships between three or more levels in a categorical variable. Student T-test was used for assessing differences between the positive and negative groups on a continuous dependent variable. Bartlett’s test for Homogeneity of variances was performed in consideration of a one-way ANOVA test. A one-way ANOVA test was considered but not used due to a notable possibility for heteroskedasticity. Due to the response rate (see study population), type 2 error was expected to be small. However, to avoid the overfitting of models, one must recognize to not easily reject the null hypothesis by minimizing the significance level of a hypothesis test. In cases where both unadjusted and adjusted p-values show <0.001 one can more confidently reject the null hypothesis. To check assumptions, a quantile-quantile plot was used to assess the distribution and representation of continuous variables. Scatterplots and boxplots were used to analyze distribution and outliers, and density plots were used to assess normality assumptions. Logistic Regression To begin considerations for logistic regression, a Directed Acyclic Graph (DAG) was created in the open daggity browser. Variable considerations and potential biasing paths were considered, particularly by confounders that would need inclusion in the model or by colliders that would need to be excluded from the model. DAGs were useful in determining sources of bias in model creation as well as in the limitations in the measured variables. Considerations for unmeasured variables were also included in the DAG for considerations in latent variables. Based on the output, predictors of interest from the literature, and on the created DAG, 54 models were selected for 54 symptoms. The model also adjusted for Time after COVID-19 Test, CCI, Race/Ethnicity, Vaccination Status, Sex, Smoking Status, and Obesity. Y=Symptom (yes/no) (binomial) Ŷi=α+β1iXPositive COVID Test(y/n)+β2iXTime after test + β3iXCCI + β4iXRace + β5iXVaccination Status + β6iXSex + β7iXsmoking_status+β8iXObesity To prevent overfitting or type 1 error, data was split into training and test groups, evaluating how well models with coefficients calculated using training data to fit validation data output. Missing values are removed by using listwise deletion Page 4 of 22 MSEM LCA Multilevel Structural Equation Modeling (MSEM) adjusts for statistical dependence while disaggregating variance into each level of analysis. This allows for the specification of distinct models for each level to distinguish level associations and effects. In our data, there is a 3-level hierarchy by the 13 clinics housing 428 Primary Care Providers for the within-person variances of linking patients to physicians. Multi-level modeling methods will account for clustering and possible heteroskedasticity at the clinical practice level, such that both withincluster heterogeneity and between-cluster heterogeneity will be considered. Multivariate multiple regression models will include multiple dependent and independent variables, such that by fit linear mixed-effects model the 3-level model can be reduced to a single model. For these models, there will be considerations for ordinal scale symptom severity, as well as combined severity scores creating. Compared to fixed effects and residual, random effects for this data indicate low variability due to hierarchical structure. From the lme4 package, lmer() function in R was used, with a manually written iterating for loop in testing model fitting. Figure 1. Hierarchical structure in COVID-19 intervention that may lead to cluster specific results at the clinical practice level Least Absolute Shrinkage and Selection Operator (LASSO) was used in assessing predictor performance and for model selection. LASSO was selected as a “soft” threshold method, so would not miss any moderately influential predictors. This means the penalty coefficient is not a hard cutoff for valid coefficients, but rather shrinks coefficients and shrinks non-influential coefficients to zero. LASSO also does not follow the hierarchy principle, so would be a solid contrast and robustness check to the fit linear mixed-effects model results and assumptions. As an iterative process in finding the penalty coefficient and also to prevent overfitting, a 10-fold cross validation was selected with the minimization criterion (see figure 3). Satterwaite’s method t-test is also used for detailed predictor analysis in analyzing predictor subcategories. SS(A|B, AB) for factor A Page 5 of 22 SS(B|A, AB) for factor B This study used latent class analysis to examine subpopulation COVID-19 Test results based on self-reports of long-term symptom severities. Latent Class Analysis divides data into subgroups that relate to an unobserved latent variable. LCA used categorical variables as indicators, as an exhaustive and mutually exclusive approach to exploratory research into what may be COVID-19 reinfections vs long-haul COVID in symptom severity responses. Latent class analysis relies on the assumption of local independence assumes that indicator variables are not correlated and will only be related through the latent variable. (Lazarsfeld and Henry 1968) Figure 2. local independence assumption for Latent Class Model. Model selection was determined based on BIC and AIC values comparing different models (smaller values interpreted as better). “Entropy” was calculated as a measure of how accurately patients were assigned to the classes (a score of 1 indicates 100% certainty of allocation, and scores below 0.9 were treated as possibly non-trivial differences between outputs by LCA). By parsimony rule, such that simple models are preferred if the simple model does the work(Collins & Lanza, 2010), BIC will be an important consideration as BIC is more conservative and will suggest fewer classes. Also, we will rely on the available theory underlying the model, looking for potential discrepancies with theoretical knowledge and then try to deduce from the theory how many classes are to be expected. By an NIHR themed review40, there may be 4 possible syndromes underlying long-haul COVID. At first, 1-10 potential latent classes were analyzed, but due to clear indicators by entropy results, 1-6 potential latent classes were considered. Page 6 of 22 Results (Disclaimer: data update is currently ongoing. The results shown here do not necessarily represent future final publishable paper results. Table 2 includes the full exclusions and data update.) Demographic characteristics were presented as absolute numbers and percentages for categorical variables. The population percentages are by subcategory, and the COVID-19 Test horizontal percentages were calculated by comparing the positive test group to the negative test group selection. See Statistical logic for details on P-value calculations and considerations. Page 7 of 22 Page 8 of 22 Figure 3: Frequently Reported Symptoms (Pre-data update) Page 9 of 22 Figure 4. Odds ratios for the 10 most common symptoms. (pre-data update) The top 20 frequently reported symptoms are shown in table 3 and figure 1, including frequency, and percentage of reporting symptom “Yes”. Unadjusted P-values calculated applying Fisher’s exact test. Adjusted P-values were calculated by the symptom-specific logistic regression. A significant P-value (P<0.05) suggests an association between a COVID-19 test result and reported symptoms. Anxiety was shown to have an insignificant P-value. There were notably high odds ratios among most common symptoms and notably significant 95% confidence intervals above 1. For shortness of breath, an odds ratio of 2.68 (95%CI [2.13-3.39], P<0.001) suggests that the odds of having shortness of breath 3 or more months after a positive COVID-19 test is 2.68 times the odds of having the symptom after having a negative COVID-19 test 3 or more months prior. A flowchart, a poster for the DFPM summit, and 5 other tables were created. They are not shown here but can be found in our final consideration of the publishable paper. These tables Include: Table 4: Frequently reported symptoms by sex Table 5: Frequently reported symptoms by age group Table 6: Frequently reported symptoms by ethnicity Table 7: Frequently reported symptoms by race Page 10 of 22 Table 8: Frequently reported symptoms by time since COVID-19 test In table 8, it was found that Shortness of breath (OR 2.06 [95% CI: 1.34-3.15], P<0.001); was associated with positive test results among patients more than a year out from their COVID-19 diagnosis date. Multi-level Modeling Considerations Linear mixed model fit by maximum likelihood and t-tests use by Satterthwaite’s method. Random effects in this model by severity score sum (pseudo-continuous), indicating low variability due to the hierarchical structure compared to the residual. Ordinal and binomial versions of this model also formed and considered with model selection considerations by criterion AIC and BIC. Page 11 of 22 Predictor considerations (LASSO) In selection of the “best” LASSO model, there was a minimization of test MSE of the penalty coefficient. The manuscript for the publishable paper will compare LASSO to results without LASSO for considerations in improving each symptom model. Figure 5: Penalty coefficient selection by minimizing Mean-squared error. 10-fold cross validation with minimization criterion. Latent Class Analysis The low entropy among classes 5 and above was a clear indicator and support for parsimony. It was found that a 2 class, 3 class and 4 class model all have high entropy, such that the model output supports a near 100% certainty of patient allocation to each class. Both output criterion and literature review (NIHR)40 supported that there may be 4 possible syndromes underlying long haul COVID. Page 12 of 22 Figure 4: 4 discrete identifiable classes derived based on symptom severities. Figure 5: Symptom allocation to 4 classes, with probability to allocation and population share. Page 13 of 22 Figure 6: Symptom allocation to 2 classes, with probability to allocation and population share. Class 1 showing high probability of common symptoms, and class 2 showing varying low degree of common symptoms. Discussion For the publishable paper utilizing logistic regressions, our analysis had key findings: PCS-related symptoms are present in multiple body systems. The brain and nervous system, mental well-being, and general symptoms are the most impacted systems. PCS-related symptoms are common in primary care, independent of COVID-19 status. Memory problems, concentration problems, and shortness of breath are the symptoms with the notable differences and odds ratios between COVID-19 positive and negative groups. The three most common symptoms in each group and overall were fatigue, anxiety, and difficulty sleeping. Out of the top twenty most prevalent symptoms, only anxiety showed statistically insignificant results. Differences in reported symptoms exist and were investigated in tables and figures with respect to age, sex, race, and ethnicity. Page 14 of 22 Fit linear mixed-effects models were used in considerations for hierarchical effects (see statistical logic for details). Random effects in this model by severity score sum (pseudocontinuous), indicates low variability due to the hierarchical structure compared to the residual. Ordinal and binomial versions of this model were also considered. With no clear favorability in model selection by criterion in AIC, BIC, or log likelihood, and also with low interpretability for predictors and betas outputs, these models have not yet been considered for needed inclusion in the final publishable paper. However, as these models also show statistical significance in the same common symptoms as previous models, this is a valid robustness check to the results in the publishable paper. Comorbidity (CCI), Age at Visit, and categorized obesity status were not selected for the Lasso “best” model. Categorized time after COVID-19 test, Vaccine status, race, sex, smoking status, health clinic, primary care provider, and health insurance were selected as nonzero coefficients in the final LASSO model. By Satterthwaite’s method t-test, smoking status “yes/no” is predictive, but the other subcategories are not. The following class assumptions were defined off of a LCA 4 class model. Class 1 shows low probability of memory problems, difficulty breathing, and brain fog/concentration problems, but moderate probability of other common symptoms. Class 2 shows the highest probability of fatigue, memory problems, and joint pain. Class 3 shows the highest probability of headaches, difficulty sleeping, difficulty breathing, and brain fog/concentration problems. Class 4 shows low probability of common symptoms. As even 2 classes show high entropy, using only the positive test group, 2 classes were assessed to attempt to define “recovered” patients to “long-haul” COVID patients. Class 1 shows high probability of common symptoms whereas class 2 shows low probability of common symptoms. Strengths and Weaknesses This study uses a large sample size, which can aid with certainty, normality assumptions, and accounting for type 2 errors. This study is unique in that it uses an extensive list of selfreported symptoms by non-hospitalized patients at a unique time of the COVID epidemic, with a negative COVID-19 test as a comparison group. This study benefited with the use of a Spanish language survey for minority inclusion. Some limitations include how there is a sampling bias for older adults that were more likely to be hospitalized or die from COVID-19 during the pandemic, causing their unintentional exclusion from the study. There also may be self-selection bias noted for patients that join the study to report symptoms, vs those with positive tests but little to no symptoms after an extended time after COVID-19 test. Patients were asked to respond to questions about 54 symptoms and questions were always presented in the same order for each individual, so responses tapered off towards the end of the survey questions. Patients frequently skipped Page 15 of 22 sections where their symptoms did not apply rather than marking no symptoms. Commonly experienced symptoms like concentration problems and fatigue may have also influenced patients’ ability to complete the survey. There was a need to consider short-term COVID, and to consider the new ICD-10-CM diagnosis criteria that went effective October 2021, after survey responses were already completed. In LCA interpretability, there is a risk of the “naming fallacy” of falsely naming or defining class membership. In LASSO interpretability, it ignores non-significant variables that still may be important in describing the study population, with non-selection possibly due to correlation between features. Concluding Finding There is a significantly greater long-term symptom burden in patients who have had COVID-19 among a multitude of symptoms, with particularly high odds ratios for Shortness of Breath, Memory Problems, and Concentration Problems. Multi-level, mixed-effects models also found there is a significantly greater long-term symptom burden in patients who have had COVID-19. Using Lasso, categorized time after COVID-19 test, vaccine status, race, sex, smoking status, health clinic, primary care provider, and health insurance were selected as predictors of Long-Haul COVID. Latent Class Analysis can be a useful aid in investigating unmeasured variables and describing types of COVID-19 patients. Next Steps Data update currently ongoing for date range 03/01/2020 to 10/01/2021. When completed, the publishable paper draft will be sent to external reviewer, revised, and then published. A second future publishable paper will also be considered with additional considerations, including as a qualitative study, a study with follow-up surveys for patients, or with controls for symptoms developed before COVID-19. References Page 16 of 22 1. Adeloye D, Elneima O, Daines L, Poinasamy K, Quint JK, Walker S, et al. The long-term sequelae of COVID-19: an international consensus on research priorities for patients with preexisting and new-onset airways disease. Lancet Respir Med. 2021 Dec;9(12):1467–78. 2. Shah W, Hillman T, Playford ED, Hishmeh L. Managing the long term effects of covid-19: summary of NICE, SIGN, and RCGP rapid guideline. BMJ. 2021 Jan 22;372:n136. 3. Huang C, Huang L, Wang Y, Li X, Ren L, Gu X, et al. 6-month consequences of COVID-19 in patients discharged from hospital: a cohort study. The Lancet. 2021 Jan;397(10270):220–32. 4. Davis HE, Assaf GS, McCorkell L, Wei H, Low RJ, Re’em Y, et al. Characterizing long COVID in an international cohort: 7 months of symptoms and their impact. EClinicalMedicine. 2021 Aug;38:101019. 5. Sudre CH, Murray B, Varsavsky T, Graham MS, Penfold RS, Bowyer RC, et al. Attributes and predictors of long COVID. Nat Med. 2021 Apr;27(4):626–31. 6. Huang Y, Pinto MD, Borelli JL, Mehrabadi MA, Abrihim H, Dutt N, et al. COVID Symptoms, Symptom Clusters, and Predictors for Becoming a Long-Hauler: Looking for Clarity in the Haze of the Pandemic. MedRxiv Prepr Serv Health Sci. 2021 Mar 5;2021.03.03.21252086. 7. Lopez-Leon S, Wegman-Ostrosky T, Perelman C, Sepulveda R, Rebolledo PA, Cuapio A, et al. More than 50 long-term effects of COVID-19: a systematic review and meta-analysis. Sci Rep. 2021 Aug 9;11(1):16144. Page 17 of 22 8. Pérez-González A, Araújo-Ameijeiras A, Fernández-Villar A, Crespo M, Poveda E. Long COVID in hospitalized and non-hospitalized patients in a large cohort in Northwest Spain, a prospective cohort study. Sci Rep. 2022 Mar 1;12(1):3369. 9. Townsend L, Dowds J, O’Brien K, Sheill G, Dyer AH, O’Kelly B, et al. Persistent Poor Health after COVID-19 Is Not Associated with Respiratory Complications or Initial Disease Severity. Ann Am Thorac Soc. 2021 Jun;18(6):997–1003. 10. Goërtz YMJ, Van Herck M, Delbressine JM, Vaes AW, Meys R, Machado FVC, et al. Persistent symptoms 3 months after a SARS-CoV-2 infection: the post-COVID-19 syndrome? ERJ Open Res. 2020 Oct;6(4):00542–2020. 11. Vance H, Maslach A, Stoneman E, Harmes K, Ransom A, Seagly K, et al. Addressing Post- COVID Symptoms: A Guide for Primary Care Physicians. J Am Board Fam Med. 2021 Nov;34(6):1229–42. 12. Desgranges F, Tadini E, Munting A, Regina J, Filippidis P, Viala B, et al. Post‑COVID‑19 Syndrome in Outpatients: a Cohort Study. J Gen Intern Med. 2022 Mar 22; 13. Huston P, Campbell J, Russell G, Goodyear-Smith F, Phillips RL, Weel C van, et al. COVID- 19 and primary care in six countries. BJGP Open [Internet]. 2020 Oct 1 [cited 2022 May 11];4(4). Available from: https://bjgpopen.org/content/4/4/bjgpopen20X101128 14. Mughal F, Khunti K, Mallen C. The impact of COVID-19 on primary care: Insights from the National Health Service (NHS) and future recommendations. J Fam Med Prim Care. 2021;10(12):4345. Page 18 of 22 15. Starfield B. Is primary care essential? Lancet Lond Engl. 1994 Oct 22;344(8930):1129–33. 16. Finley CR, Chan DS, Garrison S, Korownyk C, Kolber MR, Campbell S, et al. What are the most common conditions in primary care? Can Fam Physician. 2018 Nov;64(11):832–40. 17. United States Census Bureau. QuickFacts - Salt Lake City city, Utah [Internet]. United States Census Bureau. Available from: US Census https://www.census.gov/quickfacts/saltlakecitycityutah 18. Ali MK, Shah DJ, Del Rio C. Preparing Primary Care for COVID-20. J Gen Intern Med. 2020 Jun 9; 19. Greenhalgh T, Knight M, A’Court C, Buxton M, Husain L. Management of post-acute covid-19 in primary care. BMJ. 2020 Aug 11;m3026. 20. Parkin A, Davison J, Tarrant R, Ross D, Halpin S, Simms A, et al. A Multidisciplinary NHS COVID-19 Service to Manage Post-COVID-19 Syndrome in the Community. J Prim Care Community Health. 2021 Jan 1;12:21501327211010990. 21. Ladds E, Rushforth A, Wieringa S, Taylor S, Rayner C, Husain L, et al. Developing services for long COVID: lessons from a study of wounded healers. Clin Med. 2021 Jan;21(1):59–65. 22. Dennis A, Wamil M, Alberts J, Oben J, Cuthbertson DJ, Wootton D, et al. Multiorgan impairment in low--risk individuals with post-C- OVID-19 syndrome: a prospective, community-based study. Open Access. :11. Page 19 of 22 23. Nehme M, Braillard O, Chappuis F, Courvoisier DS, Guessous I. Prevalence of Symptoms More Than Seven Months After Diagnosis of Symptomatic COVID-19 in an Outpatient Setting. Ann Intern Med. 2021 Sep 21;174(9):1252–60. 24. Nittas V, Gao M, West EA, Ballouz T, Menges D, Wulf Hanson S, et al. Long COVID Through a Public Health Lens: An Umbrella Review. Public Health Rev. 2022 Mar 15;43:1604501. 25. Sathyamurthy P, Madhavan S, Pandurangan V. Prevalence, Pattern and Functional Outcome of Post COVID-19 Syndrome in Older Adults. Cureus. 2021 Aug 15;13(8):e17189. 26. Staner L. Sleep and anxiety disorders. Dialogues Clin Neurosci. 2003 Sep 30;5(3):249–58. 27. Altevogt BM, Colten HR, others. Sleep disorders and sleep deprivation an unmet public health problem. 2006; 28. Bungenberg J, Humkamp K, Hohenfeld C, Rust MI, Ermis U, Dreher M, et al. Long COVID- 19: Objectifying most self-reported neurological symptoms. Ann Clin Transl Neurol. 2022 Feb;9(2):141–54. 29. Becker JH, Lin JJ, Doernberg M, Stone K, Navis A, Festa JR, et al. Assessment of Cognitive Function in Patients After COVID-19 Infection. JAMA Netw Open. 2021 Oct 22;4(10):e2130645. 30. Anjana NKN, Annie TT, Siba S, Meenu MS, Chintha S, Anish TSN. Manifestations and risk factors of post COVID syndrome among COVID-19 patients presented with minimal symptoms A study from Kerala, India. J Fam Med Prim Care. 2021 Nov;10(11):4023–9. Page 20 of 22 31. Cirulli ET, Schiabor Barrett KM, Riffle S, Bolze A, Neveux I, Dabe S, et al. Long-term COVID-19 symptoms in a large unselected population [Internet]. Infectious Diseases (except HIV/AIDS); 2020 Oct [cited 2022 Jan 31]. Available from: http://medrxiv.org/lookup/doi/10.1101/2020.10.07.20208702 32. Sagar-Ouriaghli I, Godfrey E, Bridge L, Meade L, Brown JSL. Improving Mental Health Service Utilization Among Men: A Systematic Review and Synthesis of Behavior Change Techniques Within Interventions Targeting Help-Seeking. Am J Mens Health. 2019 Jun 11;13(3):1557988319857009. 33. Artiga S, Hill L, Haldar S. COVID-19 cases and deaths by race/ethnicity: Current data and changes over time. Kais Fam Found Retrieved Novemb. 2021;6:2021. 34. Wielen LMV, Gilchrist EC, Nowels MA, Petterson SM, Rust G, Miller BF. Not Near Enough: Racial and Ethnic Disparities in Access to Nearby Behavioral Health Care and Primary Care. J Health Care Poor Underserved. 2015;26(3):1032–47. 35. Callard F, Perego E. How and why patients made Long Covid. Soc Sci Med. 2021 Jan;268:113426. 36. National Institute for Health and Care Excellence. (2021, November 11). Overview | COVID-19 rapid guideline: Managing the long-term effects of COVID-19 | Guidance | NICE. NICE.https://www.nice.org.uk/guidance/ng188 37. https://www.cdc.gov/nchs/data/icd/announcement-new-icd-code-for-post-covidcondition-april-2022-final.pdf 38. NIHR Themed Review: Living with Covid19; October 2020; doi:10.3310/themedreview_41169 Page 21 of 22 Acknowledgments: Supervisory Committee: Greg Stoddard, Chair Bradley Barney, PhD Dominik Ose, PhD With assistance from DFPM Team. Special thanks to Elena Gardner, Robert Owens, and James Vanderslice, PhD. Long-haul COVID survey data from the DFPM study Team Primary Investigator: Bernadette Kiraly Co-Investigators: Jennifer Leiser, Kirsten Stoesser, Camie Schaefer, and Dominik Ose. The created figures and tables are my own, and do not necessarily reflect any future publications. Page 22 of 22