Behavioral Health Integration in Pediatric Primary Care

Pediatric behavioral health problems are a significant issue in healthcare. Over 15 million children are diagnosed with behavioral health problems in the United States. The cost of managing behavioral health in pediatric patients is estimated to be 40 billion dollars. Access to behavioral health care in the pediatric population is a significant challenge, including the difficulty of navigating the healthcare system, wait times for behavioral health treatment, and a shortage of behavioral health providers. Primary care providers and behavioral health providers continue to note dissatisfaction with managing pediatric behavioral health. A quality improvement project was developed to improve provider satisfaction with managing pediatric behavioral health and improve pediatric behavioral health outcomes by integrating a behavioral health provider into a private pediatric primary care clinic. In collaboration with a behavioral health clinic, a licensed social worker was at the private pediatric primary clinic for one month to provide access to behavioral health services onsite. Primary care providers and behavioral health providers completed a pre-intervention survey and a post-intervention survey to evaluate the effectiveness of the intervention and evaluate if the intervention improved provider satisfaction with behavioral health management in pediatric patients. Baseline and post-intervention PHQ9 and GAD7 scores were obtained to determine if the intervention improved behavioral health outcomes in the pediatric population. Primary care providers referred 35 patients to the onsite licensed social worker during the intervention, and n=25 (71%) were seen on the same day as the referral reducing the gap from referral to initial behavioral health contact. There was a 50% increase in primary care providers reporting not burdensome or slightly burdensome with managing behavioral health after the intervention. Behavioral health integration into pediatric primary care appears to improve access to initial behavioral health contact and improved primary care provider satisfaction in managing behavioral health. Replication of this quality improvement is needed with a longer duration to determine if it will improve pediatric behavioral health outcomes.

1 Behavioral Health Integration in Pediatric Primary Care Corinne Bartschi, Lisa Taylor-Swanson, ElLois Bailey College of Nursing: The University of Utah NURS 7703: DNP Scholarly Project III April 24, 2022 2 Abstract Background: Pediatric behavioral health problems are a significant issue in healthcare. Over 15 million children are diagnosed with behavioral health problems in the United States. The cost of managing behavioral health in pediatric patients is estimated to be 40 billion dollars. Access to behavioral health care in the pediatric population is a significant challenge, including the difficulty of navigating the healthcare system, wait times for behavioral health treatment, and a shortage of behavioral health providers. Primary care providers and behavioral health providers continue to note dissatisfaction with managing pediatric behavioral health. Methods: A quality improvement project was developed to improve provider satisfaction with managing pediatric behavioral health and improve pediatric behavioral health outcomes by integrating a behavioral health provider into a private pediatric primary care clinic. In collaboration with a behavioral health clinic, a licensed social worker was at the private pediatric primary clinic for one month to provide access to behavioral health services onsite. Primary care providers and behavioral health providers completed a pre-intervention survey and a post-intervention survey to evaluate the effectiveness of the intervention and evaluate if the intervention improved provider satisfaction with behavioral health management in pediatric patients. Baseline and post-intervention PHQ9 and GAD7 scores were obtained to determine if the intervention improved behavioral health outcomes in the pediatric population. Results: Primary care providers referred 35 patients to the onsite licensed social worker during the intervention, and n=25 (71%) were seen on the same day as the referral reducing the gap from referral to initial behavioral health contact. There was a 50% increase in primary care providers reporting not burdensome or slightly burdensome with managing behavioral health after the intervention. Conclusions: Behavioral health integration into pediatric primary care appears to improve access to initial behavioral health contact and improved primary care provider satisfaction in managing behavioral health. Replication of this quality improvement is needed with a longer duration to determine if it will improve pediatric behavioral health outcomes. 3 Behavioral Health Integration in Pediatric Primary Care Problem Description Habegar and Venable (2017) and Lauerer et al. (2018) report that up to 20% of children in the United States are diagnosed with a behavioral health (BH) problem. This equates to over 15 million children nationwide (Walter et al., 2019). BH problems most often present in primary care (Habegar & Venable, 2017; Herbst et al., 2020). The cost of managing BH in pediatric patients is higher than many other pediatric medical problems, with annual costs estimated to be 40 billion dollars (Lauerer et al., 2018; Walter et al., 2019). Access to BH services is often difficult due to gaps in care. These gaps include patient and family problems navigating the healthcare system, patients lost in the system between primary care and BH due to long wait times for appointments, and a shortage of behavioral health providers (BHP) (Habegar & Venable, 2017; Lauerer et al., 2018; Okafor et al., 2018). Of those children with a BH diagnosis, only 15 to 25% receive care (Habegar & Venable, 2017; Lauerer et al., 2018). These barriers make it difficult for pediatric patients to receive necessary BH treatment (Rodriguez et al., 2018). Available Knowledge Pediatric primary care clinics are most often the initial point of contact for parents regarding concerns about behavior and development (Habegar & Venable, 2017; Hansel et al., 2017; Schlesiger, 2017). An estimated 50% of BH diagnoses manifest before a child is 14 years old (Habegar & Venable, 2017). According to Lauerer et al. (2018), "In the care of children and adolescents, behavioral health integration is not just important, but imperative" (p. 39). By integrating BH into primary care, long-term outcomes can be improved by focusing on early identification and management (Lauerer et al., 2018). Integration of BH and pediatric primary care has significant potential to expand BH services in primary care, increase the ability to triage BH concerns, and decrease wait times for BH initial evaluation, assessment, and treatment (Habeger & Venable, 2017). The American Academy of Pediatrics (AAP), the American Academy of Child and Adolescent Psychiatry (AACAP), the National Committee for Quality Assurance (NCQA), and the Institute of Medicine (IOM) have strongly recommended increased 4 collaboration between pediatric primary care and BH (Kazak et al., 2017; Lauerer et al., 2018; Owens et al., 2021; Schlesinger, 2017; Talmi et al., 2016; Walter et al., 2019). Several barriers exist in BH treatment of pediatric patients, among them a shortage of BHP, including child psychiatrists, psychiatric mental health nurse practitioners (PMHNPs), licensed professional counselors (LPCs), and licensed social workers (LSWs) (Lauerer et al., 2018; Walter et al., 2019). This shortage in available clinicians contributes to obstacles in accessing BH and extended wait times for BH services (Habeger & Venable, 2017; Lauerer et al., 2018). Pediatric primary care providers (PCPs) often refer to BH services, but completion of these services, for example, going to a BH appointment, is low – ranging from 17 to 65% (Hacker et al., 2013). The shortage of pediatric BHPs leads to pediatric PCPs being the primary prescribers for most medications for BH diagnoses such as depression and anxiety (Hansel et al., 2017; Schlesiger, 2017). Integrating BH into pediatric primary care improves access to and availability of BH care. Primary care facilities are typically easier to access than BH offices partly because of the frequency of wellness visits in pediatric primary care (Hansel et al., 2017; Walter et al., 2019). Integrating BH into primary care can also lessen the stigma associated with accessing BH care (Hansel et al., 2017). BH and physical health separation lead to higher healthcare costs and poor patient outcomes (Habeger & Venable, 2017). Talmi et al. (2016) and Rodriguez et al. (2019) note that integration of BH into pediatric primary care can help meet the needs of pediatric patients and their families and improve health disparities in diverse populations, as well as decrease barriers to access BH treatment (Rodriguez et al., 2019). In addition, rural communities have increased barriers to BH treatment compared to urban communities, resulting in more time away from work for parents, greater difficulty navigating the healthcare system (i.e., referral from primary care to BHPs), and long wait times (Habeger & Veneable, 2017). These barriers and gaps can lead to inadequate treatment, including difficulty managing and treating BH and social, educational, occupational, and economic deficits (Walter et al., 2019). Provider satisfaction and burnout are additional concerns in managing BH in primary care. Many pediatric PCPs feel an increased burden with managing BH in primary care, including stress with a 5 disconnected system, lack of resources for BH diagnoses, and decreased communication between pediatric PCPs and BHPs, which can lead to dissatisfaction and difficulty co-managing pediatric BH diagnoses (Rodriguez et al., 2019; Schlesinger, 2017). Rodriquez et al. (2019) found increased provider satisfaction and improved job satisfaction after BH integration. Several studies have demonstrated improved patient care with the integration of BH into pediatric primary care. Integration initiatives have lowered the barriers to BH management and have improved BH outcomes. Walter et al. (2019) discussed their integration program at a large pediatric primary care network. This program was launched in 2013 and evaluated data over five years, after which the authors found a 19% decrease in emergency room costs associated with BH spending. Rodriguez et al. (2019) found that access to BH care increased because of integration. They also found that integration provided more patient-centered care for BH, with greater convenience for the patients and their families (Rodriguez et al., 2019). Children's Community Pediatric Behavioral Health Service developed a BH integration program. They found that access to BH services was more timely, efficient, and cost-effective than primary care not integrated with BH (Schlesinger, 2017). In a cross-sectional study using electronic health record (EHR) data, Talmi et al. (2016) determined that most BH consults were provided during well-child visits when PCP and parents discussed BH concerns. A study by Rodriguez et al. (2019) demonstrated that pediatric PCPs noted more access to BH service after integration, including increased patientcentered care and more engagement in treatment because the patients and families already had an established rapport with the PCP. Several elements must be considered when integrating BH into pediatric primary care. One integration model has discussed the importance of leadership buy-in (Schlesinger, 2017). Another has reviewed the importance of funding and financial sustainability (Talmi et al., 2016). Owens et al. (2021) discussed the importance of having a provider champion to ensure buy-in from other providers in the practice. Another challenge to consider is accommodating the volume of pediatric patients who may need a BH intervention: many patients often need BH care in the primary care setting, which could call for additional providers and office space (Lauerer et al., 2018). 6 Rationale BH integration aims to provide treatment and intervention when BH needs are initially identified (Hallas, 2018). Such early identification and intervention can help counteract future detrimental BH outcomes (Hallas, 2018). The Intercepting Behavioral Health (IBH) conceptual model supports this (see Figure 1). The fundamental concepts of this model are early identification of potential and current behavioral health problems in children and adolescents, followed by a behavioral intervention (Hallas, 2018). The model calls for a shift from the typical practice of observing and waiting to early recognition and implementation of evidence-based interventions (Hallas, 2018). Adverse outcomes may be mitigated by recognizing early concerns (including family history, behavior, observations, and current BH diagnoses), focused BH assessments, early BH interventions, and follow-up in primary care (Hallas, 2018). The IBH conceptual model illustrates the importance of having a foundation in primary care for BH evaluation and intervention and staging early interventions so that children and adolescents can develop the coping skills to grow into healthy adults (Hallas, 2018). The model uses the mathematical term intercept as an analogy. Intercept is the point "where a curve intersects an axis with positive and negative points above and below the x-axis" (Hallas, 2018, p. 7). When applying the IBH conceptual model to pediatric behaviors, the mathematical intercept and interventions are used to influence the direction of those behaviors by creating more positive behaviors (Hallas, 2018). In addition, the IBH conceptual model uses a football analogy depicted in the model's graphic. The pediatric patient corresponds to a football team's quarterback (Hallas, 2018). Adverse patient behaviors will be intercepted by a child's support team (parents, teachers, PCPs, BHPs), early interventions will occur, and the adverse behaviors will evolve into positive outcomes (Hallas, 2018). Hallas (2018) discussed ways BH can be integrated into pediatric primary care, including having BH services in primary care in one setting where collaboration can occur between BHPs and pediatric PCPs to maximize the physical, behavioral, social, and emotional well-being of pediatric patients. This project aimed to provide this collaboration between pediatric primary care and BH to identify BH concerns early and timely intervention to improve outcomes in the pediatric population. This model 7 acknowledges the breadth of BH concerns in pediatric patients and the importance of early identification and treatment of these patients. By having BH support in primary care, pediatric primary care providers will be less burdened. They thus will be more satisfied with the management of pediatric BH concerns and diagnoses. Specific Aims The purpose of this project was twofold: to improve provider satisfaction with the management of BH and to improve BH outcomes in the pediatric population in a private pediatric primary care clinic in Twin Falls, ID. There were several objectives for this project. The initial objective was to assess provider satisfaction with the current management of pediatric BH diagnoses and assess baseline BH outcomes in the pediatric population. Additional objectives included developing and implementing a practice-specific process for BH integration in a private pediatric primary care office. Identifying cultural barriers to BH management in pediatric patients was also an objective of this project. The final objective was to evaluate provider satisfaction and BH outcomes after the intervention. Methods Context The intervention occurred in a private pediatric primary care clinic, Frontier Pediatric Partners, PLCC, in rural Twin Falls, ID. Twin Falls has a population of 47,000 and serves as the hub for many surrounding rural communities (Demographic Information, 2022). Twin Falls and the surrounding communities have a population of about 100,000 (Twin Falls, Idaho, n.d.). The primary industry of Twin Falls is agribusiness, including food processing, dairy, and manufacturing (Twin Falls, Idaho, n.d.). Twin Falls’ ethnicity mix comprises 69% White, 28% Hispanic, 1% Black, and 1% Asian (Demographic Information, 2022). Twin Falls is also home to a small community college. The primary care clinic, Frontier Pediatric Partners, PLCC, had six pediatric providers, including two nurse practitioners (NPs), one physician's assistant (PA), and three medical doctors (MDs). Most patients are Non-Hispanic or Hispanic, but the clinic serves a smaller number of refugee patients and children in foster care. There are also transgender adolescents and pediatric patients with learning 8 disabilities. Upper, middle, and lower socioeconomic backgrounds are all represented. Total patient visits from September 2020 to September 2021 were 4618, and 23% of all visits were BH visits. The payer mix is approximately 56% private payers, 45% Medicaid, and 1% uninsured. A Chance 4 Change, LLC is a private BH clinic with seven BHPs with varying degrees and credentials. Before the intervention, pediatric patients that needed BH services were referred to outside clinics, including A Chance 4 Change, LLC. Intervention The first step of the intervention was to identify provider satisfaction (both PCP and BH) with the management of BH diagnoses in pediatric patients. This included pediatric PCPs at Frontier Pediatric Partners, PLCC, and BHPs at A Chance 4 Change, LLC. In addition, baseline data were obtained regarding patient depression and anxiety scores by running structured data reports on the Patient Health Questionnaire-9 (PHQ9) and Generalized Anxiety Disorder-7 (GAD7) questionnaires from the clinic's electronic medical record (EMR) to provide a baseline frequency and percentage of the depression and anxiety scores of the patients. The second step of the intervention was developing program details for integrating a BH provider at Frontier Pediatric Partners, PLLC, in collaboration with A Chance 4 Change, LLC. These details included a professional agreement, EMR access for the LSW, and a process for referral to the LSW from the pediatric providers. The LSW was given access to Frontier Pediatric Partners, PLCC, EMR so that the PCP could send the referral directly to the LSW. The LSW could then review the patient's past medical history and document the visit between the LSW and the patient and their family. A diagram was developed to illustrate this collaboration process between the PCPs and BHP (see Figure 2). An education and training program was provided to the PCPs and the BHPs. The third step of the intervention consisted of A Chance 4 Change, LLC, providing an onsite LSW at Frontier Pediatric Partners, PLCC, Monday through Friday 11 am-4 pm to provide immediate assistance and intervention for pediatric patients and their families' BH concerns. The intervention lasted one month. The BH needs included crisis management, suicidal ideations, coordination of care for 9 inpatient BH treatment, initial intake for therapy, coordination of initial therapy for same-day therapy if needed, and ongoing scheduling for subsequent therapy. The intervention helped bridge the gap between PCP and BH referral when the need was identified. The final step of the intervention consisted of evaluating provider satisfaction (PCP and BH) after the intervention by comparing pre-intervention surveys with post-intervention surveys using inferential statistical analysis. Descriptive statistics were also used for questions in the pre-and post-intervention that could not be analyzed using the Wilcoxon signed-rank test. Thematic analysis was used on qualitative data (Miles et al., 2019). Post-implementation data regarding patient depression and anxiety scores were obtained by running structured data reports from results from PHQ9 and GAD7 from the clinic's EMR. Changes in PHQ9 and GAD7 scores were examined by comparing post-intervention scores to preintervention scores. Study of the Intervention Surveys were used to establish provider satisfaction (PCP and BH) with the management of BH diagnoses. All PCP and BHPs who were a part of the intervention received a pre and post-intervention survey. The surveys were collected and managed using the Research Electronic Data Capture (REDCap) tools hosted by the University of Utah (Harris et al., 2019). Both surveys collected provider demographics. The post-survey included all questions from the pre-survey with additional questions regarding the intervention. Structured data reports were run from Frontier Pediatric Partners, PLLC, EMR on PHQ9 and GAD7 scores pre-intervention and post-intervention. Other EMR data included top BH diagnoses (see Appendix 1) in the private pediatric clinic. The number of BH visit types before and during the intervention was obtained. Before the project, providers expressed dissatisfaction with the management of pediatric BH. We assumed that any improvement in provider satisfaction with the management of pediatric BH diagnoses and any improvement in PHQ9 and GAD7 scores were related to the intervention. No comparison groups were used, and this study was not blinded. 10 Measures Three measures were used to evaluate the quality improvement intervention. First, a survey was developed (see Appendix 2) specifically for this intervention. A test group and Dr. Lisa Taylor-Swanson reviewed the survey tool for validation. The first survey question asked if the respondent was a pediatric PCP or a BHP. This answer then directed the respondent to the appropriate subsequent survey based on the individual's role. Each survey (specific to PCP or BHP) consisted of 14 questions: four demographic questions, six questions regarding satisfaction with BH management, and four open-ended questions. The six questions using the Likert-type scale had five responses from which to choose. Four questions from the pre-intervention and post-intervention (see Appendix 2) survey were statistically analyzed using the Wilcoxon signed-rank test to show statistical significance after the intervention was implemented. The remaining questions that could not be analyzed using the Wilcoxon signed-rank test were evaluated using descriptive statistics. We reviewed and categorized the qualitative responses using thematic analysis (Miles et al., 2019). The second and third measures used in the quality improvement intervention were the PHQ9 and GAD7, respectively, measuring depression and anxiety. They were administered at baseline and after completion of the intervention. The questionnaires have proven validity and reliability and are often used in primary care and BH. The PHQ9 had nine items with answers ranging from 0-3 (0-not at all, 1-several days, 2-more than half the days, 3-nearly every day). Two additional questions for the PHQ9 inquired about suicidal attempts and feeling depressed most days. The GAD7 questionnaire includes seven questions with answers ranging from 0-3 (0-not at all, 1-several days, 2-more than half the days, 3-nearly every day). Both questionnaires are scored by calculating a total score. High scores equate to increased depression PHQ9 and anxiety GAD7. Cutoff scores for PHQ9 and GAD7 screenings are 10, indicating severe depression and/or anxiety, and further evaluation is indicated with a score of 10 or above. Given how the data were extracted from the EMR, a comparison of numbers and percentages was completed for analysis. The accuracy of the data was ensured by collaboration among the project lead, project chair, content expert, and statistician. 11 The project provided an intervention to address the problem, but there were barriers to the intervention. A primary barrier included the possible loss of revenue for the licensed clinical social worker (LCSW) and A Chance 4 Change, LLC. Initially, the project’s plan was to utilize an LCSW, and services offered would be billed for. However, due to abrupt changes with Medicaid reimbursement for crisis care, an LCSW could not be used, as it would take away from the LCSW’s ability to bill for services at A Chance 4 Change, LLC. Therefore, an LSW (who cannot bill for services) was used. An LSW holds a bachelor's degree in social work and can practice general social work such as assessment, case management, referrals, supportive counseling, and advocacy. An LCSW holds a master's degree in social work, has completed 3,000 clinical hours under supervision from an already licensed LCSW, and completed a clinical licensure process through the Idaho Department of Occupational and Professional Licenses. LCSWs can administer clinical assessments, psychotherapy, psychosocial interventions, consultation, family therapy, and group therapy. Because LCSWs hold specialized knowledge about social work in a clinical application, they can provide treatment independently or supervise a Licensed Master's Social Worker (LMSW). An LMSW must complete 3,000 supervised clinical hours and pass a clinical licensure exam before becoming an LCSW. The benefits of the intervention, even though an LSW was used instead of an LCSW, had the potential to improve provider satisfaction with the management of BH diagnoses and improve patient outcomes. This project would not have been possible without the collaboration of A Chance 4 Change, LLC, and their willingness to provide the LSW at no cost for the duration of the intervention. Still, this collaboration was challenging because of the LSW’s time required at Frontier Pediatric Partners, PLLC. The BHP was in a new location with new colleagues, but this did not prove to be a barrier. When the LSW was not with patients at Frontier Pediatric Partners, PLLC, she continued to work on A Chance 4 Change, LLC's responsibilities remotely (from Frontier Pediatric Partners, PLLC). The adjustment was smooth, and Frontier Pediatric Partners, PLCC providers, and staff were welcoming and pleased to offer this service at the pediatric clinic. The intervention demonstrated the need for an in-house LCSW. If the 12 BHP is an LCSW, services can be billed for, and therefore, the future cost could be offset. Frontier Pediatric Partners, PLLC, intends to continue this project in the future with in-house, employed BHPs. Analysis Descriptive statistics were used to describe the provider study sample, both PCPs, and BHPs, as well as the frequency of BH visit types and percentage of male children versus female children in total BH visits. PHQ9 and GAD7 responses were analyzed with descriptive statistics for frequency and percentage pre-intervention and post-intervention. Pre-intervention and post-intervention surveys were analyzed using change statistics to determine whether a statistically significant change occurred. The Wilcoxon signed-rank test measured the change between pre-intervention and post-intervention scores among the providers who took the survey to demonstrate improved provider satisfaction in managing pediatric BH diagnoses after the intervention was implemented. In addition, descriptive statistics were used to analyze questions that could not be analyzed with inferential statistics. A thematic analysis of provider open-ended survey questions, read the answers word for word, coded them, and then categorized and summarized the coded data (Miles et al., 2019). Ethical Considerations This study was quality improvement in nature and not subject to institutional review board (IRB) oversight. There were no conflicts of interest concerning this study. Although the pediatric population is vulnerable, an ethics review indicated the study was exempt. Results A survey conducted before the intervention included provider demographics and questions regarding satisfaction with BH management. Before the intervention, baseline data was obtained on patient visits and depression and anxiety scores. A post-intervention survey included all pre-intervention survey questions and additional questions regarding feedback for the intervention. Post-intervention data was obtained on patient visits and depression and anxiety scores. The pre-intervention and post- 13 intervention surveys were tailored to each provider discipline (PCP and BHP). The only modification made to the intervention during the project was the utilization of an LSW instead of an LCSW. Thirteen total providers completed the pre and post-intervention survey. Six providers were pediatric PCPs, and seven were BHPs, with various degrees and credentials for both groups (see Table 1). The pediatric PCPs included three medical doctors (MDs) and three advanced practice providers (APPs), including two nurse practitioners (NPs) and one physician's assistant (PA). The BHPs included three LMSWs, two LCSWs, one LSW, and one licensed professional counselor (LPC). There were n=621 (23%) BH visit types three months before the intervention. There were n=618 (28%) BH visit types during the month of the intervention and two months post-intervention. Thus, there was an increase in BH visits by 5% during and after the intervention (see Table 2). During the intervention, PCPs referred 35 patients to the onsite LSW. Twenty-five (71.43 %) pediatric patients, on the same day as the referral, reducing the gap from PCP referral to initial BH contact. The LSW contacted the remaining ten patients after the actual date of appointment with the pediatric PCPs. Twenty-three (65.71%) of patients who had referrals placed during the intervention had the entire intake completed by the LSW in preparation for scheduling ongoing counseling/therapy services. Three (8.6%) of patients with onsite BH referrals were scheduled for continued services during the month of the intervention. Two of those patients continue to be actively seen for therapy, while one was seen and discharged because they were doing so well after therapy. This scheduling was based on patient acuity after the LSW evaluated the patients to determine those with the greatest need. The remaining patients were scheduled or placed on a waiting list for continued services based on need and schedule availability at A Chance 4 Change, LLC. There are 11 patients still on the waiting list waiting for services, and six were able to get in sooner at another BH facility. The remaining patients did not return calls to move forward with services or declined additional services after the initial same-day contact. The LSW was also able to coordinate a safety plan and help with the preparation for an inpatient admission when a pediatric patient presented with active suicidal ideations, which allowed the pediatric PCP to continue with other patient visits. 14 Analysis of the qualitative data yielded findings of interest. Although the provider comments have been edited for brevity, the original meaning of each participant's quote remains intact. When asked, "What cultural barriers exist in treating pediatric patients with behavioral health diagnoses?" the pediatric PCP and BHP responses included the following themes: family, language, providers, and mental health stigma. These themes overlap between both provider groups (see Table 3). Then both provider groups were asked, "What is your perception of ease of access to behavioral health care for pediatric patients in need?". Main themes overlap, but there were also differences in the details within these themes depending on provider type. For example, pediatric PCPs listed themes regarding poor access, referral process, communication, and family barriers. Behavioral health providers' responses included themes of limited access and family barriers (see Table 4). Pediatric PCPs were asked, "What is the most difficult part of managing behavioral health in pediatric primary care?". Behavioral health providers were asked, "What is the most difficult part of managing pediatric behavioral health?". Themes for pediatric PCPs included referrals, schedules, providers, and patients. Themes for BHPs included schedules, providers, patients, and family. The responses of the two groups were similar. However, the responses indicate that each provider group faces difference challenging in managing BH (see Table 5). The intervention resulted in clinically notable findings, particularly pediatric PCPs responses. Before the intervention, none of the PCPs responded that it was easy or very easy to coordinate BH services such as counseling. After the intervention, n=3 (50%) reported that it was easy or very easy to coordinate BH services such as counseling. When pediatric PCPs were asked, "How effective does counseling/therapy seem to be for pediatric patients with BH diagnoses?" we found a 16.67% increase in PCPs responding that counseling was effective or very effective (see Table 6). Before the intervention, only n=1 (16.66%) PCP noted that managing BH in their practice was not burdensome or was slightly burdensome. After the intervention, n=4 (66.67%) reported not burdensome or slightly burdensome. In addition, in the pre-survey, n=0 (0%) PCPs answered that it was less than one or two weeks for the counseling intake to be completed and services started. After the 15 intervention, n=2 (33.33%) noted that it took less than two weeks for intake to be completed and services started (see Table 7). When BHPs were asked before the intervention, "How easy is it to communicate with a patient's primary care provider regarding mutual patients?", n=2 (28.57%) providers said it was easy or very easy. After the intervention, n=3 said it was easy or very easy (a 14.29% increase). When the BHPs were asked, "How comfortable are you attempting communication with pediatric patients' primary care providers?", we found no change between pre-intervention and post-intervention scores. The table demonstrates pediatric PCP and BHP pre and post-survey answers directed to each discipline separately (see Table 8). When asked, "How responsive is a pediatric patients' primary care provider when attempting communication regarding mutual patients?", prior to the intervention, n= (28.58 %) of BHPs said responsive or very responsive. After the intervention, n=3 (42.86%) said responsive or very responsive. Pediatric PCPs were asked, "What is the breadth and level of quality available to behavioral health counseling/therapy in the community?". Although they answered that several counselors in the community were of high quality and responsive, many noted gaps in care, limited resources, and a lack of specialized services. Specialized services noted as limited in the community include play therapy, Eye Movement Desensitization and Reprocessing (EMDR), and Parent-Child Interactive Therapy (PCIT). Other respondents note such barriers, including long waiting lists, limited acceptance of various insurance plans, and high turnover. Behavioral health providers were asked, "What do you view as the barriers to treatment of pediatric behavioral health patients?". Their responses included parental participation, availability in schedules for both patients/families and providers alike, attendance at appointments, level of engagement during therapy sessions, and the amount of added work requirements for Medicaid patients. In addition, BHPs discussed that making initial contact with the parents was difficult, getting paperwork complete, a signed consent, and release of information was problematic. They also noted that there was difficulty with communication and that collaboration between providers (PCP and BHP) would enhance patient care. 16 Other responses regarding BHPs’ perception of barriers to treatment included that BH services rely heavily on "adult action, desire, and level of commitment," and if the parents are not engaged and do not desire the treatment for their children, then the children are not able to get the help that they need. In addition, when there is a history of failed adult relationships and broken trust, pediatric patients often do not trust BHPs, which becomes a barrier to treatment. Many children, especially teens, are hesitant to engage in counseling because of the stigma regarding BH. BHPs noted other barriers to treatment, including transportation issues, especially for lower socioeconomic families, limited treatment hours, financial barriers, and the steps needed to get service started. Many of the same questions were asked to assess for differences after the intervention in the postintervention survey. In addition, we added new questions. When asked, "What cultural barriers exist in treating pediatric patients with behavioral health diagnoses?", there were no changes in provider groups. However, they mentioned two additional barriers: religion and gender. When asked, "What is your perception of ease of access to behavioral healthcare for pediatric patients in need?" pediatric PCPs noted that although access is still difficult, and barriers remain, having an LSW at Frontier Pediatric Partners, PLLC, made it easier to coordinate care and complete intake, and that ease of access is improving. They noted an improvement in "drop out" and more family reassurance that the BH office was engaged in their children's care. They also noted that BH offices continued to be backlogged, with limited access and long wait times. Other pediatric PCPs said that it remained challenging. Behavioral health providers did not note changes post-intervention but reported that the system remains largely overwhelmed. Post-intervention, pediatric PCPs were asked, "What is the breadth and level of quality available for behavioral health treatment like counseling/therapy in the community?". There was no change in responses from pre-intervention to post-intervention. When asked post-intervention, "What is the most difficult part in managing pediatric behavioral health?" pediatric PCPs noted the long wait times for appointments continued to be a problem. Still, they observed improvement with the LSW in the office and that services were more streamlined. Responses 17 included that there continued not to be enough BHPs to meet the needs of pediatric patients in the community. Other difficulties for pediatric PCPs remained the same in the post-intervention survey compared to the pre-intervention survey. Behavioral health providers had no change, except one provider noted that it was challenging to work with parents, pediatricians, and schools to get everyone on the same page for the child. When BHPs were asked, "What do you view as the barriers to treatment of pediatric behavioral health patients?" we found no change, except the addition of developmental delays in children as a barrier. Post-intervention feedback was also obtained from both provider groups regarding the quality improvement project, including positive and or negative feedback if the project was helpful to the providers' clinical practice and recommendations for project improvement (see Table 9). Both PCPs and BHPs answered four questions on the pre and post-survey that were statistically analyzed using the Wilcoxon signed-rank test using an online calculator. The Wilcoxon signed-rank test is a nonparametric test which was why this test was used for statistical analysis (Wilcoxon Signed-Rank Test Calculator, 2022). The significance level was set at .05, and it was two-tailed. Both surveys had 13 respondents, with some giving the same answer in the pre and post-survey. When a respondent's difference score was zero (they had the same score before and after the intervention), the Wilcoxon signed-rank test disregarded that respondent's result. Thus, the sample size was reduced. For the question, "How satisfied are you with managing pediatric behavioral health diagnoses?" the result was Z = 0.9102. In addition, it was not possible to calculate an accurate p-value. The W value was found to be 11.5. The critical value of W at N=8 (p < .05) was 3. The result not significant at p < .05. The second question that was analyzed with the Wilcoxon signed-rank test was "How burdensome is it to manage behavioral health in your practice?" with a result Z = 1.427 and the p-value = .15272. The W value was found to be 13.5. The critical value of W at N=10 (p < .05) was 8. The result not significant at p < .05. The next question that was analyzed with the Wilcoxon signed-rank test was, "When a patient is referred/you refer a patient for counseling or other behavioral health services/interventions, how long does the intake take to be completed and services started?". After this question was analyzed, the Z value 18 was -0.5331. It was not possible to calculate an accurate p-value. The W value was found to be 18. The critical value of W at N=9 (p < .05) was 5. The result not significant at p < .05. The final question that was analyzed with the Wilcoxon Signed-Rank test was "how difficult is it to treat pediatric patients with behavioral health diagnoses if cultural barriers exist?". After the question was analyzed, the Z value was zero. It was not possible to calculate an accurate p-value. The W value was found to be 22.5. The critical value of W at N=9 (p < .05) was 5. The result not significant at p < .05. Table seven shows the descriptive statistics for these four questions. Depression (PHQ9) and anxiety (GAD7) scores were analyzed pre and post-intervention. Tables demonstrate the pre and post PHQ9 and GAD7 scores. There was no clinically significant change found between pre-intervention and post-intervention scores (see Table 10 and Table 11). Contextual elements that interacted with the intervention include differences in providers pediatric PCPs versus BHPs. In addition, the BH provider onsite was an LSW, not an LCSW, and thus was not able to perform psychotherapy onsite. It was observed that the intervention was more beneficial to the pediatric PCPs than the BHPs. The duration of the intervention was also a contextual element that interacted with the intervention – a more extended duration would have produced better outcomes. Data collection for the PHQ9 and GAD7 was cumbersome and did not show improvement in BH outcomes – this could be partly due to the collection process. No significant unexpected benefits, problems, failures, or costs were associated with the intervention. There was not any missing data from the surveys. All participants completed the pre and post-survey. However, some respondents had no change in their responses, which yielded smaller sample sizes for the Wilcoxon signed-rank test. Discussion Summary While the reported findings were not statistically significant, they were clinically relevant. Many cultural barriers were identified, including family, language, providers, and stigma. Many BH access themes were mentioned, including limited access, lack of BH professionals, long wait times for appointments, and fragmented communication between pediatric PCPs and BHPs. There were many 19 barriers and frustrations found in treating pediatric patients with BH diagnoses from both the pediatric PCP and BHP perspectives, including the referral process, lack of availability of appointments for BH treatment, difficulty finding BHPs who can manage complex patients, and compliance with treatment recommendations. The intervention improved pediatric patients' initial contact with BHPs, as the LSW saw 25 patients on the day of referral. The project had two specific aims: improving provider satisfaction with BH management and improving BH outcomes in the pediatric population. There was an improvement in coordination of BH services, as demonstrated by pediatric PCPs ' post-survey responses. The intervention also was found to improve communication and collaboration between the two provider groups. Even though it was only one month, this intervention improved provider satisfaction and left providers wanting more permanent integration of services. While pediatric PCPs reported, on average, more satisfaction than BHPs, this is not surprising given the continued shortage of qualified BHPs and the minimal change to their workload. Unfortunately, the intervention was not long enough to show significant improvement in BH outcomes in pediatric patients. It was anticipated that after the intervention, there would be more statistically significant changes, but given the length of the intervention this was not unexpected. Strengths of the project include improving access for patients and their families to BH services and decreasing the gap between PCP referral to BH services and actual contact with BHPs. It was easier for patients to have their initial assessment and intake completed. Patients and their families were receptive to this initial visit with the LSW at the time of referral. In addition, increased collaboration between PCP and BHP was observed, and this collaboration continues between the two offices, Frontier Pediatric Partners, PLCC, and A Chance 4 Change, LCC. Interpretation We anticipated having statistically significant outcomes, but the duration of the intervention was not long enough to demonstrate statistical significance. However, the qualitative findings did align with the expected results. Themes included barriers to treatment, challenges in managing pediatric BH 20 diagnoses, and access to BH services. The intervention also showed increased satisfaction in the management of pediatric BH, particularly among the pediatric PCPs. Findings from this project echo findings from other publications and continue to demonstrate the need for BH integration in primary care. Integration of BH into primary care increases access to BH services when it is often difficult due to shortages of BHPs, and gaps in the referral process between PCPs and BHPs (Walters et al., 2019). Schlesinger (2017) discusses that BH integration into pediatric primary care increases timely access to BH services. Habeger and Venable (2017) found that an integrated care model between BH and primary care expands the availability of services, and providers' ability to triage patients to reduce wait times, and provides more timely initial screening. The project had a clinically meaningful impact on people and systems. We anticipate that if the project were to continue, we would find a more substantial impact and a further decrease in the barriers and gaps in BH care in the pediatric population. The project significantly impacted pediatric PCPs, as it did help with their BH workload. In addition, it provided timely contact for pediatric patients with BH concerns to a BHP. Differences between observed and anticipated outcomes result primarily from the duration of the intervention and the need to utilize an LSW instead of an LCSW in the intervention. We would expect statistically significant findings if the intervention lasted six months to one year. In addition, if an LCSW had been able to be onsite instead of an LSW, additional services could have been offered, including onsite scheduled therapy during the duration of the intervention. The intervention could not have been implemented without the collaboration of Frontier Pediatric Partners, PLCC with A Chance 4 Change, LLC. The LSW was a current employee of A Chance 4 Change, LCC, thus incurred no additional costs for this project. The LSW was able to use her free time at Frontier Pediatric Partners, PLCC, to complete most of her regular day-to-day duties. However, with the further continuation of this intervention, there would be a cost associated with hiring an LCSW. The trade-off would be an onsite LCSW who could provide all the services during the intervention and 21 services that could be billed for, including psychotherapy, thus offsetting the cost of hiring an LCSW for Frontier Pediatric Partners, PLCC. Limitations The project did have notable limitations. Unfortunately, the data collection for the PHQ9 and GAD7 proved to be a significant limitation in assessing BH outcomes in the pediatric population – primarily how the EMR data were extracted. The PHQ9 and GAD7 questionnaires are obtained from the pediatric patient at the time of visit. However, each question is not linked to the specific patient to get a total of the individual patient scores. Each question is part of a structured data field, and extracting each question had to be done individually and for each response. In addition, because templates were changed intermittently for visit types, we had multiple structured data fields for each question, which likely contributed to the increase in the number of PHQ9 and GAD7 responses from pre and post-intervention. Ideally, questionnaire data would be extracted by patient to compare the individual's total score to a previous score for that patient. In addition, it was difficult and time-consuming to pull every question and each possible response individually. Another limitation was the duration of the intervention. An intervention like this should last at least six months to one year to show statistical significance. Unfortunately, the intervention could only take one month, given time constraints and staffing availability. We believe we would have added significant changes in responses with a longer intervention duration beyond the qualitative findings. The final major limitation includes the utilization of an LSW instead of an LCSW. Initially, the intervention plan was to utilize an LCSW. However, due to abrupt changes in Medicaid billing, it was not beneficial for A Chance 4 Change, LCC to have an LCSW onsite at Frontier Pediatric Partners, PLCC without the ability to bill for those services, which would have decreased access for their current patients, and future patients in need of therapy at their location. While the LSW was able to provide assessment, case management, intake, and supportive counseling, these services are not billable under the licensure of the LSW. If there had been an LCSW, additional services such as psychotherapy, family therapy, and even group therapy could have been offered. 22 Limited internal validity could include some providers not needing to use the services of the LSW given their patient panel or their amount of BH patients. In addition, the BHP group likely did not see a significant benefit, as they continued to have a high workload, and this intervention did not have the same impact as it did on the pediatric PCP workload. In addition, the small sample size itself was a limitation. Conclusion The project included the placement of an LSW in a pediatric primary care clinic for one month to assist in referrals, patient intake, crisis intervention, and case management. Twenty-five pediatric patients were able to meet on the same day as the referral with the LSW, which helped decrease the gap between pediatric PCP referral and BHP initial contact. The participants in the project found this placement worthwhile and provided positive feedback about the project. The placement of the LSW was more helpful to the pediatric PCP group than the BHP group. The project is sustainable, mainly if the pediatric primary care office hires an LCSW to provide these services and provides BH therapies such as counseling, psychotherapy, family therapy, and group therapy. It can easily be applied to other settings; however, the integration will be most impactful with a full-time LCSW to provide ongoing BH integration into pediatric primary care. Frontier Pediatric Partners, PLCC's management, and the owner team have discussed the importance of BH integration in pediatric primary care. There continues to be a shortage of BHPs and services offered. Having an LCSW can provide ongoing support and treatment in collaboration with pediatric primary care to help bridge the gap in BH treatment, decrease barriers to access BH services, improve provider satisfaction with managing pediatric BH patients, and ideally improve BH outcomes in pediatric BH patients. A further step of this intervention is to hire a full-time LCSW. Frontier Pediatric Partners, PLCC, intends to proceed with hiring an LCSW for BH integration in the next three to six months. 23 Acknowledgments We want to thank Melissa Osen, LCSW, owner and founder of A Chance 4 Change, LLC; Rachel Knight, LSW, who was the onsite LSW at Frontier Pediatric Partners, PLCC for the entire intervention; Dr. Jared Hansen, MD, for training on the initial EMR data collection; and Christine Pickett, editor. We would also like to acknowledge all the providers and staff at Frontier Pediatric Partners, PLCC, and A Chance 4 Change, LCC, for participating in this project. 24 References Demographic Information. (2022, January 20). Twin Falls Area Chamber of Commerce. https://twinfallschamber.com/live/demographic-information/ Habeger, A. D., & Venable, V. M. (2017). Supporting families through the application of a rural pediatric integrated care model. Journal of Family Social Work, 21(3), 214–226. https://doi.org/10.1080/10522158.2017.1342468 Hallas, D. (2018). Behavioral pediatric healthcare for nurse practitioners: A growth and developmental approach to intercepting abnormal behaviors (1st ed.). Springer Publishing Company. Hansel, T., Rohrer, G., Osofsky, J., Osofsky, H., Arthur, E., & Barker, C. (2017). Integration of mental and behavioral health in pediatric health care clinics. Journal of Public Health Management and Practice, 23, S19–S24. https://doi.org/10.1097/phh.0000000000000649 Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O'Neal, L., McLeod, L., Delacqua, G., Delacqua, F., Kirby, J., & Duda, S. N. (2019). The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics, 95, 103208. https://doi.org/10.1016/j.jbi.2019.103208 Herbst, R. B., McClure, J. M., Ammerman, R. T., Stark, L. J., Kahn, R. S., Mansour, M. E., & Burkhardt, M. C. (2020). Four innovations: A robust integrated behavioral health program in pediatric primary care. Families, Systems, & Health, 38(4), 450–463. https://doi.org/10.1037/fsh0000537 Kazak, A. E., Nash, J. M., Hiroto, K., & Kaslow, N. J. (2017). Psychologists in patient-centered medical homes (PCMHs): Roles, evidence, opportunities, and challenges. American Psychologist, 72(1), 1–12. https://doi.org/10.1037/a0040382 25 Lauerer, J. A., Marenakos, K. G., Gaffney, K., Ketron, C., & Huncik, K. (2018). Integrating behavioral health in the pediatric medical home. Journal of Child and Adolescent Psychiatric Nursing, 31(1), 39–42. https://doi.org/10.1111/jcap.12195 Miles, M. B., Huberman, M. A., & Saldana, J. (2019). Qualitative Data Analysis: A Methods Sourcebook (4th ed.). SAGE Publications, Inc. Okafor, M., Ede, V., Kinuthia, R., & Satcher, D. (2018). Explication of a behavioral health-primary care integration learning collaborative and its quality improvement implications. Community Mental Health Journal, 54(8), 1109–1115. https://doi.org/10.1007/s10597-017-0230-8 Owens, C. R., Haskett, M. E., Monroe, R. T., & Dow-Smith, C. (2021). Integrating behavioral health care into an urban hospital-based pediatric primary care setting. Journal of Health Care for the Poor and Underserved, 32(1), 179–190. https://doi.org/10.1353/hpu.2021.0017 Rodríguez, E. M., Gulbas, L. E., George-Jones, J., Leija, A., Burrows, D., & Neavel, C. (2018). Interdisciplinary perspectives on an integrated behavioral health model of psychiatry in pediatric primary care: A community-based participatory research study. Community Mental Health Journal, 55(4), 569–577. https://doi.org/10.1007/s10597-018-0330-0 Schlesinger, A. B. (2017). Behavioral health integration in large multi-group pediatric practice. Current Psychiatry Reports, 19(3). https://doi.org/10.1007/s11920-017-0770-1 Talmi, A., Muther, E. F., Margolis, K., Buchholz, M., Asherin, R., & Bunik, M. (2016). The scope of behavioral health integration in a pediatric primary care setting. Journal of Pediatric Psychology, 41(10), 1120–1132. https://doi.org/10.1093/jpepsy/jsw065 Twin Falls, Idaho. (n.d.). Southern Idaho Economic Development. https://www.southernidaho.org/twinfalls.html 26 Walter, H. J., Vernacchio, L., Trudell, E. K., Bromberg, J., Goodman, E., Barton, J., Young, G. J., DeMaso, D. R., & Focht, G. (2019). Five-year outcomes of behavioral health integration in pediatric primary care. Pediatrics, 144(1), e20183243. https://doi.org/10.1542/peds.2018-3243 Wilcoxon Signed-Rank Test Calculator. (2022). Social Science Statistics. https://www.socscistatistics.com/tests/signedranks/default2.aspx 27 Tables and Figures Table 1 Demographics Demographic Type of Provider MD DO NP PA LMSW LCSW LCPC LPC LSW Age 20-25 26-30 31-35 36-40 41-45 46-50 51-55 56-60 61-65 Gender Female Male Years in practice 0-5 6-10 11-15 16-20 21-25 26-30 31-35 N=13 Number (%) 3 (23.08%) 0 (0%) 2 (15.38%) 1 (7.69%) 3 (23.08%) 2 (15.38%) 0 (0%) 1 (7.69%) 1 (7.69%) Number (%) 0 (0%) 1 (7.69%) 2 (15.38%) 5 (38.46%) 2 (15.38%) 2 (15.38%) 0 (0%) 1 (7.69%) 0 (0%) Number (%) 9 (69.23%) 4 (30.77%) Number (%) 5 (38.46%) 2 (15.38%) 4 (30.77%) 1 (7.69%) 0 (0%) 1 (7.69%) 0 (0%) 28 Table 2 Visit Types Date Range 7/1/21-09/30/21 7/1/21-09/30/21 11/1/21-1/31/22 11/1/21-1/31/22 9/1/20-9/30/21 9/1/20-9/30/21 Visit Type: number (%) Total BH visit types All visit types BH visit types All visit types BH visit types All visit types 617 (23%) 2647 618 (28%) 2186 1075 (23%) 4618 Female Child 214 (17%) 1246 218 (22%) 1005 408 (18%) 2234 Male Child 403 (29%) 1401 400 (34%) 1181 667 (28%) 2384 29 Table 3 Thematic Analysis Cultural Barriers What cultural barriers exist in treating pediatric patients with behavioral health diagnoses? Family Difficulty with follow-through Lack of understanding of the process for parents The concept that a pediatric patient can just get over mental health concerns Financial Educational Family support Transportation Past parent experiences with their own behavioral health treatment Perception that counseling did not work for parents so why would it work for their children Non-white cultural perception of BH concerns BH diagnosis is a crutch to lean on and blame for other familial concerns Language Language barriers Lack of interpreters Inability to access the same resources as English-speaking patients/families Less resources in BH than primary care for non-English speaking patients Providers Finding providers that are culturally sensitive Predominately white population of behavioral health providers (BHP) Not accepting concepts of Western Medicine Behavioral Health Stigma Pediatric patients not being open with feelings because of BH stigmas Stigma against more severe BH diagnoses Financial Type of insurance Self-pay status 30 Table 4 Thematic Analysis Ease of Access What is your perception of ease of access to behavioral health care for pediatric patients in need? Access Pediatric Primary Care Provider Poor access Shortage of BHPs Limited number of BHPs to take complex BH patients Very difficult to get psychiatric services Limited resources Extended wait times for BH appointments Behavioral Health Care Provider Lack of available professionals Difficulty getting patients in for an appointment Long waiting lists to get BH services Inability to meet everyone's expectations and needs Poor access for patients without insurance, high deductible, and those who do not qualify for Medicaid Referral Process Pediatric Primary Care Provider Time between referral to BH to actual appointment is too long Communication Pediatric Primary Care Provider Communication between pediatric and BH offices is difficult and limited Family Pediatric Primary Care Provider Parents lacking proactive approach to getting counseling started Behavioral Health Care Provider Lack of engagement with parents/families to get service to children in need Parents denying services after referral from PCP 31 Table 5 Thematic Analysis Most Difficult Part of Managing BH What is the most difficult part of managing behavioral health in pediatric primary care/managing pediatric behavioral health? Referral Pediatric Primary Care Provider Inability to get patients to counseling due to length of time from referral to appointment Referral process Schedules Pediatric Primary Care Provider Time required for BH visits Getting services due to the lack of appointments available Behavioral Health Care Provider Finding after school appointments Provider Pediatric Primary Care Provider Difficulty finding available provider for complex patients Deciding the best type of specialized BH services for the patient Knowing the next medication option after multiple medication failures Behavioral Health Care Provider Client load Patient Pediatric Primary Care Provider Complexity of patients that need enhanced collaboration between PCP and BHP Behavioral Health Care Provider Attendance at appointments Following treatment recommendations Patient behaviors Family Behavioral Health Care Provider Parental participation and engagement 32 Table 6 Pediatric Primary Care Provider Survey Results (Pre and Post) Question Response How difficult is it to coordinate BH services like counseling when you identify a patient in need? Very difficult Difficult Neutral Easy Very easy How effective does counseling/therapy seem to be for pediatric patients with BH diagnoses? Not effective Fairly effective Neutral Effective Very effective Pre N (%) Post N (%) 2 (33.33%) 3 (50%) 1 (16.67%) 0 (0%) 0 (0%) N=6 0 (0%) 1 (16.67%) 1 (16.67%) 3 (50%) 1 (16.67%) N=6 1 (16.67%) 1 (16.67%) 1 (16.67%) 3 (50%) 0 (0%) N=6 0 (0%) 1 (16.67%) 0 (0%) 4 (66.67%) 1 (16.67%) N=6 33 Table 7 Behavioral Health Provider Survey Results (Pre and Post) Question Response How easy is it to communicate with a patient’s primary care provider regarding mutual patients? Very difficult Difficult Neutral Easy Very easy How comfortable are you attempting communication with pediatric patients' primary care providers? Very uncomfortable Uncomfortable Neutral Comfortable Very comfortable How responsive is a pediatric patients’ primary care provider when attempting communication regarding mutual patients? Very unresponsive Unresponsive Neutral Responsive Very responsive Pre N (%) 0 (0%) 1 (14.29%) 4 (57.14%) 2 (28.57%) 0 (0%) N=7 1 (14.29%) 0 (0%) 1 (14.29%) 4 (57.14%) 1 (14.29%) N=7 0 (0%) 2 (28.57%) 3 (42.86%) 1 (14.29%) 1 (14.29%) N=7 Post N (%) 0 (0%) 0 (0%) 4 (57.14%) 3 (42.86%) 0 (0%) N=7 0 (0%) 1 (14.29%) 1 (14.29%) 3 (42.86%) 2 (28.57%) N=7 0 (0%) 0 (0%) 4 (57.14%) 2 (28.57%) 1 (14.29%) N=7 34 Table 8 Pre and Post-survey Results for PCP and BH Questions Question Response PCP Pre N (%) 0 (0%) PCP Post N (%) 1 (16.67%) BH Pre N (%) 0 (0%) BH Post N (%) 0 (0%) How satisfied are you with managing pediatric BH diagnoses? Very unsatisfied Unsatisfied 4 (66.67%) 1 (16.67%) 0 (0%) 0 (0%) Neutral 1 (16.67%) 2 (33.33%) 4 (57.14%) 3 (42.86%) Satisfied 0 (0%) 2 (33.33%) 3 (42.86%) 4 (57.14%) 1 (16.67%) 0 (0%) 0 (0%) 0 (0%) N=6 N=6 N=7 N=7 0 (0%) 1 (16.67%) 0 (0%) 1 (14.29%) Slightly burdensome 1 (16.67%) 3 (50%) 2 (28.57%) 4 (57.14%) Neutral 2 (33.33%) 1 (16.67%) 3 (42.86%) 1 (14.29% Moderately burdensome 2 (33.33%) 1 (16.67%) 2 (28.57% 1 (14.29%) Very burdensome 1 (16.67%) 0 (0%) 0 (0%) 0 (0%) N=6 N=6 N=7 N=7 Less than one week 0 (0%) 1 (16.67%) 0 (0%) 0 (0%) Less than two weeks 0 (0%) 1 (16.67%) 2 (28.57%) 1 (14.29%) Less than three weeks 0 (0%) 0 (0%) 2 (28.57% 1 (14.29%) Less than 4 weeks 1 (16.67%) 2 (33.33%) 2 (28.57%) 3 (42.86%) Greater than 4 weeks 5 (83.33%) 2 (33.33%) 1 (14.29%) 2 (28.57%) N=6 N=6 N=7 N=7 Very difficult 3 (50%) 2 (33.33%) 1 (14.29%) 2 (28.57%) Difficult 3 (50%) 3 (50%) 3 (42.86% 3 (42.86%) Neutral 0 (0%) 1 (16.67%) 3 (42.86%) 2 (28.57%) Easy 0 (0%) 0 (0%) 0 (0%) 0 (0%) Very Easy 0 (0%) 0 (%) 0 (0%) 0 (%) N=6 N=6 N=7 N=7 Very satisfied How burdensome is it to manage BH in your practice? When you refer a patient for counseling or other BH services/interventions (or a patient is referred to you for services), how long does the intake take to be completed, and services started? How difficult is it to treat pediatric patients with BH diagnoses if cultural barriers exist? Not burdensome 35 Table 9 Thematic Analysis Post-Intervention Feedback from Pediatric PCPs and BHPs Have you heard positive feedback regarding the behavioral health integration project from patients? If yes, please elaborate. Pediatric Primary Care Provider It has been easier for patients to get intake completed. Patients have noted how great it was to have a social worker in the office at the time of need. It was comforting to talk with LSW and know the process was started. Behavioral Health Care Provider It has benefited ease of access and collaboration. Patients were appreciative to be able to take care of everything during one visit to the provider's office instead of waiting for a call on the referral or having to reach out. Some parents discussed how it was helpful for the therapists to have direct communication with the pediatricians. Patients/families could receive answers to their immediate questions and recommendations at the time of referral. Patients/families appreciated the time spent discussing their concerns and situation. Have you heard negative feedback regarding the behavioral health integration project from patients? If yes, please elaborate. Pediatric Primary Care Provider No Behavioral Health Care Provider Patients do not have time to do the initial client tasks, as they were already missing school or work to attend their appointment with their primary care provider. Some patients/families were reluctant to sign consents at time of initial intake. Some patients wanted time to think about it, and look at other options. Siblings who accompanied the patient to the primary care visit created distractions and limited the patient/caregiver's time and patience in getting everything taken care of. Was it helpful in your clinical practice to have a Licensed Social Worker onsite at Frontier Pediatrics? Pediatric Primary Care Provider It was very helpful. I was able to make several referrals. In addition, the LSW was able to take a significant load off me when there was an immediate crisis like suicidal ideations. Yes. The LSW could begin the intake process. But it was still going to be awhile to get them into ongoing services. Yes, it was great to have someone that could support the time-consuming "counseling" sides of patients' needs as well as improving the transition into counseling services. I felt we were more empowered and that having a social worker available was reassuring that we can help the patients and help them quickly. 36 Yes. It was helpful to have evaluation and give advice to families. Behavioral Health Care Provider It aided in the primary care providers' understanding of therapists' availability. Yes, it helps improve client care and reduce staff time. It helped to have someone in person who could sit with the family and discuss their concerns and ask questions of the youth. It decreased the number of phone calls and "phone tag" between the practice and client. Professional growth. Yes. The LSW was able to complete onsite screenings, fill out paperwork, get all paperwork signed, get patients scheduled, and help answer parent questions. Please provide any feedback that you may feel would have made this project more beneficial in your clinical practice. Pediatric Primary Care Provider It would have been great if the intervention could have lasted longer. It was very helpful and it would be great if the office could employ a LCSW for day to day interventions, as well as onsite counseling services. Having a full-time LCSW on site and not just an LSW would be great. Behavioral Health Care Provider Collaboration of care I feel it would be beneficial for Frontier to hire a social worker to maintain the work that was being done there but it was difficulty to have the lost time with an employee and have other employees take on additional work because of it. I truly feel like this project was at a disadvantage due to factors completely outside of the scope of anyone's control. The sheer volume of behavioral health needs in our community coupled with the number of behavioral health providers likely skewed the data. We had a number of Frontier-referred clients during execution of this project who ultimately withdrew from seeking services due to the extensive wait list. This was an amazing experience! We are so grateful that we were able to be a part of this project. The parents were participated were more engaged because they knew their pediatrician was more engaged. 37 Table 10 Depression (PHQ9) (Pre and Post) Question 1 - Feeling down, depressed or hopeless 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 2 - Little interest or pleasure doing things 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 3 - Trouble falling asleep or staying asleep or sleeping too much 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 4 - Poor appetite or overeating 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 5 - Feeling tired or having little energy 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 6 - Feeling bad about yourself or that you are a failure or have let yourself or your family down 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= Pre July - September Post November - January 358 (49%) 206 (28%) 102 (14%) 66 (9%) 732 601 (48%) 353 (28%) 170 (14%) 125 (10%) 1249 337 (48%) 183 (26%) 102 (15%) 77 (11%) 699 608 (49%) 334 (27%) 180 (14%) 127 (10%) 1249 300 (40%) 190 (25%) 130 (17%) 138 (18%) 758 513 (38%) 343 (25%) 225 (17%) 266 (20%) 1347 349 (49%) 171 (24%) 100 (14%) 87 (12%) 707 625 (45%) 309 (22%) 173 (12%) 295 (21%) 1402 295 (41%) 228 (31%) 107 (15%) 96 (13%) 726 530 (41%) 395 (30%) 195 (15%) 184 (14%) 1304 383 (53%) 174 (24%) 88 (12%) 73 (10%) 718 689 (54%) 299 (23%) 142 (11%) 144 (11%) 1274 38 Question 7 - Trouble concentrating on things like school 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= Pre July - September Post November - January 335 (46%) 190 (26%) 102 (14%) 102 (14%) 729 577 (43%) 360 (27%) 195 (15%) 201 (15%) 1333 8- Moving or speaking so slowly that other people could have noticed 0 - Not at all 409 (59%) 1 - Several days 138 (20%) 2 - More than 1/2 the days 73 (11%) 3 - Nearly every day 70 (10%) N= 690 9 - Thoughts that you would be better off dead 0 - Not at all 474 (75%) 1 - Several days 101 (16%) 2 - More than 1/2 the days 34 (5%) 3 - Nearly every day 21 (3%) N= 630 10 - Have you ever in your whole life tried to kill yourself or made a suicidal attempt? 0 - Yes 467 (87%) 1 - No 70 (13%) N= 537 11 - Have you felt depressed or sad most days, even if you felt ok? 0 - Yes 236 (40%) 1 - No 361 (60%) N= 597 757 (62%) 244 (20%) 121 (10%) 107 (9%) 1229 865 (76%) 167 (15%) 69 (6%) 39 (3%) 1140 242 (23%) 831 (77%) 1073 156 (16%) 847 (84%) 1003 39 Table 11 Anxiety (GAD7) (Pre and Post) Question Before After 44 (15%) 150 (50%) 40 (13%) 67 (22%) 301 142 (15%) 338 (36%) 215 (23%) 233 (25%) 928 91 (30%) 93 (31%) 60 (20%) 58 (19%) 302 236 (29%) 170 (21%) 197 (25%) 199 (25%) 802 71 (24%) 110 (37%) 59 (20%) 58 (19%) 298 204 (22%) 302 (32%) 210 (22%) 223 (24%) 939 95 (30%) 106 (34%) 63 (20%) 52 (16%) 316 247 (26%) 297 (32%) 205 (22%) 188 (20%) 937 5 - Being so restless that it's hard to sit still 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 107 (34%) 106 (34%) 43 (14%) 56 (18%) 312 309 (28%) 304 (28%) 230 (21%) 243 (22%) 1086 6 - Becoming easily annoyed or irritated 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 55 (18%) 98 (32%) 73 (23%) 85 (27%) 311 131 (14%) 304 (33%) 230 (25%) 260 (28%) 925 1 - Feeling nervous, anxious or on edge 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 2 - Not being able to stop or control worrying 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 3 - Worrying too much about different things 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 4 - Trouble relaxing 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= 40 Question 7 - Feeling afraid, as if something awful might happen 0 - Not at all 1 - Several days 2 - More than 1/2 the days 3 - Nearly every day N= Before After 126 (40%) 101 (32%) 48 (15%) 38 (12%) 313 335 (37%) 277 (30%) 163 (18%) 134 (15%) 909 41 Figure 1 IBH, Intercepting Behavioral Health Figure design by James Nguyen 42 Figure 2 Flow Diagram 43 Appendices Appendix 1 Top Behavioral Health Diagnoses ICD 10 Code F90.2 F84.0 F41.1 F90.0 F41.9 F41.8 F34.1 F91.3 F33.2 F43.23 F41.0 F90.1 F43.10 F43.21 F93.9 F43.22 F51.5 F63.3 F43.25 F50.00 F84.5 F90.8 F95.9 F41.3 F43.24 F43.29 F50.82 F95.2 F31.30 F31.81 F50.02 F60.3 Diagnosis Attention-deficit hyperactivity disorder, combined type Autistic disorder Generalized anxiety disorder Attention-deficit hyperactivity disorder Anxiety disorder unspecified Other Specified Anxiety Disorders Dysthymic disorder Oppositional defiant disorder Major depressive disorder, recurrent severe without psychotic features Adjustment disorder with mixed anxiety and depressed mood Panic disorder Attention-deficit hyperactivity disorder, predominantly hyperactive type PTSD unspecified Adjustment Disorder with depressed mood Childhood emotional disorder unspecified Adjustment disorder with anxiety Nightmare disorder Trichotillomania Adjustment disorder with mixed disturbance of emotions and conduct Anorexia nervosa Asperger's Attention deficit hyperactivity disorder, other type Tic disorder Other Mixed Anxiety Disorders Adjustment disorder with disturbance of conduct Adjustment disorder with other symptom Avoidant/restrictive food intake disorder Tourettes Disorder Bipolar disorder, current episode hypomania Bipolar II Anorexia nervosa, binge eating/purging type Borderline personality disorder N= 375 177 163 137 122 50 42 26 15 12 11 9 8 6 6 5 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 44 ˆAppendix 2 Pre-Intervention Survey 45 46 47 48 49 Appendix 3 Post-Intervention Survey 50 Page 1 Pediatric Primary Care Provider Pos t Implementation Survey Please complete the survey below regarding your perspective regarding pediatric primary care and behavioral health management. Please complete by October 5, 2021. Thank you! Corinne Bartschi, MSN, FNP-C Gender Male Female Gender Neutral Transgender Age 20-25 26-30 31-35 36-40 41-45 46-50 51-55 56-60 61-65 Role MD DO NP PA How many years have you been in practice? 0-5 6-10 11-15 16-20 21-25 26-30 31-35 How satisfied are you with managing pediatric behavioral health diagnoses? Very unsatisfied Unsatisfied Neutral Satisfied Very satisfied How burdensome is it to manage behavioral health in your practice? Not burdensome Slightly burdensome Neutral Moderately burdensome Very burdensome How difficulty is it to coordinate behavioral health services like counseling when you identify a patient in need? Very difficult Difficult Neutral Easy Very easy 04/03/2022 3:38pm projectredcap.org 51 52 53 54 Appendix 4 55