Improving Compliance of Measurement-Based Care in an Urban Community Mental Health Clinic: A Quality Improvement Initiative

Measurement-based care (MBC) is a systematic approach to clinical decision-making that involves collecting, analyzing, and applying patient-reported data to guide treatment planning and decisions. The use of MBC has become increasingly common across various healthcare fields, including mental health, where it has demonstrated positive outcomes. However, while the implementation of MBC is wide, clinicians may not fully understand the reasons behind its use and its fiscal implications. As it has become integrated into clinical practices, its financial impact on the healthcare system has also become critical for clinicians to understand. An urban community mental health clinic serving underserved populations requires using two MBC tools: the Outcome Questionnaire-45.2 (OQ) and the Patient Health Questionnaire-9 (PHQ-9). Financial reimbursement and a federal grant require compliance with using these tools. Lack of documentation and low compliance with these MBC tools puts the clinic's financial stability at risk. This quality improvement (QI) project implemented targeted interventions to improve low compliance rates with the OQ and PHQ-9 through pre- and post-intervention surveys, education, process development, and data surveillance. Surveys incorporating a Likert scale and open-ended questions assessed the clinician's baseline understanding and perceptions of MBC, OQ, and PHQ-9. Based on this data, the project implementation phase included a tailored training plan and monitoring to improve compliance rates. Clinicians were required to chart OQ and PHQ-9 scores during designated weeks. Throughout the project, the team monitored compliance rates. One month after the intervention, compliance was reassessed to evaluate sustainability. A post-survey measured changes in clinician understanding and perception while identifying ongoing barriers to MBC adoption. Clinical and administrative staff completed the pre-intervention survey with an 86% response rate (n = 31) and the post-intervention survey with a 72% response rate (n = 26). Most participants (58.1%) were therapists or therapy students with or working on graduate degrees. The rest were nurses, peer support specialists, administrative staff, or reception staff. OQ compliance showed no statistically significant change from pre- to post-intervention (p = 0.672) but significantly improved during the intervention period (p = 0.003). PHQ-9 compliance increased significantly from pre- to post-intervention (p = 0.001) and during the active intervention phase (p = 0.001). PHQ-9 compliance remained relatively low at the end of the project at 21%. Overall, the clinician's perceptions and understanding of MBC, OQ, and PHQ-9 improved from pre- to post-intervention. While clinician understanding and perception of MBC improved, compliance rates increased the most during the active intervention period. However, compliance declined after the intervention ended. These findings suggest that clinicians recognize the importance of MBC; however, system-related barriers, particularly limitations with the current electronic health record (EHR), hinder its sustainability. Increased administrative burden and workflow inefficiencies may prevent long-term adoption, even when clinicians acknowledge MBC's benefits. Future efforts should focus on integrating MBC into EHR systems, reducing administrative workload, and implementing automated reminders to support sustained compliance.

1 Improving Compliance of Measurement-Based Care in an Urban Community Mental Health Clinic: A Quality Improvement Initiative Shaw Akutsu, Rebekah Perkins, ElLois Bailey College of Nursing: The University of Utah NURS 7703: DNP Scholarly Project III March 30, 2025 2 Abstract Background Measurement-based care (MBC) is a systematic approach to clinical decision-making that involves collecting, analyzing, and applying patient-reported data to guide treatment planning and decisions. The use of MBC has become increasingly common across various healthcare fields, including mental health, where it has demonstrated positive outcomes. However, while the implementation of MBC is wide, clinicians may not fully understand the reasons behind its use and its fiscal implications. As it has become integrated into clinical practices, its financial impact on the healthcare system has also become critical for clinicians to understand. Local Problem An urban community mental health clinic serving underserved populations requires using two MBC tools: the Outcome Questionnaire-45.2 (OQ) and the Patient Health Questionnaire-9 (PHQ-9). Financial reimbursement and a federal grant require compliance with using these tools. Lack of documentation and low compliance with these MBC tools puts the clinic’s financial stability at risk. Methods This quality improvement (QI) project implemented targeted interventions to improve low compliance rates with the OQ and PHQ-9 through pre- and post-intervention surveys, education, process development, and data surveillance. Interventions Surveys incorporating a Likert scale and open-ended questions assessed the clinician’s baseline understanding and perceptions of MBC, OQ, and PHQ-9. Based on this data, the project implementation phase included a tailored training plan and monitoring to improve compliance 3 rates. Clinicians were required to chart OQ and PHQ-9 scores during designated weeks. Throughout the project, the team monitored compliance rates. One month after the intervention, compliance was reassessed to evaluate sustainability. A post-survey measured changes in clinician understanding and perception while identifying ongoing barriers to MBC adoption. Results Clinical and administrative staff completed the pre-intervention survey with an 86% response rate (n = 31) and the post-intervention survey with a 72% response rate (n = 26). Most participants (58.1%) were therapists or therapy students with or working on graduate degrees. The rest were nurses, peer support specialists, administrative staff, or reception staff. OQ compliance showed no statistically significant change from pre- to post-intervention (p = 0.672) but significantly improved during the intervention period (p = 0.003). PHQ-9 compliance increased significantly from pre- to post-intervention (p = 0.001) and during the active intervention phase (p = 0.001). PHQ-9 compliance remained relatively low at the end of the project at 21%. Overall, the clinician's perceptions and understanding of MBC, OQ, and PHQ-9 improved from pre- to post-intervention. Conclusion While clinician understanding and perception of MBC improved, compliance rates increased the most during the active intervention period. However, compliance declined after the intervention ended. These findings suggest that clinicians recognize the importance of MBC; however, system-related barriers, particularly limitations with the current electronic health record (EHR), hinder its sustainability. Increased administrative burden and workflow inefficiencies may prevent long-term adoption, even when clinicians acknowledge MBC’s benefits. Future efforts 4 should focus on integrating MBC into EHR systems, reducing administrative workload, and implementing automated reminders to support sustained compliance. Keywords: Measurement-based care (MBC), Outcome Questionnaire-45.2 (OQ-45.2), Patient Health Questionnaire-9 (PHQ-9), compliance, community mental health 5 Improving Compliance of Measurement-Based Care in an Urban Community Mental Health Clinic: A Quality Improvement Initiative Problem Description Measurement-based care (MBC) can enhance treatment outcomes and improve clinical decision-making by systematically collecting client-reported data, which clinicians can use to tailor treatment adjustments. A community mental health clinic serving urban, underserved populations in Utah faced a challenge in maintaining compliance with MBC tools, specifically the Outcome Questionnaire-45.2 (OQ) and the Patient Health Questionnaire-9 (PHQ-9). Compliance with these tools is required as part of the clinic’s contract with Medicaid insurance and to maintain secure funding for a Substance Abuse and Mental Health Services Administration (SAMHSA) grant. Clinicians must administer the OQ upon initial assessment, every 30 days, and before client discharge. Clients must also complete the PHQ-9 upon admission, at six months, and before discharge. During initial assessments, clinicians reliably complete the OQ and PHQ-9; however, compliance with subsequent administration and documentation remains low. Despite management efforts, including multiple training sessions, compliance with these MBC tools remains minimal. Administrators have struggled to identify the exact nature and extent of the barriers, complicating practical efforts to address the problem. Frequent staff turnover and regular rotations of students and interns exacerbated the challenges by creating inconsistencies in training and adherence to compliance protocols within an already overstretched clinic environment. Since the clinic is a non-profit organization, it relies on federal and state funding, as well as private donations, to sustain its operations. Most clients are uninsured or underinsured and rely on Medicaid, which covers billable services. Therefore, 6 compliance with MBC tools is not only a clinical requirement but also a financial necessity; failure to meet compliance standards can result in the loss of the SAMHSA grant and paybacks of Medicaid billing. If Medicaid audits a client’s documentation and identifies non-compliance with OQ requirements twice, they may request repayment for all billable services provided, ranging from one to six months of services. The repayment depends on the client’s level of care, and payback amounts range from $1,400 to $22,000. Clinicians must ensure PHQ-9 compliance to demonstrate the effectiveness of the clinic’s treatment programs for SAMHSA, which is essential for ongoing grant support. The clinic receives approximately $930,000 annually as a Certified Community Behavioral Health Clinic (CCBHC) through SAMHSA-funded grants, totaling $3.9 million. SAMHSA requires PHQ-9 data as the key reporting requirement. Ensuring compliance with MBC tools is vital for maintaining financial stability, upholding Medicaid requirements, and demonstrating the clinic’s effectiveness. Without proper compliance, the clinic risks incurring financial penalties, losing funding, and reducing its capacity to provide essential mental health services to the community. They also miss out on the enhanced treatment outcomes and improved clinical decision-making that MBC offers. Available Knowledge The efficacy of MBC in enhancing patient care outcomes is well-documented in the literature. De Jong et al. (2021) reported that using MBC significantly reduces symptoms across diverse case types, attributing this success to the simplicity and cost-effectiveness of MBC. Although integrating MBC, including the necessary training and ongoing usage, increased the clinical staff’s workload, its overall benefits—such as improved patient outcomes and costeffectiveness—justified its implementation despite the additional demands. 7 The PHQ-9 demonstrates high reliability and validity (Cronbach’s ⍺ = 0.892), indicating strong internal consistency, and researchers have validated the PHQ-9 in primary care settings, highlighting its simplicity, rapid application, and effectiveness for screening depression severity (Costantini et al., 2021; Sun et al., 2020). The PHQ-9 also exhibits high sensitivity (0.85) and specificity (0.85), reinforcing its utility as a robust diagnostic tool (Negeri, 2021). Despite these strengths, Korsen and Gerrish (2022) found that primary care providers underutilize the PHQ-9 for monitoring depressive symptoms over time, underscoring the importance of consistently integrating MBC into clinical practice to enhance depression management. Researchers validated the OQ as an effective MBC tool, with Matavovsky et al. (2023) demonstrating its strong internal consistency (Cronbach’s α = 0.951) and cultural sensitivity in measuring both subclinical and clinical symptoms. Redmayne et al. (2024) found that the OQ offers unique treatment insights, reinforcing the importance of selecting MBC tools based on clinical goals and patient populations. MBC enhances clinician fidelity, treatment adherence, and patient outcomes in mental health settings (Hallgren et al., 2022; Lewis, Marti, et al., 2022; Wray et al., 2023). In a clusterrandomized trial, Lewis, Marti, et al. (2022) found that tailored MBC implementation outperformed standardized approaches in improving clinician fidelity and outcomes for depression. Hallgren et al. (2022) and Lewis, Marti, et al. (2022) observed that a digital, remotely delivered MBC system increased client and clinician engagement. Wray et al. (2023) found that intensive strategies, such as external facilitation or QI teams, drove greater adoption and effectiveness. Similarly, Lewis et al. (2019) concluded that MBC facilitates improvements and enables clinicians to detect patient deterioration earlier. Payers and grantors assess care 8 quality, monitor outcomes, and ensure compliance with value-based payment models by linking reimbursement to treatment effectiveness rather than service quantity (Connors et al., 2021). Despite the evidence demonstrating the efficacy of tools like the PHQ-9 and OQ, several barriers hinder the widespread adoption of MBC. Murphy et al. (2021) identified a lack of familiarity among clinicians with validated tools as a significant obstacle, exacerbated by deficits in knowledge, education, and training. This unfamiliarity undermines clinicians’ confidence in using and interpreting MBC tools, limiting their effectiveness. Additionally, Yule et al. (2023) noted that perceived barriers to MBC implementation vary across clinical settings and staff roles. Professionals with varying educational backgrounds, specialties, and responsibilities exhibit differing levels of exposure to and comfort with MBC tools. Clinics must improve communication, education, and training across all staff levels to address these challenges. Keepers et al. (2023) found that clinicians and staff supported MBC and recognized its potential to enhance client care. Many expressed a willingness to undergo additional training to enhance their clinical practice. To address barriers and foster greater MBC adoption, clinics should implement targeted education and training initiatives, which could improve both client outcomes and the overall quality of care. Rationale The Johns Hopkins Evidence-Based Practice Model (JHEBP) guided this quality improvement initiative. The model facilitated a continuous assessment process, allowing for the refinement and adaptation of interventions throughout the project. Data were initially gathered through a survey to identify knowledge gaps and understand staff attitudes and perceptions towards MBC. These data were then used to create practical and relevant interventions tailored to clinical needs. The outcomes of the interventions were assessed, evaluated, and refined as 9 needed to continue enhancing their effectiveness. Using the JHEBP Model enabled the project to cycle through multiple iterations of assessments, refine interventions, and synthesize and analyze collected data, thereby improving the feasibility and sustainability of the project for the clinic. Specific Aims This Doctor of Nursing Practice (DNP) evidence-based quality improvement (QI) initiative aimed to increase compliance with obtaining, documenting, and using the minimum required MBC standards at an urban community mental health clinic through training and systems development to ensure continued clinic funding and improved client care. Methods Context The author conducted this quality improvement project at an urban community mental health clinic that predominantly serves uninsured or underinsured individuals. Many clients enroll in Medicaid, making services Medicaid-reimbursable. Additionally, a large portion of clients receive court orders to participate in treatment programs for mental health, substance use, and domestic violence. The clinic employs or precepts 36 clinicians, clinician students, and staff. Licensed clinical social workers provide clinical services with additional support from interns, master's-level social work students, and nurse practitioner students. Nurse practitioners prescribe medications, while nurses administer certain medications, such as long-acting injectables. Any available clinician conducts assessments on a first-come, first-served basis or during scheduled appointments. Aside from clinical care, the clinic assists clients with securing long-term housing, employment, food security, and disability services. Compliance with the OQ is necessary for reimbursement through Medicaid, which requires the form to be completed at admission, monthly, and at discharge. If Medicaid audits a 10 client twice and finds non-compliance, the clinic may be required to repay the service cost. The clinic also operates as a CCBHC and receives federal funding from SAMHSA. This four-year, $3.7 million grant ($930,000 annually) requires that clients with major depressive disorder (MDD) complete the PHQ-9 to measure treatment outcomes. However, the clinic lacked a structured tracking system for PHQ-9 compliance, creating challenges in meeting funding requirements and optimizing client care outcomes. Interventions To enhance clinicians' documentation and tracking of client progress using the OQ and PHQ-9, the author implemented targeted training and systematic documentation strategies (Appendix A). Data were collected over two months (November–December 2024), with two designated weeks each month, resulting in four weeks of data collection. This timeline enabled sufficient data gathering and allowed for timely adjustments to the intervention. The training was reinforced through group supervision meetings, which included structured sessions focused on MBC tools. Additionally, biweekly all-staff meetings provided opportunities for follow-up, addressing training needs, and answering staff questions. For those unable to attend in person, the author distributed a training email and required recipients to confirm they had reviewed and understood the content. Using a standardized template, clinicians documented OQ scores in the Electronic Health Records (EHR) system. They recorded the score in a miscellaneous note with the subject header "OQ Score" and updated it when they were due. If a client declined to complete the OQ, clinicians documented the date and the client’s decision to ensure accurate recordkeeping. The author trained clinicians to use specific billing codes for PHQ-9 documentation, enabling systematic tracking. Since the current EHR lacked a PHQ-9 tracking system, the author 11 developed a workaround using billing codes to monitor completed PHQ-9s. Clinicians used the Current Procedural Terminology billing code ‘G0444’ for annual depression screening, documenting encounter notes that included the total PHQ-9 score. They used the billing code ‘G8431’ to document positive depression screenings with a PHQ-9 score above nine, along with a documented follow-up plan. For negative screenings with a PHQ-9 score below nine, clinicians applied the billing code ‘G8510.’ When a client declined to complete the PHQ-9, clinicians used the billing code ‘G8433’ to indicate refusal. These billing procedures aimed to streamline data collection and enhance the reliability of tracking PHQ-9 (Appendix A). Study of the Interventions The author administered a pre-intervention survey to collect baseline data on staff knowledge, attitudes, and perceptions regarding current practices for assessing and complying with MBC standards. Using a Likert scale, the survey provided valuable insights into staff needs and perspectives, informing the development of a targeted approach to enhance compliance (Appendix B). To evaluate the impact of the interventions, the author tracked compliance rates for the OQ and PHQ-9 during two-week intervals in November and December of 2024. To assess the sustainability of the intervention, the author conducted a follow-up audit one month after completing the interventions at the end of January 2025. Clinical staff discussed barriers and difficulties during biweekly meetings, allowing for continuous adjustments and refinements to the intervention. Key project sponsors and stakeholders met regularly throughout the project to discuss results, feedback, and feasible intervention strategies, allowing participants to review successful interventions, ensure alignment with the clinic’s goals, and confirm the expected outcomes of improved compliance with the required MBC documents. After project completion, the author 12 distributed a post-intervention survey that mirrored the pre-intervention survey but included additional Likert scale questions assessing satisfaction, feasibility, and sustainability. The survey evaluated staff perceptions of the interventions, including usability and long-term implementation potential (Appendix C). The author created an executive summary for the project sponsor and stakeholders, summarizing the QI project results to inform future initiatives and improve similar projects (Appendix D). Measures The pre-intervention survey, comprised of questions to assess clinicians’ baseline knowledge, attitudes, confidence, and perceptions regarding MBC, OQ, and PHQ-9, included both Likert scale items and open-ended questions. Likert responses were measured on a fivepoint scale (1 = Strongly Disagree to 5 = Strongly Agree), with higher scores indicating greater knowledge, positive attitudes, and confidence in using MBC tools. Mean scores were calculated for each domain and overall MBC components. Open-ended responses provided qualitative insights into perceived barriers and opportunities for improving compliance. After the intervention, the author conducted a post-survey identical to the preintervention survey. The post-intervention survey mirrored the same questions to measure changes in clinicians’ knowledge, attitudes, and domain-specific measures. Additionally, the post-intervention survey included additional Likert scale questions to assess clinicians’ perceptions of feasibility, usability, and sustainability. The post-intervention survey retained the same open-ended questions from the pre-survey to explore remaining challenges or new barriers, ensuring the clinic could adapt and expand interventions as needed. The data administration team tracked compliance with the OQ and PHQ-9 through scheduled audits, which occurred when MBC screeners were due. These audits provided ongoing 13 compliance data, enabling the author to evaluate whether the interventions achieved the desired outcomes. Project sponsors and stakeholders reviewed progress during biweekly meetings, addressing challenges, refining interventions, and aligning strategies with clinic goals. This iterative process ensured that data collection and evaluation remained responsive and effective. The author used standardized Likert scales for pre- and post-intervention comparisons to ensure data reliability and validity. The author applied a 2-sample t-test to measure statistical differences in MBC, OQ, and PHQ-9 scores, a chi-square test to assess statistical differences in OQ and PHQ-9 compliance, and a qualitative analysis of open-ended responses to identify recurring themes and inform targeted improvements. The author also calculated mean Likert scale scores for feasibility, usability, and sustainability to assess clinicians’ perceptions of the intervention’s overall effect on whether the implementations were sustainable for future use. Analysis Demographic data were analyzed using descriptive statistics to calculate frequencies and percentages for variables such as age ranges, clinical roles, and the highest level of education attained. Quantitative data from the Likert scale surveys were analyzed to assess changes in clinicians' knowledge, attitudes, confidence, and frequency of MBC tool usage. Likert responses were numerically coded (1 = Strongly Disagree to 5 = Strongly Agree), and then mean calculations were completed for each domain and overall MBC components. To assess statistical differences between pre- and post-intervention scores, two-sample independent t-tests were applied. Additionally, the author calculated mean Likert scale scores for feasibility, usability, and sustainability to evaluate clinicians’ perceptions of the intervention’s long-term applicability. Audits conducted during scheduled documentation weeks monitored compliance with the OQ 14 and PHQ-9. A chi-square test compared compliance rates across the pre-intervention, intervention, and post-intervention periods, as well as during the designated screening weeks. For the qualitative analysis, open-ended survey responses were first transcribed and carefully read to familiarize ourselves with the data before analyzing the data. Initial codes were then systematically generated to capture key concepts and patterns relevant to clinicians' experiences with MBC implementation. The author organized these codes into potential themes and reviewed and refined them to ensure they accurately reflected the data. Themes were then clearly defined and named, identifying recurring issues, perceived barriers, and suggestions for improving MBC compliance. This process and the analytical findings provided valuable insights to inform future strategies for enhancing MBC implementation and sustainability. Ethical Considerations This project followed the institution’s Umbrella IRB protocol for DNP-QI projects, which pre-approves low-risk QI studies. Projects meeting criteria—minimal participant risk, no sensitive data collection beyond necessity, and no intent to generalize findings—undergo review to ensure alignment. After confirming eligibility, the author documented the project under the Umbrella IRB and obtained written acknowledgment. Participation was voluntary, with informed consent provided. The author maintained confidentiality and anonymity by de-identifying data and found no conflicts of interest. Results This section presents the findings from pre- and post-intervention surveys, compliance audits, and qualitative analyses of open-ended responses. Results are organized by participant demographics, compliance rates, survey outcomes, and qualitative insights regarding barriers to MBC implementation. 15 Participant Demographics Clinicians and staff participated well in the surveys, with 31 (86% response rate) completing the pre-survey and 26 (72% response rate) completing the post-survey. Among the pre-survey participants, 18 identified as therapists, five as administrative staff, and eight as 'other,' including front desk staff, nurses, and peer support specialists. Therapists were either Licensed Clinical Social Workers (LCSWs), Clinical Social Workers (CSWs), or CSW students completing their master’s internships. The most common age range was 26–30 (27.5%), with a median age of 35. Most participants (58.1%) identified as therapists and held a master’s degree as their highest level of education. Additionally, 38.7% reported 2–4 years of professional experience (Table 1). OQ and PHQ-9 Compliance Rates The author categorized pre- and post-intervention survey responses into four domains: knowledge, confidence, attitude, and frequency, with mean scores calculated on a Likert scale (1–5; 1 = strongly disagree, 5 = strongly agree). The post-intervention survey also included questions about feasibility, usability, and sustainability. OQ-45.2 Compliance Measurement of compliance with the OQ and PHQ-9 occurred through audits conducted before, during, and after the QI intervention. Compliance rates increased from 42% at baseline to 50% by week 3 but slightly declined to 47% by week 4 and decreased to 40% one month postintervention. (Table 2; Figure 1) A chi-square test comparing pre- and post-intervention OQ compliance indicated no statistically significant change (𝑥 ! = 0.18, p = 0.672); however, weekby-week analysis during the intervention showed a statistically significant change (𝑥 ! = 14.33, p = 0.003; Table 3). 16 PHQ-9 Compliance PHQ-9 compliance showed more significant initial improvements, rising from 3% to 41% in week 3 and declining to 21% one month later (Table 2; Figure 1). A chi-square test comparing pre- and post-intervention PHQ-9 compliance showed a statistically significant change (𝑥 ! =36.94, p = 0.001; Table 3). Week-by-week analysis also indicates (𝑥 ! = 30.01, p=0.001; Table 3). While compliance initially improved, adherence declined after the intervention. Comparison of the MBC, OQ, and PHQ-9 Domains from Pre- to Post-Intervention Post-implementation results showed improvements across all domains, including knowledge, confidence, attitude, and frequency. Clinicians responded with increased knowledge of MBC and confidence in MBC and the OQ. The frequency of use also improved for MBC, OQ, and PHQ-9. However, attitudes towards MBC, OQ, and PHQ-9 showed the least change, with MBC experiencing the smallest improvement (Table 4, Figure 2). Two Sample T-tests of MBC, OQ, and PHQ-9 The author conducted two-sample independent t-tests to compare the MBC, OQ, and PHQ-9 results from the pre- and post-surveys. The results showed p-values of 0.0002 for MBC, 0.0004 for OQ, and 0.0044 for PHQ-9, all of which fell below the significance threshold of 0.05. The null hypothesis stated that no difference existed between pre- and post-intervention survey scores following the intervention, while the alternative hypothesis proposed a significant difference due to the intervention. Since all three p-values were below 0.05, the analysis rejected the null hypothesis and confirmed statistically significant improvements across MBC, OQ, and PHQ-9 domains (Table 5). Feasibility, Usability, and Sustainability 17 The post-survey included nine Likert-scale questions that assessed the feasibility, usability, and sustainability of the QI project. The mean score across these nine questions was 3.836 on a 1-5 scale, indicating a leaning toward agreement. Feasibility and usability ratings were both 3.86, while sustainability scored slightly lower at 3.6 (Table 6). Pre-Implementation Survey Open-Ended Questions Before implementing the QI project, the author asked clinicians three open-ended questions to gather insights on barriers to MBC adoption. Clinicians were asked what (1) barriers prevented them from incorporating the OQ and PHQ-9 more frequently into their charting, (2) what improvements could help increase the use of MBC in client charting at the clinic, and (3) what challenges in the current system made it difficult to use MBC more effectively. Four key themes were identified from participants’ responses and included (1) training and education, (2) technological and EHR challenges, (3) time constraints, and (4) process and workflow issues. Training and Education Responses indicated a need for more comprehensive training, specifically on using MBC, interpreting scores, and integrating them into clinical practice. Clinicians expressed a need for additional training, examples, and structured education on implementation. Some clinicians stated that they needed "more training and examples" and "training on how to use it better," while others noted that there had been "limited training on them." Technological and EHR Challenges Many clinicians described the current EHR system as difficult to use and poorly integrated with MBC charting. They reported difficulties accessing the system, multiple login requirements, and an inefficient tracking process. Some clinicians described the current EHR system as “tedious” and noted that "not all assessments are in one place," making it challenging 18 to streamline documentation. Others mentioned that multiple "login screens make it timeconsuming to measure." Time Constraints Clinicians repeatedly cited time constraints as barriers to incorporating MBC documentation into their workflow. Many clinicians struggled to complete MBC documentation between sessions, after, or during therapy. Administrative tasks further reduced the time available for direct client care. One clinician referred explicitly to "time in session" as a barrier to completing documentation efficiently. Process and Workflow Issues Clinicians identified workflow inconsistencies, a lack of standardized procedures, and challenges tracking progress over time. They expressed concerns about unclear guidelines, describing "inconsistency in process," "no standardized documentation procedures," and "not knowing how to track progress over time." These workflow issues made it challenging to implement MBC consistently across different clinicians. Post-Implementation Survey Open-Ended Questions The post-survey included three additional open-ended responses on MBC implementation to examine the remaining barriers. Clinicians were asked (1) what barriers prevented the incorporation of the OQ and PHQ-9 into charting, (2) what improvements could increase the use of MBC in client documentation, and (3) what challenges in the current system made MBC implementation difficult. The analysis identified these four key themes: (1) time constraints and workload, (2) systems integration, (3) client engagement, and (4) data accuracy. Time Constraints and Workload 19 Clinicians cited time constraints and heavy workloads as significant barriers to integrating MBC into their workflow. Newer clinicians described feeling overwhelmed by the added responsibilities. One stated, “It feels overwhelming as a new clinician to try and keep up with this initiative.” Another noted difficult finding “enough time to put the scores into UWITS.” Systems Integration Clinicians expressed frustration with the EHR system’s lack of seamless integration, which is making MBC documentation more burdensome. One clinician stated, “An easier way to put the scores into UWITS would help.” Another noted, “Scores have to be entered manually, which has challenges.” Clinicians suggested that an automated reminder system could improve compliance, as increased workload and poor EHR integration led to missed documentation. Additionally, frequent process changes and inconsistent training across the clinic contributed to confusion. One clinician emphasized the need for consistency, stating, “We have been doing many changes in processes. The more we stay consistent with expectations, the more MBC will be incorporated consistently.” Client Engagement and Accuracy Clinicians questioned the accuracy of self-reported MBC scores, noting that patients underreported symptoms, distrusted the system, or denied their needs. One clinician stated, “Some of my clients score very low on the OQ for various reasons like underreporting needs, denial, or distrust of the system.” 20 Discussion Summary This project evaluated the impact of training, education, and system changes on compliance and clinician agreement with MBC, specifically OQ and PHQ-9, to determine whether targeted interventions improved documentation and compliance. Training and education improved staff and clinician agreement on MBC in general and OQ and PHQ-9 specifically, as indicated by a positive shift in pre- and post-survey scores. However, OQ compliance rates showed no statistically significant change from pre- to postintervention (p = 0.672). In contrast, PHQ-9 compliance significantly improved (p = 0.001), likely due to the presence of a tracking method in the intervention where there was no ability to track the PHQ-9 before the intervention. Although OQ compliance showed no significant change from pre- to post-intervention, it improved significantly during the intervention period. Although PHQ-9 compliance has increased, it remains low overall. The persistently low compliance rates continue to place the clinic at financial risk regarding Medicaid reimbursements and grant funding. Despite improved knowledge, confidence, and attitude towards MBC, open-ended survey responses revealed persistent barriers, including time constraints, EHR limitations, and the absence of a reminder system, which hindered compliance. Although staff and clinicians recognized MBC’s importance, sustained compliance requires further systemic changes, such as streamlined EHR integration, automated reminders, and workflow modifications tailored to clinician needs. Strong stakeholder support from administrators and management facilitated clinician engagement, contributing to high survey participation and a low attrition rate. This support 21 strengthened the reliability of data collection and increased clinicians' willingness to adopt MBC practices. Interpretation The results indicate that targeted training and system interventions can improve clinicians' immediate engagement with MBC tools. Clinicians' increased confidence and knowledge align with previous research by Lewis, Boyd, et al. (2022), who found that structured interventions enhanced MBC adoption in behavioral health settings. However, similar to their findings, this project also observed a decline in compliance after the intervention, suggesting that external accountability and continuous reinforcement are crucial for sustaining MBC practices. This pattern suggests clinicians may reduce MBC usage once external monitoring and reminders are withdrawn, underscoring the importance of establishing long-term reinforcement strategies. These findings align with those of Lewis, Boyd, et al. (2022), who found that PHQ-9 compliance increased during active intervention but declined once external accountability and support ended, mirroring the temporary gains observed in this project. System barriers, particularly the lack of integrated MBC tracking within the EHR, increased clinician workload and discouraged longterm adherence. Without built-in tracking, clinicians relied on workarounds, such as documenting billing codes for PHQ-9 and maintaining separate notes for OQ, which added an administrative burden and reduced efficiency. This project clarified the need for system-level improvements in ease of use and integration into the EHR. Persistent non-compliance continues to pose a significant financial risk, as failure to meet OQ requirements increases the likelihood of Medicaid audits and potential repayment demands. Such financial penalties, compounded by ongoing budget constraints and shifting Medicaid policies, could jeopardize clinic funding, staffing capacity, and service availability. While 22 education and training improve clinician understanding of MBC, these strategies alone are insufficient to achieve long-term adherence. Sustained compliance requires systemic changes, including reducing the documentation burden, enhancing EHR integration, and implementing automated reminders to prompt timely documentation. Without addressing these structural barriers, compliance rates are likely to continue fluctuating, limiting the long-term success of MBC implementation. Future interventions should prioritize reducing the documentation burden, enhancing EHR integration, and implementing automated reminders to sustain adherence. Limitations This project has several limitations related to project design, context, and data collection methods, which may influence the generalizability and interpretation of the findings. Study Duration and Data Collection The two-month duration of the project limited the data collection period, restricting OQ and PHQ-9 charting to just four weeks. This shortened timeline may limit clinicians' ability to adjust to the intervention and refine compliance practices. Although the author increased charting frequency from once to twice monthly to mitigate this constraint, an extended data collection period may have allowed for more meaningful insights into long-term trends and the refinement of interventions. Future projects should consider extended timelines to better evaluate sustainability outcomes over time. Contextual Limitations This project was conducted within a single urban community mental health clinic serving a unique and underserved population. The findings may not be generalizable to other clinics, particularly those with higher rates of depressive disorders or more established MBC protocols, 23 which may already have systems in place for sustained compliance. Additionally, varying perceptions of the OQ across community mental health clinics may influence how well they integrate it into practice. Data Collection Methods and Survey Limitations Reliance on Likert scale surveys and open-ended responses may not fully capture participants’ perspectives. The survey’s length may have contributed to respondent fatigue, causing some clinicians to select "neither agree nor disagree" out of ambivalence rather than accuracy. The clinic’s EHR system created another obstacle, necessitating workarounds rather than providing seamless integration. The author attempted to mitigate this issue by aligning the intervention with existing clinician workflows. However, the system’s limitations likely hindered sustained compliance. Systemic Barriers: EHR Limitations The clinic’s EHR system posed significant challenges to consistent MBC documentation. Due to the lack of seamless EHR integration, clinicians used manual workarounds, such as entering scores in separate notes or billing codes. Although the intervention attempted to align with existing workflows to minimize disruption, these system limitations likely contributed to reduced efficiency and long-term adherence challenges. Research by Wray et al. (2023) similarly identified that inefficient EHR processes increase administrative burden and reduce clinician engagement. Future efforts should prioritize the development of automated, integrated EHR solutions to streamline MBC documentation. Resistance to Change Although not directly measured, resistance to change may have influenced clinicians’ willingness to adopt and consistently use MBC tools. New documentation processes often 24 require behavioral adjustments and additional effort, which can meet resistance, especially in high-demand clinical environments. To address this potential limitation, project members participated in group supervision meetings and open discussions, encouraging feedback and support. However, future interventions may benefit from assessing attitudes towards change more directly and implementing strategies to enhance clinician buy-in, such as change management frameworks or peer mentorship programs. Conclusions This project demonstrated that training and education improved clinician perception towards MBC. However, this did not necessarily translate into increased adoption or compliance, particularly with OQ and PHQ-9. Sustained compliance remained a challenge, reinforcing the need for structural and systemic changes to support long-term adherence. Before introducing new screening tools or documentation requirements, clinic leadership must consider system support, workflow, workload, training, and consistent accountability. Adoption may be higher if clinicians have a strong understanding and comfort level with MBC tools before implementation. Integration with system support is essential to sustain MBC use, particularly for tools with compliance mandates. Implementation efforts may fail if they create unnecessary administrative burdens for clinicians. Streamlined workflows, EHR integration, and automated reminders can facilitate adoption while minimizing disruptions to clinical practice. Additionally, consistent training and clear communication about expectations and timing are crucial for longterm adherence. Other community mental health settings could adopt similar interventions, particularly those centers struggling with compliance and dependent on Medicaid or state and federal grant funding. Future efforts should explore strategies that balance compliance requirements with 25 clinician workload, ensuring that MBC tools enhance patient care rather than becoming an additional burden. Organizations should prioritize system-wide changes to enhance sustainability, such as embedding MBC tracking into EHRs and establishing routine compliance monitoring. Furthermore, future research should investigate whether long-term reinforcement strategies, such as ongoing training and policy adjustments, enhance adherence beyond the intervention period. In conclusion, this project highlights the importance of combining education with systemic improvements to achieve sustainable compliance with MBC. Data from this project may inform future interventions where clinician awareness and support are present. However, compliance may remain low in systems not designed for usability despite the potential benefits for clients. 26 Acknowledgments I want to express my sincere gratitude to Cassie Petty from the VOA for her assistance in auditing and supplying the data for this project. I also would like to thank the key stakeholders who supported this initiative: Rich Garcia, Ashley Barajas, Rob Snow, Marie Jackson, and Chaelyn Vidal. A special thank you to my father, Dr. Hiromi Akutsu, for his expertise and guidance in analyzing the data and statistics for this project. Lastly, I would like to thank Dr. Rebekah Perkins for her ongoing support throughout this entire project. Your encouragement and guidance throughout this process have been invaluable to me. 27 References Connors, E. H., Douglas, S., Jensen-Doss, A., Landes, S. J., Lewis, C. C., McLeod, B. D., Stanick, C., & Lyon, A. R. (2021). What gets measured gets done: How mental health agencies can leverage measurement-based care for Better Patient Care, clinician supports, and organizational goals. Administration and Policy in Mental Health and Mental Health Services Research, 48(2), 250–265. https://doi.org/10.1007/s10488-020-01063-w Costantini, L., Pasquarella, C., Odone, A., Colucci, M. E., Costanza, A., Serafini, G., Aguglia, A., Belvederi Murri, M., Brakoulias, V., Amore, M., Ghaemi, S. N., & Amerio, A. (2021). Screening for depression in primary care with Patient Health Questionnaire-9 (PHQ-9): A systematic review. Journal of Affective Disorders, 279, 473–483. https://doi.org/10.1016/j.jad.2020.09.131 de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021). Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis. Clinical Psychology Review, 85, 102002. https://doi.org/10.1016/j.cpr.2021.102002 Hallgren, K. A., Cohn, E. B., Ries, R. K., & Atkins, D. C. (2022). Delivering remote measurement-based care in community addiction treatment: Engagement and usability over a 6-month clinical pilot. Frontiers in Psychiatry, 13. https://doi.org/10.3389/fpsyt.2022.840409 Keepers, B. C., Easterly, C. W., Nora, D., Domino, M. E., & Bhalla, I. P. (2023). A survey of behavioral health care providers on use and barriers to use of measurement-based care. Psychiatry Services, 74(4), 349–357. https://doi.org/10.1176/appi.ps.202100735 28 Korsen, N., & Gerrish, S. (2022). Use of PHQ-9 for monitoring patients with depression in integrated primary care practices. Annals of Family Medicine, 20, 2769. https://doi.org/10.1370/afm.20.s1.2769 Lewis, C. C., Boyd, M. R., Marti, C. N., & Albright, K. (2022). Mediators of measurement-based care implementation in community mental health settings: Results from a mixed-methods evaluation. Implementation Science, 17(1). https://doi.org/10.1186/s13012-022-01244-1 Lewis, C. C., Boyd, M., Puspitasari, A., Navarro, E., Howard, J., Kassab, H., Hoffman, M., Scott, K., Lyon, A., Douglas, S., Simon, G., & Kroenke, K. (2019). Implementing measurement-based care in behavioral health. Journal of the American Medical Association Psychiatry, 76(3), 324. https://doi.org/10.1001/jamapsychiatry.2018.3329 Lewis, C. C., Marti, C. N., Scott, K., Walker, M. R., Boyd, M., Puspitasari, A., Mendel, P., & Kroenke, K. (2022). Standardized versus tailored implementation of measurement-based care for depression in community mental health clinics. Psychiatric Services, 73(10), 1094–1101. https://doi.org/10.1176/appi.ps.202100284 Matavovszky, D., Nguyen Luu, L. A., & Karner, O. (2024). Development of an instrument to assess the mental health of university students: Validation of the outcome questionnaire45 in a Hungarian sample. Frontiers in Psychology, 15(1334615). https://doi.org/10.3389/fpsyg.2024.1334615 Murphy, J. K., Michalak, E. E., Liu, J., Colquhoun, H., Burton, H., Yang, X., Yang, T., Wang, X., Fei, Y., He, Y., Wang, Z., Xu, Y., Zhang, P., Su, Y., Huang, J., Huang, L., Yang, L., Lin, X., Fang, Y., . . . Chen, J. (2021). Barriers and facilitators to implementing measurement-based care for depression in Shanghai, China: A situational analysis. BioMed Central Psychiatry, 21(1), 430. https://doi.org/10.1186/s12888-021-03442-5 29 Negeri, Z. F., Levis, B., Sun, Y., He, C., Krishnan, A., Wu, Y., Bhandari, P. M., Neupane, D., Brehaut, E., Benedetti, A., & Thombs, B. D. (2021). Accuracy of the Patient Health Questionnaire-9 for screening to detect major depression: Updated systematic review and individual participant data meta-analysis. British Medical Journal. https://doi.org/10.1136/bmj.n2183 Redmayne, K. A., Reese, R. J., Zetzer, H. A., Conoley, C. W., & Owen, J. (2023). Do routine outcome monitoring measures tell the same story? Evaluating outcome trajectories and clinical outcomes for the Outcome Rating Scale and Outcome Questionnaire - 45. Psychotherapy Research, 34(2), 195–204. https://doi.org/10.1080/10503307.2023.2181113 Sun, Y., Fu, Z., Bo, Q., Mao, Z., Ma, X., & Wang, C. (2020). The reliability and validity of PHQ-9 in patients with major depressive disorder in psychiatric hospital. BioMed Central Psychiatry, 20(1). https://doi.org/10.1186/s12888-020-02885-6 Wray, L. O., Oslin, D. W., Leong, S. H., Pitcock, J. A., Tauriello, S., Drummond, K. L., & Ritchie, M. J. (2023). Enhancing implementation of measurement-based mental health care in primary care: A mixed-methods study. Psychiatric Services, 74(7), 746–755. https://doi.org/10.1176/appi.ps.20220140 Yule, A. M., Youn, S. J., Dean, K., Woodward, D. W., Firmin, E. S., Kramer, J., Stone, M., Marques, L., & Wilens, T. E. (2024). Different factors identified by stakeholder group for barriers and facilitators to measurement-based care implementation in behavioral health clinics. Journal of Clinical Psychology, 80(3), 576–590. https://doi.org/10.1002/jclp.23639 30 Tables and Figures Table 1 Sociodemographic Characteristics of Participants Characteristic n % 20 – 25 3 9.7% 26 – 30 8 25.8% 31 – 35 5 16.1% 36 – 40 4 12.9% 41 – 45 4 12.9% 46 - 50 4 12.9% 51 - 55 0 0% 56 - 60 1 3.2% Administrative 5 16.1% Therapist 18 58.1% Prescriber or provider 0 0% Other 8 25.8% GED or high school 3 9.7% Associate Degree 1 3.2% Age range in years Clinical role Highest attained education 31 Characteristic n % Bachelor’s Degree 8 25.8% Master’s degree 18 58.1% Doctorate 1 3.2% Other 0 0% 0 – 1 years 8 25.8% 2 – 4 years 12 38.7% 5 – 7 years 6 19.4% 8 – 10 years 0 0% 10+ years 5 16.1% Years in profession Note: The clinical role of ‘other’ encompasses peer support specialists, nurses, and front desk staff members. 32 Table 2 OQ 45.2 and PHQ-9 Audits of Compliance Rates Time Frame OQ 45.2 PHQ-9 Pre-Intervention Month 42% 3% Week 1 28% 12% Week 2 42% 34% Week 3 50% 41% Week 4 47% 27% Post Intervention Month 40% 21% 33 Table 3 Chi-Squared Results of OQ 45.2 and PHQ-9 Compliance Rates MBC Tool Time Frame Pre- vs. Post- OQ 45.2 Intervention Weeks 1- 4 Pre- vs. Post- PHQ-9 Intervention Weeks 1- 4 𝛸! df p 0.18 1 0.672 14.33 3 0.003 36.94 1 0.001 30.01 3 0.001 34 Table 4 Mean Pre-and Post-Intervention Survey Score by Domains Survey Topic M Knowledge Confidence Attitude Frequency Pre MBC 3.05 3.23 3.71 2.65 Post MBC 3.79 4.08 4.06 3.60 Difference +0.74 +0.85 +0.35 +0.95 Pre OQ 3.69 3.18 3.74 2.78 Post OQ 4.28 4.18 4.32 3.63 Difference +0.59 +1.0 +0.58 +0.85 Pre PHQ-9 3.82 3.48 3.84 2.77 Post PHQ-9 4.34 4.06 4.24 3.64 Difference +0.52 +0.58 +0.4 +0.87 Note: Pre-survey n = 31 and post-survey n = 26. 35 Table 5 Two Sample Independent t-Tests Comparing Pre- and Post-Intervention Scores for MBC, OQ45.2, and PHQ-9 Survey t df p MBC -4.10 55 0.0002 OQ -3.85 55 0.0004 PHQ -2.97 55 0.0044 Note: Pre-survey n = 31 and post-survey n = 26. 36 Table 6 Mean Scores of Feasibility, Usability, and Sustainability Post-Intervention (n = 26) Question Number Domain Likert Scale Count M 1 2 3 4 5 Q28 F 0 3 10 10 3 3.64 Q29 U 0 6 10 6 4 3.44 Q30 U 0 0 9 12 5 4 Q31 U 0 1 7 11 7 4.08 Q32 U 1 0 9 9 7 3.96 Q33 U 1 0 11 9 6 3.8 Q34 U 0 0 11 11 5 3.88 Q35 F 0 0 11 8 7 4 Q36 F 0 0 9 13 4 3.96 Q37 S 1 4 8 8 5 3.6 Overall Mean 3.836 F Mean 3.87 U Mean 3.86 S Mean 3.6 Note. The post-intervention survey (questions #28-#37) included items assessing feasibility (F), usability (U), and sustainability (S). Responses were assigned numerical values: 1 (Strongly Disagree), 2 (Disagree), 3 (Neither Agree nor Disagree), 4 (Agree), and 5 (Strongly Agree). 37 Figure 1 OQ 45.2 and PHQ-9 Compliance Rates 60% 50% 40% 30% 20% 10% 0% Pre Intervention Month Week 1 Week 2 Week 3 OQ 45.2 PHQ-9 Week 4 Post Intevention Month Note. OQ-45.2 compliance increased during the intervention period but returned to baseline postintervention. PHQ-9 compliance showed a more significant improvement, mainly due to the absence of a prior tracking system. However, compliance with both measures declined following the intervention. . 38 Figure 2 Changes in Measured Domains of the Likert Scale Responses from Pre- to Post- Intervention Note. Likert scale survey questions were grouped into four domains: knowledge, confidence, attitude, and frequency of use for MBC, OQ-45.2, and PHQ-9. Mean scores were calculated preand post-intervention. Results showed improvements across all domains for each category. 39 Appendix A Training and Education Presentation 40 41 42 43 44 45 46 47 48 Appendix B Pre-Intervention Survey 49 50 51 52 Appendix C Post-Intervention Survey 53 54 55 56 57 Appendix D Executive Summary 58