Enhancing Childhood Immunization Rates: Implementing an Evidenced-Based Provider Toolkit & Training in Pediatric Primary Care

The World Health Organization identifies vaccine hesitancy as a top ten global health threat. Growing hesitancy has decreased childhood immunization rates, significantly threatening public health and increasing the risk of vaccine-preventable disease outbreaks. The COVID-19 pandemic has exacerbated this issue by amplifying distrust in healthcare providers. At a suburban pediatric clinic in Utah, providers lacked standardized approaches and up-to-date tools for addressing vaccine hesitancy. Despite serving a population with increasing rates of vaccine exemptions, providers felt unprepared to effectively communicate with vaccine-hesitant parents. This quality improvement project utilized the Johns Hopkins Evidence-Based Practice Model. Pre- and post-implementation surveys assessed providers' confidence in addressing vaccine hesitancy. Retrospective chart reviews compared vaccination rates for children aged 0-6 years. Weekly Plan-Do-Study-Act (PDSA) cycles tracked toolkit usage and gathered real-time feedback, enabling refinement. Interventions: Based on evidence-based research, a comprehensive vaccine hesitancy toolkit was developed, incorporating communication strategies and supporting materials. A mixed-media training program, including presentations of key concepts, interactive exercises, and scenario-based practice sessions, was implemented. Six healthcare providers (four Pediatricians, one Family Nurse Practitioner, and one Physician Assistant) participated in the training. Post-intervention, providers reported improvements in communication confidence (33% to 67% feeling "confident") and comfort discussing vaccine risks and benefits (33% to 83%). Implementation assessment demonstrated high acceptability, with 83% of providers agreeing that the toolkit added value. Vaccination rates showed mixed results, with improvements in some age groups (12-month Hepatitis A rates increased by 15% and MMR/Varicella by 23.2%) while others declined significantly (4-6-year-old DTaP/IPV decreased by 62.7% and 4-6-year-old MMRV decreased by 60.3%). Primary barriers were parental vaccine hesitancy, provider time constraints, and ingrained practice habits. This quality improvement project demonstrated the feasibility of implementing an evidence-based vaccine hesitancy toolkit in pediatric primary care. While providers reported increased confidence, the intervention did not consistently increase immunization rates across all age groups, highlighting the need for comprehensive, multi-level approaches to address vaccine hesitancy.

1 Enhancing Childhood Immunization Rates: Implementing an Evidenced-Based Provider Toolkit & Training in Pediatric Primary Care Micah R. Nixon and Djin L. Tay College of Nursing: The University of Utah NURS 7703: DNP Scholarly Project III April 18, 2025 2 Abstract Background: The World Health Organization identifies vaccine hesitancy as a top ten global health threat. Growing hesitancy has decreased childhood immunization rates, significantly threatening public health and increasing the risk of vaccine-preventable disease outbreaks. The COVID-19 pandemic has exacerbated this issue by amplifying distrust in healthcare providers. Local Problem: At a suburban pediatric clinic in Utah, providers lacked standardized approaches and up-to-date tools for addressing vaccine hesitancy. Despite serving a population with increasing rates of vaccine exemptions, providers felt unprepared to effectively communicate with vaccine-hesitant parents. Methods: This quality improvement project utilized the Johns Hopkins Evidence-Based Practice Model. Pre- and post-implementation surveys assessed providers' confidence in addressing vaccine hesitancy. Retrospective chart reviews compared vaccination rates for children aged 0-6 years. Weekly Plan-Do-Study-Act (PDSA) cycles tracked toolkit usage and gathered real-time feedback, enabling refinement. Interventions: Based on evidence-based research, a comprehensive vaccine hesitancy toolkit was developed, incorporating communication strategies and supporting materials. A mixedmedia training program, including presentations of key concepts, interactive exercises, and scenario-based practice sessions, was implemented. Six healthcare providers (four Pediatricians, one Family Nurse Practitioner, and one Physician Assistant) participated in the training. Results: Post-intervention, providers reported improvements in communication confidence (33% to 67% feeling "confident") and comfort discussing vaccine risks and benefits (33% to 83%). Implementation assessment demonstrated high acceptability, with 83% of providers agreeing that the toolkit added value. Vaccination rates showed mixed results, with improvements in some age 3 groups (12-month Hepatitis A rates increased by 15% and MMR/Varicella by 23.2%) while others declined significantly (4–6-year-old DTaP/IPV decreased by 62.7% and 4–6-year-old MMRV decreased by 60.3%). Primary barriers were parental vaccine hesitancy, provider time constraints, and ingrained practice habits. Conclusion: This quality improvement project demonstrated the feasibility of implementing an evidence-based vaccine hesitancy toolkit in pediatric primary care. While providers reported increased confidence, the intervention did not consistently increase immunization rates across all age groups, highlighting the need for comprehensive, multi-level approaches to address vaccine hesitancy. Keywords: Vaccine hesitancy, pediatric primary care, provider communication, quality improvement, evidence-based practice, childhood immunization 4 Enhancing Childhood Immunization Rates: Implementing an Evidence-Based Provider Toolkit & Training in Pediatric Primary Care Problem Description The World Health Organization identifies vaccine hesitancy as a top ten global health threat. Growing vaccine hesitancy, evidenced by decreased routine childhood immunizations, significantly threatens public health (Sharif-Nia et al., 2024; Opel et al., 2011; Olson et al., 2020; Seither et al., 2023). Reduced vaccination rates increase the risk of outbreaks of vaccinepreventable diseases and associated health complications, especially for vulnerable populations (Halim et al., 2020; Dyda et al., 2020). The COVID-19 pandemic has exacerbated this issue, amplifying distrust in healthcare providers (HCPs) and further decreasing vaccination rates (Oduwole et al., 2022; Marshall et al., 2021). Survey data from 2020 revealed that one in four parents in the United States (U.S.) express hesitancy towards childhood vaccinations, and over one-third of U.S. children under 35 months old were not following the recommended childhood immunization schedule (Olson et al., 2020). The escalation of vaccine hesitancy is paralleled by an increase in non-medical school vaccine exemptions, allowing children to enter school without required vaccinations (Olive et al., 2018; Nguyen et al., 2022). Recent data for Utah follows national trends and illustrates a concerning rise in vaccine exemptions, particularly in charter and private schools, indicating an escalating local problem (Utah Department of Health & Human Services, 2024). The rising trend of parental vaccine hesitancy necessitates that primary care providers develop enhanced competencies in conducting crucial and multifaceted vaccine-related conversations. Although evidence-based strategies for addressing vaccine hesitancy exist in 5 recent research, many pediatric providers feel unprepared and under-trained to effectively communicate with hesitant caregivers (Shen & Dubey, 2019). Available Knowledge Research illustrates practical approaches to improve vaccine uptake. First, a presumptive approach in initiating vaccine conversations is three to five times more effective than a participatory approach in increasing vaccine acceptance (Mbaeyi et al., 2020; Williams et al., 2020). Next, strong provider recommendations to vaccine-hesitant parents have improved pediatric vaccine uptake (O'Leary et al., 2024; Opel, 2023). Utilizing motivational interviewing techniques within vaccine conversations has also improved patient-provider communication (Limaye et al., 2021). These findings offer evidence-based strategies to improve provider communication by altering provider-led vaccine-related conversations' delivery and approach. Further, various communication training programs for HCPs have been developed and implemented with positive outcomes. For instance, the "Effective Communication without Confrontation" (ECC) training implemented by Kaiser Permanente Northern California resulted in increased physician comfort and perceived effectiveness in discussing vaccines with hesitant parents (Glanternik et al., 2020). Similarly, the Announce, Inquire, Mirror, Secure (AIMS) Method for Healthy Conversations demonstrated improvements in providers' communication skills and confidence in addressing vaccine-hesitant caregivers (VHC) (Barton et al., 2022). These programs illustrate the potential for targeted provider training to enhance providers’ abilities in addressing vaccine hesitancy. While individual approaches illustrate vaccine rate improvements, research indicates that multi-component interventions yield the most substantial results. Combination approaches often include communication training, updated evidence-based materials, and structured decision- 6 making tools (Rand & Humiston, 2021). Such interventions often encompass a range of strategies, including provider training in advanced communication techniques, development, and implementation of customized resources, integration of provider prompts into electronic health records, and implementation of broader quality improvement initiatives (Rand & Humiston, 2021; MacDonald et al., 2018). Implementing these strategies together strengthens provider communication and increases childhood vaccination rates. Rationale HCPs are essential in addressing vaccine hesitancy, as parents consistently identify them as the most reliable source of vaccine-related information (Ames et al., 2019; Lin et al., 2021; Smith et al., 2017). Although these interactions are vital, providers often remain unaware of proven evidence-based strategies, and clinics frequently lack standardized approaches and current tools for engaging vaccine-hesitant caregivers (Shen & Dubey, 2019; Bianco et al., 2019). This illustrates a need for evidence-based strategies to improve vaccine communication and uptake. This project utilized the Johns Hopkins Evidence-Based Practice (EBP) Model to guide the process of improving provider communication with vaccine-hesitant caregivers (Johns Hopkins Medicine, 2024). The Johns Hopkins EBP Model provides a structured framework for identifying clinical issues, evaluating evidence, and implementing evidence-based changes into practice (Dang et al., 2021). This Model revolves around key components, including inquiry, evidence, and translation, which form an iterative cycle of practice and learning. In the inquiry phase of this project, vaccine hesitancy in pediatric primary care was investigated, and gaps in provider communication with hesitant caregivers were identified. These findings incited a straightforward practice question that directed subsequent phases of the practice model. Next, the 7 evidence phase of the project involved a comprehensive review of current research, which highlighted that training providers with an evidence-based toolkit could improve provider confidence and skills in communicating with vaccine-hesitant parents, ultimately leading to increased childhood vaccination rates (Oduwole et al., 2022). This evidence then guided the translation phase, where a project on implementing an evidence-based vaccine-hesitancy toolkit and training in a clinical setting was developed (Dang et al., 2021). A vital strength of the Johns Hopkins EBP model is its emphasis on continuous learning and reflection, which aligns well with the quality improvement aspect of this project. The implementation process incorporated several Plan-Do-Study-Act (PDSA) cycles, allowing for continuous evaluation and refinement of the intervention (Dang et al., 2021). The Johns Hopkins EBP Model framework guided the project by laying out a step-bystep process, including assessing the clinic's vaccine pre-intervention rates, adapting an evidence-based toolkit, developing a training program, implementing the intervention, and evaluating outcomes. The Model's structured approach, combined with continuous refinement, provided a strong framework for translating evidence into practice and achieving meaningful improvements in provider communication and vaccination rates. Specific Aims The purpose of this Doctor of Nursing Practice (DNP) quality improvement initiative is to implement and evaluate the feasibility, usability, and satisfaction of an evidence-based vaccine hesitancy toolkit and training to increase childhood vaccination rates at a pediatric clinic in Utah. 8 Methods Context A quality improvement project was undertaken to provide up-to-date, multi-component resources and a standardized process for addressing vaccine hesitancy at a single suburban outpatient pediatric clinic in Utah from October 4th, 2024, to January 20th, 2025. This pediatric clinic serves an ethnically and socioeconomically diverse patient population of over 2,000. The project's targeted population includes six HCPs who provide routine childhood immunizations and interact with vaccine-hesitant parents, comprising four Pediatricians (three Medical Doctors and one Doctor of Osteopathic Medicine), one Family Nurse Practitioner (FNP), and one Physician Assistant (PA). This mix of providers offers a comprehensive representation of the typical pediatric primary care team. The clinic employs five registered nurses (RNs), 28 medical assistants (MAs), and five associated staff members. Intervention The intervention consisted of two main components: an adapted vaccine hesitancy toolkit and a comprehensive provider training program. The toolkit was developed based on current evidence-based practices and incorporated up-to-date communication strategies and supporting materials to create a comprehensive provider resource (Appendix A). A mixed-media training program was also developed, including a PowerPoint presentation covering key concepts and communication strategies, interactive exercises and scenario-based practice sessions, handouts for quick reference during patient encounters, and supporting materials for future consultation to support effective toolkit implementation (Appendix B). The toolkit and training materials were designed to be integrated seamlessly into the providers' daily workflow, offering practical resources for real-time use during patient encounters. The training was given at a provider 9 meeting, during which the toolkit contents and evidence-based communication strategies were introduced. Study of the Intervention This DNP quality improvement project utilized a multifaceted approach to assess the impact of the vaccine hesitancy toolkit and training intervention. The primary evaluation method compared pre- and post-intervention childhood vaccination rates at the clinic through a retrospective chart review for children aged 0-6 (Table 1). Additional evaluation methods included pre- and post-implementation surveys given to HCPs to record changes in their confidence and skill in addressing vaccine hesitancy and assessing the feasibility, usability, and satisfaction of the toolkit and training (Appendices C and D). The surveys, completed voluntarily with all questions required, included open- and closed-ended questions about providers' experiences with vaccine-hesitant patients, confidence levels, resource satisfaction, communication strategies, common parental concerns, and communication challenges. Participants' demographic data, including professional background, years of experience, and previous training, was also collected to contextualize the findings (Table 2). At the close of the intervention, structured interviews with providers offered more insight into the intervention's effectiveness and future areas of study (Table 3). Throughout the project, weekly PDSA cycles tracked toolkit usage and gathered real-time feedback, enabling rapid cycle changes and intervention refinement (Table 4). Regular communication with key stakeholders during implementation ensured buy-in and facilitated necessary adjustments. Expected project outcomes included increased childhood vaccination rates, enhanced provider confidence in addressing vaccine hesitancy, and improved communication skills with vaccine-hesitant caregivers. An Executive Summary was written as 10 the final piece of the project to comprehensively capture the intervention's implementation, outcomes, and recommendations (Appendix E). Measures Pre- and post-implementation assessments were developed using REDCap, an online platform for secure survey creation and data management to evaluate the intervention's effectiveness. The project's initial assessment tool was designed to capture baseline data on provider experience with vaccine hesitancy. This comprehensive survey comprised 20 questions: five multiple-choice questions, seven structured items utilizing 4- or 5-point Likert scales, three select-all-that-apply questions, and three open-response inquiries. Items assessed providers’ current vaccine communication techniques, satisfaction with existing clinic resources, and the prevalence of vaccine hesitancy encounters in their practice. Likert scale response options ranged from "disagree" to "agree" on a five-point scale, to "not at all confident/ satisfied" to "confident/ satisfied" on a four-point scale, allowing for nuanced data collection (Appendix C). A post-intervention survey was administered, following the same structure as the initial survey but including new questions to assess the toolkit's effectiveness and evaluate the intervention's feasibility, usability, and provider satisfaction. This portion included six targeted statements, such as "the vaccine hesitancy toolkit integrated seamlessly into my workflow" and "I anticipate continued use of these resources in future patient interactions." Participants selected their level of agreement using a 5-item Likert scale from "disagree" to "agree." Respondents could choose "did not utilize the toolkit," ensuring accurate data collection. The assessment concluded with two open-ended questions, inviting participants to share any implementation challenges and suggest potential improvements. This design facilitated the collection of 11 quantitative metrics and qualitative insights, enabling a comprehensive evaluation of the intervention's efficacy and areas for refinement (Appendix D). An evaluation of the post-implementation vaccination data further assessed the intervention's usability. This critical measure provided objective evidence of the toolkit's impact on vaccination rates, a key indicator of the intervention's effectiveness in real-world clinical settings. The tangible outcomes of improved provider-parent communication were quantified and illustrated by comparing pre- and post-intervention vaccination rates, which indicated whether the toolkit translated into measurable increases in vaccine uptake (Table 1). Analysis This DNP project utilized both descriptive statistics and qualitative content analysis to analyze the impact of the vaccine hesitancy toolkit and training program. The analysis focused on three key areas: vaccination rates, provider surveys, and PDSA cycle feedback. First, descriptive statistics were used to analyze pre- and post-intervention vaccination rates for children aged 0-6 years, comparing rates across different age groups and vaccine types. Descriptive statistics described the changes between pre- and post-intervention vaccination rates, providing quantitative evidence of the intervention's impact on clinical outcomes (Table 1). Next, pre- and post-implementation survey data were analyzed using quantitative and qualitative methods. Descriptive statistics characterized the study sample, including professional roles, years of experience, previous training in vaccine hesitancy communication (Table 2), and Likert-scale responses, measuring provider confidence, satisfaction, and toolkit usability (Table 5 and Figure 1). Analysis of the open-ended questions followed a structured content analysis approach (Tables 3, 6, and 7). Responses were reviewed word by word, categorized, and summarized to identify common themes regarding facilitators, barriers, and suggestions for 12 improving vaccine hesitancy. The satisfaction, feasibility, and usability sections of the postintervention survey and semi-structured interviews were analyzed using descriptive statistics to evaluate the overall effectiveness of the intervention and inform future implementations (Tables 1, 3, 8). A content analysis approach was used, where responses were coded, categorized into themes, and synthesized to identify meaningful patterns. Lastly, weekly PDSA cycles supplied ongoing data throughout the implementation period (Table 4). Qualitative feedback gathered during these cycles was analyzed through content analysis, with notes reviewed line by line to identify emerging patterns and themes. This iterative analysis process informed real-time adjustments to the toolkit and training program. Additionally, toolkit usage patterns were tracked and analyzed to assess implementation uptake and identify potential barriers to implementation. Ethical Considerations This project is quality improvement and not subject to institutional review board oversight from the University of Utah. The project involved routine quality improvement activities, including provider surveys and retrospective analysis of aggregate vaccination data. Provider participation was voluntary, and all survey responses were kept confidential. Patient data was reviewed retrospectively and analyzed in aggregate form only, with no individual patient identifiers collected or stored. There were no conflicts of interest concerning this project, and no external funding sources influenced the design or implementation of the intervention. Results Six providers from a single suburban pediatric clinic participated in this quality improvement project. The sample consisted of an equal number of male and female providers, with 50% being MDs, 17% DOs, 17% NPs, and 17% PAs (Table 2). Half of the providers had 13 over 20 years of practice experience, and only 50% had previously received any kind of targeted vaccine communication training. Pre- and post-intervention vaccination rates were obtained through a retrospective chart review of children aged 0-6 years, with the assistance of the office manager (Table 1). Rates were tracked for routine childhood immunizations, including hepatitis B (Hep B); rotavirus (RV); diphtheria, tetanus, pertussis (DTaP); Haemophilus influenzae type B (Hib), pneumococcal (PCV15/PVC20); inactivated poliovirus (IPV); measles, mumps, rubella (MMR); varicella (VAR); and hepatitis A (Hep A). This pediatric clinic used many combination vaccines for ease of administration, which includes Vaxelis (Hep B, DTaP, Hib, IVP), Pentacel (DTaP, Hib, IPV), Quadracel (DTaP, IPV), Vaxneuvance (PCV15/PCV20), and MMRV (MMR + VAR). Vaccine uptake was measured by age groups 0-1 month, 2 months, 4 months, 6 months, 12 months, 15 months, 18 months, and 4-6 years, and compared to which vaccines were due at each age range. If a child in that specific age group had received all vaccines, they were considered up to date on routine childhood immunizations. Pre-intervention rates were assessed using a 6-month retrospective review from April 3rd, 2024- October 3rd, 2024. Vaccination rates varied by vaccine and age group. Many rates fell below target coverage levels, such as only 35.8% of 1-month-olds receiving the Hepatitis B vaccine and 16.3% of 15-month-olds being up to date on their DTaP series. Post-intervention vaccination rates demonstrated notable changes, though patterns varied by vaccine and age (Table 1 and Figure 2). While some age groups saw improvements, such as a 15% increase in 12-month Hep A rates, others had concerning declines, like a 63% drop in 4–6-year-old Quadracel coverage. Many vaccine rates remained suboptimal post-intervention (Table 1). 14 Provider surveys administered pre- and post-intervention revealed improvements in several key measures (Table 5). Provider confidence in communication skills for addressing vaccine hesitancy increased, with 67% feeling "confident" post-intervention compared to 33% initially. Satisfaction with vaccine hesitancy knowledge (0% to 50%), clinic resources (0% to 50%), and organizational support (0% to 17%) also rose significantly. More providers strongly agreed post-intervention that they were comfortable discussing vaccine risks and benefits (33% to 83%). Qualitative analysis of survey-free responses identified themes in providers' approaches to vaccine conversations, including addressing specific concerns, providing education, building trust, and sharing personal experiences (Table 7). Parental misinformation and ingrained beliefs emerged as major communication challenges. Post-intervention assessments demonstrated the high acceptability of the toolkit, with 83% agreeing that it added value and all providers expressing satisfaction with its effectiveness (Table 3). However, time constraints and consistently remembering to use the tools were noted as barriers to implementation. One provider shared, "It was all too easy to slide back to old patterns and keep doing what I've always done." Structured interviews reinforced these findings, with providers valuing the toolkit's role in building confidence but facing challenges with time management and implementation (Table 3). A provider stated, "It's frustrating when you spend an extra 5 or 10 minutes at the end of the visit and half of the time they don't change their mind. And we're not reimbursed in any way for that extra time." PDSA cycles facilitated continuous intervention refinement based on provider feedback (Table 4). Key adjustments included creating quick reference materials, sending digital resources and reminders, and developing patient education handouts (Appendices F and G). 15 In summary, implementing the vaccine hesitancy toolkit significantly improved provider confidence and communication skills. However, parental vaccine hesitance, time constraints, and ingrained provider habits posed barriers to its consistent use. Vaccination rates saw mixed results, indicating a need for further study and ongoing quality improvement efforts. Discussion Summary The primary aim of this quality improvement project was to implement and evaluate an evidence-based vaccine hesitancy toolkit and training program to increase childhood vaccination rates at a pediatric clinic in Utah. Key findings demonstrated that the intervention significantly improved provider confidence and communication skills when addressing vaccine hesitancy. Pre- and post-intervention surveys revealed increases in providers' perceived ability to effectively communicate with vaccine-hesitant parents, satisfaction with toolkit resources, and comfort discussing vaccine risks and benefits. Qualitative analysis of provider interviews and free-response survey questions reinforced these findings, with themes of enhanced confidence and appreciation for the structured approach to vaccine conversations emerging. These results align with the project's rationale, which emphasizes the vital role of healthcare providers in addressing vaccine hesitancy and the need for evidence-based strategies to improve vaccine communication and uptake. However, the project highlighted notable barriers to the toolkit's consistent implementation, particularly time constraints and the difficulty of changing ingrained habits. Vaccination rates, a key outcome measure, showed mixed results post-intervention, with some age groups demonstrating improvements while others showed concerning declines. Many rates remained suboptimal, underscoring the persistent challenge of vaccine hesitancy and the need for 16 ongoing quality improvement efforts. A particular strength of the project was its use of the Johns Hopkins EBP Model to guide the systematic process of identifying the problem, evaluating evidence, and implementing change. The iterative PDSA cycles facilitated continuous intervention refinement based on provider feedback. Additionally, the comprehensive nature of the toolkit and training program was another strength of this Quality Improvement project, as it incorporated a range of evidence-based communication strategies and resources. Interpretation The results of this quality improvement project aligned with the existing literature on addressing vaccine hesitancy in pediatric primary care settings. Implementing an evidence-based vaccine hesitancy toolkit and training program resulted in improvements in provider confidence and communication skills, which is consistent with findings from other studies on the effectiveness of targeted provider education interventions (Barton et al., 2022; Glanternik et al., 2020; O'Leary et al., 2024). The project's focus on equipping providers with specific communication strategies including presumptive language, motivational interviewing, and tailored vaccine information reflects best practices identified in the literature (Opel, 2023; Williams et al., 2020). However, the project also highlighted the persistent challenge of translating improved provider skills into measurable changes in vaccination rates. Despite providers reporting increased confidence and positive perceptions of the toolkit, post-intervention vaccination rates showed mixed results, with some age groups and vaccines improving while others had concerning declines. The dramatic decrease in 4–6-year vaccination rates (63% drop in Quadracel coverage) was particularly unexpected. Several factors may have contributed to this notable variance in vaccination rates. First, the pre-intervention period (April-October 2024) 17 coincided with the back-to-school timeframe when school-required immunizations typically drive higher vaccination rates, particularly for the 4–6-year age group. Utah's immunization requirements mandate that students enrolling in kindergarten through sixth grade must be immunized according to ACIP recommendations before school entry, creating a seasonal surge in vaccination rates not captured in the post-intervention period (October-January) (Utah Department of Health & Human Services, 2024). This seasonality effect may have artificially inflated the pre-intervention baseline, making the post-intervention decline appear more substantial. Additionally, the variability in results may be partially attributed to the data collection timeframe. The post-implementation analysis ended on January 20th, 2025, which excluded potential vaccinations administered during the final 11 days of the month. Children seen during this period who subsequently received vaccinations were not captured in the analysis, potentially underrepresenting actual vaccination rates. This timing limitation could explain some observed declines in certain age groups. This discrepancy between provider-level outcomes and patientlevel vaccination behaviors has also been noted in other studies and underscores the complex, multi-factorial nature of vaccine hesitancy (Glanternik et al., 2020). While provider communication is crucial, additional patient, family, and system-level interventions are likely needed to achieve optimal vaccination rates. The project had a notable impact on the participating pediatric clinic and providers. Clinic leadership and staff were engaged in the implementation process, and providers expressed appreciation for the practical resources and structured approach to vaccine conversations. The toolkit and training addressed a key area of need identified by providers and could improve the quality of vaccine discussions and patient-provider relationships. Parental misinformation and 18 ingrained beliefs emerged as major communication challenges in pre- and post-intervention assessments (Tables 3 and 6). While the toolkit provided strategies to address common misconceptions through evidence-based resources and communication techniques, further refinement is needed to counter deeply held parental beliefs and the influence of misinformation from social media and other sources. However, the project also revealed barriers to consistent use of the toolkit, particularly time constraints in busy clinical schedules, highlighting the importance of finding ways to integrate vaccine hesitancy interventions efficiently into existing workflows to ensure sustainability. The project required upfront time investment for toolkit development and provider training. However, the resulting resources are highly scalable and can be disseminated to additional clinics with minimal additional costs (Appendix E). The ongoing resource need is additional time dedicated to engaging in the type of vaccine conversations recommended in the toolkit. While this may involve some opportunity costs of seeing fewer patients during the clinic day, the potential public health benefits of improving childhood vaccination rates outweigh these costs. Additionally, providers may save time by proactively addressing vaccine concerns that would be otherwise spent on repeated vaccine discussions or managing vaccine-preventable illnesses. Overall, the project demonstrated the feasibility and acceptability of implementing a vaccine hesitancy toolkit in a pediatric setting. Provider satisfaction with the intervention was high, and clinic leadership expressed interest in continuing to use and expand the toolkit. However, the project's mixed effects on vaccination rates suggest that additional strategies may be needed to impact hesitancy and improve uptake. Future iterations could consider integrating the toolkit into the electronic health record to enhance sustainability. While addressing vaccine 19 hesitancy remains a complex challenge, this project offers a promising foundation for empowering providers to have more effective vaccine conversations and improve protection against preventable diseases for pediatric patients. Limitations This quality improvement project had several limitations that may affect the generalizability of the findings. First, the sample size was small, with only six providers participating from a single suburban pediatric clinic. The characteristics of the provider sample may not be representative of the broader diversity of pediatric primary care providers in other settings. Additionally, the patient population served by this clinic may differ in demographic and socioeconomic factors from other regions, limiting the applicability of the findings to other contexts. The project's ability to accurately assess changes in vaccination rates was limited by several factors related to the measurement and analysis of this outcome. The frequent intervals at which different vaccines are due and multiple vaccines at each timestep made it challenging to capture a comprehensive picture of vaccine uptake. The sampling of patients within specific age ranges and dates may not have encompassed all eligible patients, as some may have had upcoming appointments shortly outside the study period where they would have received due vaccines. This could have contributed to an underestimation of accurate vaccination rates. Furthermore, using a pre-post design without a control group introduces the possibility of confounding factors influencing vaccination rates apart from the intervention. While efforts were made to minimize limitations, such as using multiple age ranges and time points to assess vaccination rates, the constraints of the project's scope and setting posed inherent challenges to obtaining fully representative and generalizable data. Future studies with 20 larger, more diverse samples and robust designs are needed to further evaluate the effectiveness of vaccine hesitancy toolkits in improving childhood vaccination rates across varied populations and settings. Conclusions This quality improvement project demonstrated the feasibility and value of implementing an evidence-based vaccine hesitancy toolkit and training program in a pediatric primary care setting. Clinicians reported increased confidence and enhanced communication skills when addressing parental vaccine concerns. However, despite improved provider engagement, the intervention did not lead to a measurable increase in routine childhood immunization rates, highlighting vaccine uptake's complex, multifactorial nature. These findings emphasize the critical role of provider education in mitigating vaccine hesitancy while underscoring the need for broader, systemic interventions to improve immunization rates. Future efforts could focus on integrating toolkit components into electronic health records or adopting a clinic-wide approach involving all staff levels, not just the providers. Future research should assess the long-term impact of this approach, its effectiveness across diverse populations, and additional strategies to address vaccine hesitancy at the community level. This project reinforces the vital role of advanced nursing practice in developing and implementing evidence-based strategies to strengthen pediatric preventive care. 21 Acknowledgments I want to thank Dr. Djin Tay, my project chair, for her guidance and expertise throughout this project. Special appreciation goes to Pam Steadman, the clinic office manager, for her operational support and continued communication, which allowed the implementation of my project to run smoothly. I am also grateful to all participating providers at the clinic, particularly Melissa Kendall, MD, who served as content expert. I also acknowledge Dr. Eli Iacob, Ph.D., MS, and the University of Utah College of Nursing faculty for their supervision and assistance with project implementation and manuscript completion. 22 References Bianco, A., Mascaro, V., Zucco, R., & Pavia, M. (2019). Parent perspectives on childhood vaccination: How to deal with vaccine hesitancy and refusal? Vaccine, 37(7), 984–990. https://doi.org/10.1016/j.vaccine.2018.12.062 Dang, D., Deatholt, S. L., Bissett, K., Ascenzi, J., & Whalen, M. (2021). Johns Hopkins evidence-based practice for nurses and healthcare professionals: Model and guidelines (4th ed.). Sigma Theta Tau International. Dyda, A., King, C., Dey, A., Leask, J., & Dunn, A. G. (2020). A systematic review of studies that measure parental vaccine attitudes and beliefs in childhood vaccination. BMC Public Health, 20(1). https://doi.org/10.1186/s12889-020-09327-8 Halim, H. A., Abdul-Razak, S., Yasin, M. M., & Isa, M. R. (2019). Validation study of the parent attitudes about childhood vaccines (PACV) questionnaire: The Malay version. Human Vaccines & Immunotherapeutics, 16(5), 1040–1049. https://doi.org/10.1080/21645515.2019.1674112 Johns Hopkins Medicine. (2024). Evidence-based practice. Johns Hopkins Medicine. https://www.hopkinsmedicine.org/nursing/center-nursing-inquiry/nursinginquiry/evidence-based-practice Limaye, R. J., Opel, D. J., Dempsey, A., Ellingson, M., Spina, C., Omer, S. B., Dudley, M. Z., Salmon, D. A., & O'Leary, S. T. (2021). Communicating with vaccine-hesitant parents: A narrative review. Academic Pediatrics, 21(4), S24–S29. https://doi.org/10.1016/j.acap.2021.01.018 23 Marshall, S., Moore, A. C., Sahm, L. J., & Fleming, A. (2021). Parent attitudes about childhood vaccines: Point prevalence survey of vaccine hesitancy in an Irish population. Pharmacy, 9(4), 188. https://doi.org/10.3390/pharmacy9040188 Mbaeyi, S., Fisher, A., & Cohn, A. (2020). Strengthening vaccine confidence and acceptance in the pediatric provider office. Pediatric Annals, 49(12). https://doi.org/10.3928/1938235920201115-02 Nguyen, K. H., Srivastav, A., Lindley, M. C., Fisher, A., Kim, D., Greby, S. M., Lee, J., & Singleton, J. A. (2022). Parental vaccine hesitancy and association with childhood diphtheria, tetanus toxoid, and acellular pertussis; measles, mumps, and rubella; rotavirus; and combined 7-series vaccination. American Journal of Preventive Medicine, 62(3), 367–376. https://doi.org/10.1016/j.amepre.2021.08.015 Oduwole, E. O., Pienaar, E. D., Mahomed, H., & Wiysonge, C. S. (2022). Overview of tools and measures investigating vaccine hesitancy in a ten-year period: A scoping review. Vaccines, 10(8), 1198. https://doi.org/10.3390/vaccines10081198 Olive, J. K., Hotez, P. J., Damania, A., & Nolan, M. S. (2018). The state of the antivaccine movement in the United States: A focused examination of nonmedical exemptions in states and counties. PLoS Medicine, 15(6), e1002578. https://doi.org/10.1371/journal.pmed.1002578 Olson, O., Berry, C., & Kumar, N. (2020). Addressing parental vaccine hesitancy towards childhood vaccines in the United States: A systematic literature review of communication interventions and strategies. Vaccines, 8(4), 590. https://doi.org/10.3390/vaccines8040590 24 Opel, D. J., Henrikson, N., Lepere, K., Hawkes, R., Zhou, C., Dunn, J., & Taylor, J. A. (2019). Previsit screening for parental vaccine hesitancy: A cluster randomized trial. Pediatrics, 144(5). https://doi.org/10.1542/peds.2019-0802 Seither, R., Yusuf, O. B., Dramann, D., Calhoun, K., Mugerwa-Kasujja, A., & Knighton, C. L. (2023). Coverage with selected vaccines and exemption from school vaccine requirements among children in kindergarten — United States, 2022–23 school year. Morbidity and Mortality Weekly Report, 72(45), 1217–1224. https://doi.org/10.15585/mmwr.mm7245a2 Sharif-Nia, H., She, L., Allen, K., Marôco, J., Kaur, H., Arslan, G., Gorgulu, O., Osborne, J. W., Rahmatpour, P., & Fomani, F. K. (2024). Parental hesitancy toward children vaccination: A multi-country psychometric and predictive study. BMC Public Health, 24(1). https://doi.org/10.1186/s12889-024-18806-1 Shen, S. C., & Dubey, V. (2019). Addressing vaccine hesitancy: Clinical guidance for primary care physicians working with parents. Canadian Family Physician, 65(3), 175-181. https://www.cfp.ca/content/65/3/175 Smith, L. E., Amlôt, R., Weinman, J., Yiend, J., & Rubin, G. J. (2017). A systematic review of factors affecting vaccine uptake in young children. Vaccine, 35(45), 6059–6069. https://doi-org.ezproxy.lib.utah.edu/10.1016/j.vaccine.2017.09.046 Utah Department of Health & Human Services. (2024). Utah immunization rule for students. Utah Department of Health & Human Services. https://immunize.utah.gov/utahimmunization-rule/ 25 Williams, J. T., O’Leary, S. T., & Nussbaum, A. M. (2020). Caring for the vaccine-hesitant family: Evidence-based alternatives to dismissal. The Journal of Pediatrics, 224, 137– 140. https://doi.org/10.1016/j.jpeds.2020.05.029 26 Tables and Figures Table 1 Comparing Pre/Post Intervention Clinic Vaccination Rates by Age Age Group Vaccine 0-1 Month Hepatitis B 2 Months Vaxelis Rotavirus Vaxneuvance 4 Months Vaxelis Rotavirus Vaxneuvance 6 Months Vaxelis Rotavirus Vaxneuvance 12 Months Hepatitis A Vaxneuvance MMRV Haemophilus 15 Months influenzae B DTaP Pentacel 18-23 Hepatitis A Months 4-6 Years Quadracel MMRV Pre-Intervention Rates Post-Intervention Rates 44/123 (35.8%) 16/37 (43.2%) 75/92 (81.5%) 36/48 (75.0%) 74/92 (80.4%) 38/48 (79.2%) 78/92 (84.8%) 38/48 (79.2%) 37/37 (100%) 54/67 (80.6%) 26/37 (70.3%) 54/67 (80.6%) 33/37 (89.2%) 48/67 (71.6%) 61/77 (79.2%) 44/64 (68.8%) 58/77 (75.3%) 41/64 (64.1%) 68/77 (88.3%) 44/64 (68.8%) 54/75 (72.0%) 40/46 (87.0%) 68/75 (90.7%) 40/46 (87.0%) 38/75 (50.7%) 34/46 (73.9%) Change in Rates +7.4% -6.5% -1.2% -5.6% -19.4% +10.3% -17.6% -10.4% -11.2% -19.5% +15.0% -3.7% +23.2% 18/92 (19.6%) 7/54 (13.0%) -6.6% 15/92 (16.3%) 40/92 (43.5%) 4/54 (7.4%) 13/54 (24.1%) -8.9% -19.4% 50/56 (89.3%) 54/54 (100%) +10.7% 1000/1508 (66.3%) 966/1508 (64.0%) 60/1651 (3.6%) 61/1651 (3.7%) -62.7% -60.3% 27 Table 2 Demographics of Needs Assessment Participants Presurvey Characteristic Gender Male Female Professional Role M.D. D.O. N.P. P.A. Years Practicing Less than 5 years 5-10 years 11-20 years More than 20 years Employment Status Full-Time Part-Time Previous Vaccine Training Yes No Medical providers N=6 (%) 3 (50%) 3 (50%) 3 (50%) 1 (17%) 1 (17%) 1 (17%) 1 (17%) 1 (17%) 1 (17%) 3 (50%) 5 (83%) 1 (17%) 3 (50%) 3 (50%) 28 Table 3 Post-Intervention Assessment of Feasibility, Usability, and Acceptability of Toolkit Through Provider Survey Responses and Provider Interviews Category Question Response Options Not at all feasible How feasible is it to incorporate Somewhat feasible the toolkit into daily clinical practice? Feasible Disagree Somewhat disagree Neither agree nor disagree Feasibility The toolkit integrated seamlessly into my workflow Somewhat agree Agree Barriers to Implementation (Free response) Usability How easy is it to navigate and use the components? Acceptability Added value to practice and improved patient care n (%) (N=6) Supporting Quotes 0 (0%) “I have enjoyed integrating the 3 toolkit into my work. The handout (50%) with the QR codes has been 3 particularly helpful for quick (50%) access to vaccine materials.” 0 (0%) 0 (0%) "I think the resources are helpful, I 3 just feel like I’m too busy to (50%) incorporate all these things into my 1 15-minute appointment slot;” (17%) “Limited time with patients makes it challenging to effectively discuss vaccines when parents express 2 hesitancy. It is so easy to get (33%) behind in your schedule when you take the time to talk about it.” “The parents continue to be a challenge for me. They come with Parental Resistance, a barrage of info they got on social Time constraints, 3 media and from their friends and Remembering to (50%) neighbors that they think is true. Implement They feel like I am trying to trick them, which sometimes makes me not want to push or even try to change their mind.” Poor adherence to 2 toolkit use (33%) "Remembering to use it;” “It was 1 hard to remember to use the new A little difficult (17%) strategies. I have my established 3 approach I’ve been using for years, Mostly easy (50%) and it was hard to remember to 2 change that.” Easy (33%) Disagree 0 (0%) “I have reached for the toolkit you Somewhat disagree 0 (0%) provided many times to access Neither agree nor 1 quick resources. I know I need to disagree (17%) spend more time looking at it, but 29 Category Question Satisfaction with toolkit's effectiveness Anticipate continued use of resources Suggested Improvements to Toolkit (Free response) n (%) (N=6) Supporting Quotes 3 you’ve linked a lot of good stuff Somewhat agree (50%) that my patients ask me for.” 2 Agree (33%) Not at all satisfied 0 (0%) “I liked having a standard flow to A little dissatisfied 0 (0%) these conversations in the back of 3 my mind. It made me feel less Somewhat satisfied (50%) nervous walking into the room 3 because I knew I had a strategy in Satisfied (50%) place in case I needed it.” Disagree 0 (0%) Somewhat disagree 0 (0%) “I enjoyed the resources included on the handouts with the QR codes. Neither agree nor 2 It was nice to have something that disagree (33%) would take me right there when I 2 needed it. I'm going to laminate the Somewhat agree (33%) card with QR codes and put it in 2 the room.” Agree (33%) Satisfied with 2 "I think it's great as it is now!" current toolkit (33%) "I would love for it to be integrated Additional resources 1 into the EMR. I would use it way needed (17%) more often if it was only one click away.” Limited experience 3 "Can't really comment, didn't have to comment (50%) enough experience with it" Response Options Note: Provider Interview Questions: 1. What were the main barriers or challenges you encountered when trying to implement these strategies in your daily practice? 2. Which resources have you continued to use regularly? 3. How could the toolkit be redesigned to be more effective or user-friendly? 4. How has having a structured approach to vaccine conversations impacted your confidence in vaccine discussions? 5. Which evidence-based communication strategy did you find most helpful and why? 6. Did you find the in-person training effective and why? 30 Table 4 PDSA Cycle Summary Analysis PDSA Summary of Engagement Suggestions for Cycle Themes Changes Implemented with QI Team Improvement # Presented training at the clinic to providers. Answered questions about toolkit - Updated toolkit with Formatting changes usage. Training was wellprovider suggestions for toolkit and received with positive Provider-Centered - Sent follow-up email 1 suggested provider interactions. One Resource Development thanking participants supplemental patient provider noted: "I can tell - Distributed eight-week materials this toolkit is going to be training schedule very helpful in my vaccine conversations." Compiled supplemental education materials for provider usage. Sent email with educational content to - Continued sending Knowledge No specific providers. Two providers reminder emails 2 Dissemination and improvement provided feedback, - Shared supplemental data Engagement suggestions received appreciating the shared data and educational articles and expressing intent to use it in their vaccine conversations. - Sent comprehensive Conducted Q&A clinic reminder email to all session to answer provider providers questions and reinforce - Created paper handout for No formal toolkit training. Engaged Implementation Barrier desk placement to remind 3 improvement with three providers, with Identification providers of the suggestions two expressing difficulties intervention remembering to use the - Provided quick summary toolkit consistently. of training and toolkit in email Presented at the clinic with donuts to facilitate feedback. - Adjusted toolkit based on Interacted with two providers Develop a quick provider feedback between their patient visits. Time Management and reference method - Created concise exam 4 Providers found materials Resource Utilization for busy clinical room reference guide helpful but highlighted time environments - Developed small graphic constraints in patient visits as for exam rooms a key implementation challenge. 31 PDSA Summary of Engagement Suggestions for Cycle Themes Changes Implemented with QI Team Improvement # Visited clinic and distributed new supplemental materials - Sent follow-up email with to support toolkit. Left two PDF version of handouts patient handouts with No specific Patient Education and - Provided detailed 5 vaccine Q&A sheets and QR improvement Resource Accessibility instructions on handout use codes on providers' desks. suggestions - Shared links to all Three providers later referenced web resources confirmed using and appreciating the materials. Prepared and sent - Recorded electronic informative email to Continuous Learning One provider version of training providers. Developed new 6 and Resource requested electronic - Distributed digital training resources to support Development training version resource to interested implementation. Refreshed providers training materials. Presented final clinic session to prepare for exit - Sent follow-up email interviews. Finalized Timing reminding providers Project Evaluation and 7 materials for provider recommendations - Facilitated survey Closure sharing. Developed and for final interviews completion distributed post- Scheduled exit interviews implementation survey. Concluded project by collecting final results and - Conducted comprehensive feedback. Providers exit interviews expressed gratitude, No specific Project Impact - Delivered thank you 8 highlighting positive aspects improvement Assessment cookies and notes such as standardized suggestions - Collected final provider approach to vaccine hesitant feedback conversations and easily accessible resources. 32 Table 5 Comparing Pre- & Post-intervention Survey Likert Scores Pre N=6 (%) Post N=6 (%) Provider Confidence How confident are you with your communication skills in addressing vaccine-hesitant parents? Not at all confident A little confident Somewhat confident Confident Missing How confident are you with your ability to effectively address common concerns raised by vaccine-hesitant parents? Not at all confident A little confident Somewhat confident Confident Missing Provider Satisfaction How satisfied are you with your current level of knowledge and understanding of strategies for communicating with vaccine-hesitant parents? Not at all satisfied A little dissatisfied Somewhat satisfied Satisfied Missing How satisfied are you with the resources and tools currently available to you for addressing vaccine hesitancy? Not at all satisfied A little dissatisfied Somewhat satisfied Satisfied Missing How satisfied are you with the support and guidance provided by your clinic / organization for addressing vaccine hesitancy? Not at all satisfied A little dissatisfied Somewhat satisfied Satisfied Missing 0 (0%) 0 (0%) 4 (67%) 2 (33%) 0 (0%) 0 (0%) 0 (0%) 2 (33%) 4 (67%) 0 (0%) 0 (0%) 1 (17%) 3 (50%) 2 (33%) 0 (0%) 0 (0%) 0 (0%) 3 (50%) 3 (50%) 0 (0%) 1 (17%) 0 (0%) 5 (83%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 3 (50%) 3 (50%) 0 (0%) 1 (17%) 4 (67%) 1 (17%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 3 (50%) 3 (50%) 0 (0%) 0 (0%) 4 (67%) 2 (33%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5 (83%) 1 (17%) 0 (0%) 33 Provider Agreement I am comfortable discussing the risks and benefits of vaccines with vaccine-hesitant parents. Disagree Somewhat disagree Neither agree nor disagree Somewhat agree Agree Missing I believe that addressing vaccine hesitancy is an important part of my role as a healthcare provider. Disagree Somewhat disagree Neither agree nor disagree Somewhat agree Agree Missing 0 (0%) 0 (0%) 1 (17%) 3 (50%) 2 (33%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 1 (17%) 5 (83%) 0 (0%) 0 (0%) 0 (0%) 1 (17%) 0 (0%) 5 (83%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 1 (17%) 5 (83%) 0 (0%) 34 Table 6 Pre-Intervention Assessment of Feasibility, Usability, and Acceptability to Inform Toolkit Through Provider Survey Responses Category Question Proportion of vaccinehesitant patients encountered Most common concerns raised by vaccine-hesitant parents Feasibility & Baseline Barriers Current communication challenges Usability Desired toolkit features Acceptability n (%) Supporting Quotes (n=6) 2 0-1 out of 10 patients (33%) 4 2-3 out of 10 patients (67%) 4-5 out of 10 patients 0 (0%) 6-10 out of 10 patients 0 (0%) Concerns about overloading 6 immune system (100%) 5 Safety concerns (83%) Fear of developmental 4 implications (67%) Perception that vaccines are 4 unnecessary (67%) Concerns about long-term side 4 effects (67%) Misinformation from social 4 media (67%) Preference for "natural" 3 lifestyle (50%) Concerns about vaccine 3 schedule (50%) 2 Concerns about ingredients (33%) 2 Fear of needles or pain (33%) Religious objections 0 (0%) 3 "When they've made up their mind Pre-existing beliefs (50%) it's hard to change it" 2 "The internet has made some of Misinformation influence (33%) them very resistant" 1 "They seem to see providers as the Provider-patient relationship (17%) enemy on this issue" 3 "Simple graphics to show safety and Educational resources (50%) effectiveness" 2 "How to help people see logical Communication strategies (33%) fallacies" 1 "Specific vaccines bullet point to Specific concern addressing (17%) why to get them based on their fear" Response Options 35 Table 7 Thematic Analysis of Pre/Post Intervention Provider Survey Free Responses Question Theme Representative Quotes “ “ "Find out what their concerns are;" "Ask about concerns or reasons why they are How do you approach hesitant" the topic of vaccinations "Provide education and references;" Providing with caregivers who "Provide studies and information to read Education/Evidence express hesitancy? more" "Build a relationship of trust;" "Show an Building interest in their concerns and validate their Trust/Relationship role as advocates for their children" "Express my personal experiences and Sharing Personal confidence in them;" "Share with them Experience what I feel about vaccines" Offering Modified "If they are hesitant, I will try to talk them Schedules into doing a modified, slower schedule" "Make a clear and strong Strong recommendation;" "I will talk and Recommendation recommend that they give vaccines" Identifying Specific Concerns Note: Themes identified from combined pre- and post-intervention responses. Individual responses may contain multiple themes. Frequency (N=6) 5 (83%) 4 (67%) 3 (50%) 3 (50%) 2 (33%) 2 (33%) 36 Table 8 Qualitative Analysis of Pre/Post Select All that Apply Survey Responses Strategies Used to Address Vaccine Hesitancy Pre Pre-Frequency Post Post-Frequency Strategy Change (N=6) % (N=6) % Presumptive Approach 3 50% 5 83% +33% Strong Provider Recommendation 3 50% 4 67% +17% Motivational Interviewing 3 50% 4 67% +17% Providing Educational Materials 4 67% 2 33% -34% Addressing Specific Concerns and Myths 6 100% 6 100% 0% Offering Additional Consultation Time 1 17% 1 17% 0% Implementing Reminder Systems 2 33% 3 50% +17% Tailoring Communication to the 3 50% 6 100% +50% Individual Resources Used to Support Vaccination Conversations Pre Pre-Frequency Post Post-Frequency Resource Change (N=6) % (N=6) % Vaccine Hesitancy Toolkit 0 0% 3 50% +50% Information from CDC/WHO 5 83% 6 100% +17% Medical Journals 1 17% 1 17% 0% Textbooks 1 17% 1 17% 0% Education Materials from Manufacturers 2 33% 2 33% 0% Help from Colleagues 1 17% 2 33% +16% Note: Percentages based on N=6 providers. Options with zero responses in both pre and post periods were excluded from the analyses. 37 Figure 1 Pre- and Post-intervention Provider Survey Response Comparison N=6 38 Figure 2 Comparing Pre/Post Intervention Clinic Vaccination Rates by Age Figure 2: Clinic Vaccination Rates by Child Age 39 Appendix A Vaccine Communication Toolkit 40 Appendix B Implementation Provider Training 41 42 43 Appendix C Pre-Intervention Provider Survey 44 45 46 Appendix D Post-Intervention Provider Survey 47 48 49 Appendix E Executive Summary 50 Appendix F Vaccine Information Infographics 51 52 Appendix G Standard Approach Infographic