An Assessment Toolkit for the University of Utah Ghana Study Abroad Program

Description: The purpose of this project was to identify outcomes of nursing study abroad, to identify tools to measure those outcomes, and to formulate a toolkit for the University of Utah College of Nursing Ghana study abroad (UUCONGSA) program to utilize for measurement of identified outcomes. Problem Statement: Many nursing study abroad (SA) programs do not evaluate outcomes. This is problematic in terms of meeting the standards of higher education and evidence-based practice, and in gaining administrative support. The UUCONGSA program did not evaluate outcomes. It was unknown whether reliable, validated tools were available to evaluate the outcomes that were identified in the literature. Objectives: The objectives of the project were to: 1. Describe the purpose, objectives, outcomes, and assessment of SA programs in nursing. 2. Identify assessment tools for identified SA outcomes. 3. Formulate a toolkit for UUCONGSA program. 5. Disseminate the toolkit to the director of UUCONGSA program, and to peers through professional poster presentation. 6. Disseminate findings to larger audience through poster or podium presentation. Literature Review: Study abroad has become common in higher education, partly due to increased focus on internationalization. The literature describes positive effects on participants and goals may be best met by international experiences. Outcomes in nursing SA are: increased cultural competence, transformative learning, development of global citizenship, personal and professional growth, and positive impacts on nursing practice. However, measurement of these outcomes has been limited. This problem was the basis for this project. Implementation & Evaluation: An extensive literature review was conducted to identify the purpose, objectives, outcomes, and assessment of SA. Tools were identified by literature review and internet search. The Consortium of Universities for Global Health (CUGH) website and the internet were searched to identify 10 colleges of nursing (CON) that were members of CUGH and that had SA programs. A survey was developed and sent to the directors of the identified study abroad programs and to SA directors at Brigham Young University and Westminster College. The survey determined if each CON measured outcomes, and identified the tools that were used to so do. Identified tools were evaluated against inclusion criteria, reviewed for validity, and appropriate tools were incorporated into a toolkit. The toolkit was reviewed by the UUCONGSA study abroad director and content experts for relevance, format and usability. The poster was accepted for presentation at the 2017 CUGH conference. The toolkit was sent to four survey respondents who had requested a copy and was provided to University of Utah CON for use with the global health grant.

Running head: AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH An Assessment Toolkit for the University of Utah Ghana Study Abroad Program Susan Kirby University of Utah In Partial Requirement for the Doctor of Nursing Practice 1 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 2 Executive Summary Description: The purpose of this project was to identify outcomes of nursing study abroad, to identify tools to measure those outcomes, and to formulate a toolkit for the University of Utah College of Nursing Ghana study abroad (UUCONGSA) program to utilize for measurement of identified outcomes. Problem Statement: Many nursing study abroad (SA) programs do not evaluate outcomes. This is problematic in terms of meeting the standards of higher education and evidence-based practice, and in gaining administrative support. The UUCONGSA program did not evaluate outcomes. It was unknown whether reliable, validated tools were available to evaluate the outcomes that were identified in the literature. Objectives: The objectives of the project were to: 1. Describe the purpose, objectives, outcomes, and assessment of SA programs in nursing. 2. Identify assessment tools for identified SA outcomes. 3. Formulate a toolkit for UUCONGSA program. 5. Disseminate the toolkit to the director of UUCONGSA program, and to peers through professional poster presentation. 6. Disseminate findings to larger audience through poster or podium presentation. Literature Review: Study abroad has become common in higher education, partly due to increased focus on internationalization. The literature describes positive effects on participants and goals may be best met by international experiences. Outcomes in nursing SA are: increased cultural competence, transformative learning, development of global citizenship, personal and professional growth, and positive impacts on nursing practice. However, measurement of these outcomes has been limited. This problem was the basis for this project. Implementation & Evaluation: An extensive literature review was conducted to identify the purpose, objectives, outcomes, and assessment of SA. Tools were identified by literature review and internet search. The Consortium of Universities for Global Health (CUGH) website and the internet were searched to identify 10 colleges of nursing (CON) that were members of CUGH and that had SA programs. A survey was developed and sent to the directors of the identified study abroad programs and to SA directors at Brigham Young University and Westminster College. The survey determined if each CON measured outcomes, and identified the tools that were used to so do. Identified tools were evaluated against inclusion criteria, reviewed for validity, and appropriate tools were incorporated into a toolkit. The toolkit was reviewed by the UUCONGSA study abroad director and content experts for relevance, format and usability. The poster was accepted for presentation at the 2017 CUGH conference. The toolkit was sent to four survey respondents who had requested a copy and was provided to University of Utah CON for use with the global health grant. Result: Tools were identified for a portion of identified outcomes and a toolkit was formulated. However, for some outcomes there were no tools that fit the inclusion criteria. Project chair: Gillian Tufts DNP APRN CFNP; Patricia Murphy CNM DrPH FACNM FAAN; Content Experts: Beth Laux MPA, Sheri Palmer DNP APRN AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 3 Table of Contents EXECUTIVE SUMMARY ............................................................................................................2 ACKNOWLEDGMENTS...............................................................................................................5 1. INTRODUCTION Problem Statement………………………………………………………………...6 Clinical Significance & Policy Implications……………………………………...7 Objectives…………………………………………………………………………9 Search Terms & Strategy………………………………………………………….9 2. LITERATURE REVIEW What Is Study Abroad?..........................................................................................10 Study Abroad in Higher Education…………………………....…………………11 Study Abroad in Nursing……………………………………..………………….13 Outcomes Assessment in Study Abroad…………………..………….………….14 Summary of Literature Reviews About Study Abroad………….………………..16 Nursing Study Abroad Outcomes………………………………………………...19 Cultural Competence …………………………………………………….19 Global Citizenship and Global Health Competencies…………………....21 Personal and Professional Growth……………………………….…….…23 Impact on Practice………………………….……………………….……23 Transformative Learning……………………………….……………..….23 University of Utah College of Nursing Ghana Study Abroad Program………....23 3. THEORETICAL FOUNDATION…………………………………………………….23 4. IMPLEMENTATION and EVALUATION………………………………..…………26 5. RESULTS……………………………………………………………………………..28 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 4 6. RECOMMENDATIONS………………………...……………………………………30 7. DNP ESSENTIALS…………………………………………………………………..30 8. CONCLUSIONS………………………………...……………………………………31 9. REFERENCES……………………………….……………………………………….33 10. PROPOSAL DEFENSE POWERPOINT ……………………………………...App. A 11. IRB EXEMPTION……………………………………………………………...App. B 12. SURVEY ……………………………………………………………………….App. C 13. ASSESSMENT TOOLKIT for a NURSING STUDY ABROAD PROGRAM..App. D 14. POSTER PRESENTATION……………………………………………………App. E 15. ABSTRACT ACCEPTANCE…………………………………………………..App. F AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Acknowledgements To my one and only Mum. Your tenacity in the face of the challenges and adversity of life are my example and inspiration. You are one of a kind and I love you always. I hope this makes you proud. 5 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 6 Problem Statement International experiences have become a popular type of experiential learning for college students. According to the Institute of International Education, for the 2013-2014 academic year 304,467 students from the United States (US) participated in study abroad (SA), a 5.2% increase over the previous year. Total numbers of US students who participate in study SA has tripled in the past 20 years ("Fast Facts, Open Doors," 2015). In 2010-2011, there was a 14% increase in health sciences students participating in SA from the previous academic year ("Fast Facts Open Doors," 2012), and in 2013-2014, 3.4% were from the health sciences. This is an increase of 4.5% from the previous year ("Fast Facts, Open Doors," 2015). While this indicates a consistent upward trend, it continues to constitute only a small percentage of college students as a whole. While the number of nursing students participating in international experiences is not known, it is clearly small. A study that looked specifically at nursing SA found that 0%-5% of baccalaureate nursing students who attended schools offering semester long SA programs actually participated in these programs. This was thought to be a function of constraints unique to nursing education and to the duration of the programs (Read, 2011). There is little data about the numbers of nursing students who participate in programs of shorter duration, although it does indicate that participation in short term programs is approximately double that of participation in semester long programs ("Fast Facts, Open Doors," 2015). An identified problem is that measurement of outcomes, and post trip assessment are lacking in many international experiences, including those in nursing. This limits the evaluation of quality, meaning, and relevance and may impact support, interest and funding of projects. Given that the available evidence strongly supports the benefits of SA and international learning, AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 7 the lack of this component is problematic. In light of the internationalization of the university and higher education, and an emphasis on global health in nursing, as well as nursing's focus on outcomes and best practice at all levels, it is particularly relevant. The University of Utah College of Nursing Ghana study abroad (UUCONGSA) program was a 6-year-old project that evolved over its tenure to follow best practice standards of study abroad, nursing, and global health practice. That said the project lacked a formal post trip assessment (A. Cutting, personal communication, July 1, 2013). Given the emphasis on assessment and outcomes in higher education, there was a need to develop this portion of the SA program. The goal of this project is to develop a toolkit for post trip assessment for the UUCONGSA program. Clinical Significance and Policy Implications Study abroad programs may be utilized to meet educational requirements for expertise in cultural competence and global health, as well as to meet the larger demand on higher education to produce engaged global citizens. Studies indicate that international experiences are far more effective in developing cultural competence than didactic teaching or local cross cultural experiences (Bentley & Ellison, 2007; Caffrey, Neander, Markle, & Stewart, 2005; (Caffrey, Neander, Markle, & Stewart, 2005; Walsh & DeJoseph, 2003). Also, international experiences develop self-confidence, autonomy, and world view in a unique way, and these unique effects of SA may impact nursing practice (Smith-Miller, Leak, Harlan, Dieckmann, & Sherwood, 2010). Increased use of nursing SA may impact nursing practice as a whole. However, there is a need for more data about outcomes in study abroad. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 8 Administrative support is needed for faculty to develop and maintain SA programs (Bentley & Ellison, 2007). Development and maintenance of SA programs is time consuming work, and it often depends on the interest of a single faculty member. The departure of that faculty member may lead to the demise of the program (Palmer, Wing, Miles, Heaston, & De la Cruz, 2011; Lipson & Desantis, 2012). Documentation that SA is a means of meeting desired outcomes is necessary for commitment of resources. In spite of the charge for universities to internationalize, faculty and administration have been reluctant to value SA as a legitimate form of education (Lewin, 2009). The de-valuing of SA can only be changed by documentation of value. Otherwise "it leaves policy makers ineffective to change its role" (Lewin, 2009, p. 299). At the inception of this project, the University of Utah College of Nursing (UUCON) supported one international global health and study abroad program. There were no plans to increase programs in the immediate future (A. Doig, personal communication June 30, 2013). Ultimately, the program went on hiatus in 2015 due to lack of administrative support. But, interest in Global Health in the College of Nursing (CON) does appear to be increasing (L. Edelman, personal communication, February 8, 2016). Recently, faculty including the former UUCONGSA program director, were awarded a Global Learning Across Disciplines grant from the University of Utah Office of Global Engagement for their project, "Bridging Gaps in Global Learning and Leadership" (A. Cutting, personal communication, March 21,2017). When documentation of the value of SA is lacking, it is hard to argue with the allotment of resources elsewhere. The development of a toolkit to evaluate outcomes of the UUCONGSA program may have given appropriate value to the program, allowed for directed change and quality improvement, and may perhaps have helped to affect policy and commitment of the UUCON to study abroad. While the toolkit was initially intended for the Ghana program, it may AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 9 be utilized by any nursing study abroad program. Assessment of outcomes as a result of utilization of the toolkit will add to the body of knowledge of study abroad and nursing as a whole. Project Objectives 1. Describe the purpose, objectives, outcomes, and outcomes assessment of study abroad programs in nursing. 2. Identify assessment tools for identified nursing study abroad outcomes. Formulate a toolkit for outcomes assessment for the UUCONGSA program that consists of tools that meet inclusion criteria: a. Measures identified outcomes b. Student population c. Validated tool, psychometric data available d. Available for review at no cost 3. Disseminate the toolkit to study abroad directors within the college of nursing, and to a larger audience through poster or podium presentation. Search Terms and Strategy CINAHL and PubMed databases were searched. Given the evolution of terminology, search terms were broad. Terms used: study abroad, international service learning, community engaged learning, international training experiences, international education, immersion experience. These were cross referenced with nursing, nursing student, outcomes, assessment, evaluation, impact, global health, program framework, best practice, transformative learning, practice impact, personal and professional growth, and competencies. Limits: English language; United States and Canada; publication date 20 years; case reports, comparative study, evaluation AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 10 study, guideline, historical article, journal article, and review. From articles obtained, classic articles were identified and located regardless of publication date. Relevant articles found in the reference list of journal articles were evaluated as were journal articles identified as relevant in "Related Searches" and "Titles with Your Search Terms" during literature search. Web sites such as Institute of International Education, and sites identified in journal articles were searched, as was the Chronicle of Higher Education. The Forum on Study Abroad, also identified in journal articles, was located but was member only access. Nursing sites such as American Nurses Association, International Council on Nursing, American Academy of Colleges of Nursing and the Institute of Medicine were utilized. Relevant books were identified from the literature, Google search and Amazon search. And lastly, Amy Cutting APRN, FNP, the director of the Ghana study abroad program, was consulted for information regarding the program. Literature Review A review of the nursing literature reveals over 40 years of interest and focus on study abroad. This section will first cover a general overview of SA; the challenge of terminology, the relevance of SA to higher education and to nursing, as well as the status of outcomes assessment in SA as a whole. Then, issues specific to nursing SA will be addressed, first through a summary of the literature reviews that were found, and then through a review of the outcomes that were found to be specific to nursing study abroad. What Is Study Abroad? The terminology of international experiences is broad. Study abroad and international service learning are commonly used, but there are many others. The components of these programs may differ, or they may overlap. Vagueness of terminology makes comparison and AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 11 evaluation difficult. However, while programs may differ widely, similarities result from adherence to best practice standards. Currently, according to The Forum on Study Abroad, the terminology that describes SA programs is "sort of a rough classification system" (VandeBerg, n.d., para. 2) with overlapping, redundant and unclear terminology that "borders on anarchy" ("Education Abroad Glossary," 2011, para. 2). While efforts are being made to standardize terminology, the literature still reflects variability. It is beyond the capability of this project to evaluate each article for standardization, therefore the term "study abroad" will be utilized regardless of the program structure. Paralleling the variability of titles is the variability of available experiences. The common feature is the international nature of the experiences. Programs are available in multiple disciplines, including the health sciences, and participants may be multi-disciplinary or from one area of study. Program duration varies from 1 week to many months. Programs may involve hands on care or service, may be purely observational or educational, or may involve community participatory learning or research. Again, terminology does not always reflect program structure. While nursing participation has most often been at the undergraduate level, associate degree students and graduate students also participate in international learning experiences (Smith & Curry, 2011; Foronda & Belknap, 2012). The available literature on assessment and outcomes generally does not discriminate between levels of training; therefore, it is not feasible to make such discrimination for this project. Study Abroad in Higher Education While higher education is considered to be "the ivory tower", it is still driven by cultural forces and societal needs. As the world globalizes, all portions of society are impacted. This AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 12 includes business and finance, national and international security, as well as healthcare, education and others. This has led to an academic focus on internationalization. In 2004, The National Association of State Universities and Land Grant Colleges (NASULGC) published "A Call to Leadership: The Presidential Role in Internationalizing the University" followed in 2007 by "A National Action Agenda for Internationalizing Higher Education" (The NASULGC Commission on International Programs [NASULGC], 2007). This document delineates desired student outcomes including "comprehension of the international dimensions of the major field of study"; "understand the importance of and exhibit sensitivity and adaptability in cross-cultural communications" (NASULGC, 2007, p. 2); as well as "experiences outside the U.S." (NASULGC, 2007, p. 3). Study abroad is identified as a means of achieving these goals (Wanner, 2009). On a broad scale, the development of global citizens, the encouragement of global competence, and the development of cross cultural understanding is a part of the responsibility of higher education and within the purview of study abroad. The University of Utah supports internationalization. This is reflected in the mission statement as "We advance rigorous interdisciplinary inquiry, international involvement and social responsibility" ("Mission Statement," n.d., para. 1), as well as the development of the Office of Global Engagement (OGE) and the appointment of a Chief Global Officer. Core values of OGE specify that "…global learning is at the heart of the U's future as a research and teaching university" and "requires a unique contribution from every discipline on campus" ("Office of Global Engagement, Mission and Vision Statement," n.d.). Concurrently, the global focus of the CON is reflected in the vision statement," To develop leaders in nursing and health care whose actions, discoveries and voices strengthen and advance the health of individuals and communities worldwide" ("Vision CON," n.d.). AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 13 Study Abroad in Nursing National initiatives and professional organizations have driven an increased interest in global experiences for nursing students. Often, the development of cultural competence, a core nursing value, is at the center of these initiatives. This obligates nursing educators to meet these requirements. The American Nurses Association (ANA) states that "Knowledge about cultures and their interaction with health care is essential for nurses, whether they are practicing in a clinical setting, education, research or administration" (Cultural Diversity in Practice, 1991, para.2). The ANA Code for Nurses mandates that the ethical code for nurses is non-negotiable and describes a code that includes elements of social responsibility (ANA Code of Ethics, 1994). In 2001, the U.S. Department of Health and Human Services Office on Minority Health delineated 14 National Standards on Culturally and Linguistically Appropriate Services (CLAS) for health care organizations and providers (National Standards, 2001). In 2003, the Agency for Healthcare Research and Quality (AHRQ) emphasized the requirement for "organizations and personnel to acquire and institutionalize cultural knowledge" (Linguistic Competence? 2003). Again, cultural knowledge and inter-cultural skill are seen as critical to health care. The American Academy of Colleges of Nursing (AACN) has established cultural competencies for undergraduate and graduate students. AACN specifies that the nursing role, which it identifies as being impacted by globalization, encompasses social justice, personal transformation, and an understanding of the social and cultural dimensions of health (Cultural Competency Baccalaureate, 2008; Culturally Competent Master's, 2009). Additionally, in 2002 the Institute of Medicine report "Unequal Treatment: Confronting Racial/Ethnic Disparities in AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 14 Health Care identified persistent racial and ethnic disparities in health care, and recommended cross-cultural education for all health care professionals (Smedley, Stith, & Nelson, 2003). Incorporating international experiences and study abroad into nursing curriculum creates an opportunity for meeting these requirements. SA programs aim to develop global citizenship, leadership and civic engagement (Brown, 2011; Lewin, 2009). Crigger et.al. stress that this is an important component of nursing education that may be facilitated through exposure to other cultures and that should include a commitment to social justice (Crigger, Brannigan, & Baird, 2006). Exposure to the reality of life in low and middle income countries demonstrates, in a profound way, how social determinants impact health. Global health competencies may be built into objectives and goals, and may be achieved in a way that is direct, real and non-theoretical. Programs may focus on the health of communities and indeed, in the world's mostly collective societies, public and community health is most relevant. Ideas about social justice have been identified as an outcome of SA (Evanson & Zust, 2006). And while the literature strongly supports the development of cultural competence from SA participation, social justice has been described as the appropriate framework for culturally competent care (Clingerman, 2011). Outcomes Assessment in Study Abroad Outcomes assessment has become an integral part of higher education. Relative to this trend, outcomes assessment has been lacking in study abroad (Bolen, 2007). As the focus and purpose of SA has evolved and broadened, the outcomes desired have changed. The original emphasis was on language development and cultural appreciation. This has evolved into a focus on the development of skills and knowledge that are needed to address changes and needs that come with globalization. With increased program complexity, comes a need for increased faculty AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 15 and resources. To justify funding in an era of decreased budgets, the benefit to the student, to the mission and to society must be made clear. Measurement of learning outcomes is standard in education. Goal attainment must be measured to allow for program adjustment and for credibility. It is crucial to measure outcomes to establish validity of SA as a means of rigorous education. It is hypothesized that the low numbers of US students participating in SA are related to a lack of faculty and department support. It is difficult to counteract the impression that SA is simply an exotic vacation without data to prove otherwise. "Assessment in international education, particularly within education abroad, is a recent phenomenon, driven largely by the trends in greater assessment and accountability within higher education in the United States" (Deardorff, 2009, p. 346). However, in 2012 the American Council on Education's Mapping Internationalization on U.S. Campus project found that despite a perceived increase in campus internationalization at a majority of responding institutions, only 37% had formally assessed progress, and only 55% had developed international or global learning outcomes (American Council on Education [ACE], 2012). Given the cost of study abroad to participants and university departments, accountability of educational disciplines to accrediting organizations, and the necessity of meeting target learning outcomes, it is crucial to evaluate outcomes of SA programs. "With globalization driving the demand for global-ready graduates, it becomes crucial for administrators to assess the outcomes of education abroad experiences to determine exactly what our students are learning while abroad and how effective our programs are in achieving stated learning outcomes" (Deardorff, 2009, p. 346). While the outcomes of service learning may be subtle and measurement of outcomes could be construed as a superficial treatment of the inherent AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 16 possibilities, they are necessary for the establishment of the legitimacy of the practice (Butin, 2010). Yet, the nursing literature indicates that there is a need for increased assessment of SA outcomes (Allen et al., 2013; Button et al., 2005; Callister & Cox, 2006; Edmonds, 2012; Harrowing et al., 2012; Larson et al., 2010; Lipson & Desantis, 2012; Walsh & DeJoseph, 2003). Summary of Literature Reviews about Nursing Study Abroad Study abroad in nursing has evolved substantially over 20 years. In 2005 Button et al (Button, Green, Tengnah, Johansson, & Baker, 2005) looked at studies from 1980-2003 from the U.S, U.K. and Europe. The purpose of the literature review was to look at program differences, at methodological approaches, and to look critically at opinions regarding the personal and professional impacts of study abroad. In general, they found a lack of quantity or quality of sources that were relevant to their purpose. They retrieved 54 papers, and found that 11 of these either indicated no methodology, or reflected a program that was not educational. The rest used qualitative, quantitative or mixed methodology. They state that little impact on lives was found. Three areas were, however, found to impact lives and practice. The first area was the benefits of study abroad: learning cultural differences, gaining the ability to compare health systems and nursing practice, and personal development. Second, they found that differences in program length, and in the level of development of the country involved, impacted student experience. However, few studies compared programs by either parameter. The argument about the impact of longer vs. shorter programs is unsettled, although they did state that longer programs may have more impact. Last, they found that culture shock is significant during transcultural adaptation. This requires student preparation and support to minimize the potential AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 17 negative impact on the experience. Most significantly, they found a need for more research and evaluation on the effects of the SA experience. Edmonds (Edmonds, 2012), looked at studies from the 1990's onward. The purpose was to visualize the historical development of SA in nursing over time. This review summarized anecdotal and research based literature to delineate gaps and inconsistencies. Anecdotal reports looked at programs from 1-6 weeks in duration. These programs went to developed and nondeveloped countries and generally described benefits, such as personal and professional growth, enlightenment, increased global sensitivity and global awareness. Research reports looked at programs ranging from 2-16 weeks. These programs went to either developed or non-developed countries and some went to both. Participants were found to be homogenous; typically, single white women in the last year of nursing school. The literature rarely noted the presence or absence of diversity. Benefits were personal development, and cultural self-efficacy or competence, and were generally short-lived. The review found no comparisons between programs in developed and non-developed countries. Although the review found studies recommending longer programs, no studies that compared the two types were discussed. Differences in measures of cultural self-efficacy were found between students who had participated in SA and those who had not. Research methods were found to be mostly qualitative for studies conducted outside the U.S. Although the author stated that quantitative studies were much more common than qualitative studies in the U.S., this was not supported in the studies that were reviewed. Although the author found themes that indicated benefits from SA, she stated that there is not enough evidence to support it "as a means of bridging the theory practice divide" (p. 33) and she recommended more evidence based research. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 18 In 2012 Kulbok et al (Kulbok, Mitchell, Glick, & Greiner, 2012) evaluated international empirical studies from 2003-2010 specific to global health experiences in nursing education. The purpose was to critically review the research literature about international education and student experiences, and also to examine educational issues of transcultural sensitivity and globalization. They evaluated previous literature reviews, and utilized resulting concerns. This process drove an inquiry into the specifics of international partnership issues such as authorship and immersion. These were evaluated in the context of a true exchange. They identified 100 articles, 40 of which were research based. Of those 23 were empirical and fit their criteria. The vast majority of studies were qualitative, followed by exploratory or descriptive studies, and 2 were program evaluations. Studies had between 5-38 participants, most of whom were undergraduate nursing students. Graduate students, doctoral students and faculty were also represented in some studies. Most were written by faculty from one country, and most students were from the US or Europe. Only one study had a co-author from a low-moderate income country (LMIC). Only 3 studies utilized a theoretical framework to evaluate and analyze student experiences. However, in an apparent attempt to address a critique of a previous study, most studies did at least evaluate student experiences. Most typically this occurred from analysis of student feedback; only 2 were from a "more formal programmatic perspective" (p. 17). The predominant concept was culture: cultural competence and differences, motivation and effort to develop cultural competence. Personal growth was another concept found. While learning about other countries and learning about global health were frequently stated goals, few authors actually focused on this or evaluated it in any way. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 19 Conclusions were that 2 way exchanges are necessary for true collaboration to occur, and that the needs of partners must be considered in educational exchanges. The authors also cautioned faculty who are often involved with non-university organizations to be aware of ethical practice when the guidelines of study abroad are not in place. Nursing Study Abroad Outcomes Cultural Competence When cultural competence outcomes were measured, or described, no consistent definition of cultural competence was used. Often, there was no definition or framework at all. Cultural competence definitions that were utilized included Campinha-Bacote (Bentley & Ellison, 2007; Larson, Ott, & Miles, 2010; Smith-Miller, Leak, Harlan, Dieckmann, & Sherwood, 2010; Allen, Smart, Odom-Maryam, & Swain, 2013), Leininger (Foronda & Belknap, 2012), Purnell (Allen, Smart, Odom-Maryam, & Swain, 2013) and Wells (Caffrey, Neander, Markle, & Stewart, 2005). Campinha-Bacote defines cultural competence as "the ongoing process in which the healthcare professional strives to achieve the ability and availability to work effectively within the cultural context of the patient (individual, family, community)", (Campinha-Bacote, 2003, p. 15) thus it is evolving and progressive in nature. Similarly, Purnell describes a culturally competent provider as one who is self-aware without allowing their awareness to impact care, and describes culturally competent care as adaptive to the client, consistent with their culture, and a conscious and non-linear process (Purnell, 2002). Leininger however, describes culturally congruent (nursing) care as care that utilizes cognitively known values, beliefs and expressions. Also, assistive, supportive, or enabling professional actions and decisions are used to help individuals within a particular culture maintain health or face illness (Leininger, 1988). Wells AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 20 describes a biphasic process incorporating cognitive and affective changes. Acquisition of knowledge moves the learner along a continuum from cultural incompetence to cultural awareness. At the same time, impacted by cognitive development, changes in attitude and behavior occur. This phase also moves along a continuum to the development of cultural proficiency and requires experience working with culturally diverse groups (Wells, 2000). It is also then, an ongoing process. Evaluation of cultural competence outcomes was done in either qualitative or quantitative studies. Analytic induction (Walsh & DeJoseph, 2003), phenomenological inquiry (Callister & Cox, 2006), qualitative descriptive methodology (Larson et al., 2010, Smith-Miller et al., 2010) and narrative methods (Foronda & Belknap, 2012) were utilized. Formats were student journals (Walsh & DeJoseph, 2003; Larson et al., 2010), focus groups and interviews (Walsh & DeJoseph, 2003; (Callister & Cox, 2006; Larson et al., 2010; Foronda & Belknap, 2012), and reflection papers (Smith & Curry, 2011). Quantitative studies all utilized pre- and post- test rating scales (Caffrey et al., 2005; Bentley & Ellison, 2007; Allen et al., 2013). All qualitative studies elucidated themes through analysis of content. Thematic categories can be broadly conceptualized as personal or professional. Themes were seen as the representation of experiences that may contribute to the development of cultural competence (Walsh & DeJoseph, 2003). Common personal themes were a change or broadening of worldview, increased appreciation of other cultures, and the sense that leaving the familiar made other cultures become real. This lead to a changed perception of reality, a development of empathy for those from other cultures through the difficulty of navigating everyday life, and a personal sense of cultural competence. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 21 Common professional themes were discovered. These included a sense of an increased ability to function in different or challenging situations, and an increase in autonomy and confidence. Understanding of the desires of patients increased as well as an understanding of the need to work with open-mindedness and without stereotyping. There was an increase in the ability to learn from and negotiate with patients, and a deeper understanding of the determinants of health. Quantitative studies indicated some similar outcomes. A changed attitude resulting from new understanding of hardships, increased comfort with those from other cultures, increased awareness of the need to assess culture, and an increase in perceived ability to provide culturally competent care was found. Additionally, a new willingness to consult about culture, an awareness of limitations, an increase in the affective components of cultural competence, and an increase in self- perception of knowledge, self- awareness and skill comfort were found and were conceptualized as markers for development of cultural competence (Caffrey et al., 2005). Outcomes in cultural competence were not all positive. Foronda (2012) showed little development of empathy or change in perspective, although some more superficial changes in learning and change in meaning scheme did occur. Harrowing (2012) found that in spite of cultural immersion, understanding of culture remained largely superficial and essentialist, overly broad, and did not "immediately transform participants into cultural experts" (Harrowing, Gregory, O'Sullivan, Lee, & Doolittle, 2012, p. 494). While effects from study abroad were not denied, they were described as more modest than had been implied in the literature. Global Citizenship and Global Health Competencies The lack of a consistent and agreed upon definition of global health has made the development of competencies challenging. However, Currier et al (2009) describe outcomes in AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 22 global competency found in a literature review of study abroad. Many outcomes overlap with cultural competencies, but they also describe specific characteristics related to global health. These were an increased understanding of determinants of global health, increased understanding of global complexity, expansion of values and social justice beliefs to include global concepts, and growth as global citizens (Currier, Lucas, & Saint Arnault, 2009). In keeping with the purpose of higher education, outcomes of study abroad may include not only increasing global health competencies but also development of global awareness and a broadened perspective. Study abroad has been shown to increase the development of global citizenship, as well as increased ability to function across cultures (Herrin, 2004). The Institute of International Education reported that 50% of study abroad alumni used the cultural expertise acquired abroad in their jobs (Grothus, 2009). Global competency conceptualized holistically may be seen to be as crucial as specific global health competencies. In 2011 Calhoun and Spencer discovered that of all health science competency models published in the literature, only one from the CDC was a global health model (Calhoun & Spencer, 2011). In 2011, the Association of Schools of Public Health (ASPH) published their Global Health Competency Model. Competency based curricula in global health have been developed for pediatric and medical residents. A recent survey of international nursing educators led to the development of delineated global health competencies for nursing (Wilson et al., 2012). Assessment tools however, have been lacking. This has created a gap in the literature regarding outcomes of global health education, including outcomes from study abroad. However, in 2013 Versa et al. published the results of a study that assessed a global health competency evaluation tool; the tool was found to have good internal consistency and validity (Veras et al., AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 23 2013). This creates an opportunity for evaluation of outcomes of global health education and of study abroad outcomes in global health. Personal and Professional Growth A commonly cited outcome for students in study abroad is personal growth (Bosworth et al., 2006; Button et al., 2005; Ryan, Twibell, Brigham, & Bennett, 2000; Ryan & Twibell, 2002). While factors in personal growth are clearly individual, self-confidence, autonomy, selfknowledge, commitment, passion, independence, and altruism have been noted (Bosworth et al., 2006). Additionally, change in personal values (Ryan et al., 2000; Ryan & Twibell, 2002), and re-thinking of personal biases has been noted (Ryan & Twibell, 2002). Long term impacts have been found in development of social consciousness and awareness of injustice (Levine, 2009), but this finding was inconsistent (Kirkham, Hofwegen, & Pankratz, 2009). Impact on Practice Impact on professional practice has been mostly observed in the short term. Bosworth et al (2006) noted the use of self in healing, increased advocacy, and an increase in ability to evaluate health care systems from a broad perspective. Changed communication with patients (Ryan et al., 2000; Ryan & Twibell, 2002), redefinition of nursing roles with an increase in sensitivity and respect in clinical practice (Adamshick & August-Brady, 2012), more sympathy to patients with language barrier, reinvestment into nursing as a career, and increased awareness of their responsibility and influence (Carpenter & Garcia, 2012) have been found. Transformative Learning Transformative learning was defined by Mezirow as a step-wise process leading to change in perspective in the domains of self- concept, self-knowledge, and belief systems. Ultimately transformative learning leads to a change in the way one lives life. Critical reflection AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 24 of disruptive or dissonant experiences, especially when expertly guided, creates a rich environment for profound change in perspective. However, the occurrence is rare (Mezirow, 2000). The experience of study abroad, particularly in a situation of great cultural dissonance, may be seen as an opportunity for transformative learning. In fact, the literature does describe transformative change even when the study abroad participants were experienced nurses (Adamshick & August-Brady, 2012; Levine, 2009). However, the need for intentional guidance and for ongoing engagement in experiences that foster the changed perspective are crucial for sustaining change (Kirkham et al., 2009). Foronda and Belknap (2012) describe a case in which transformative change did not occur despite a disruptive study abroad experience. University of Utah Ghana College of Nursing Study Abroad Program The University of Utah College of Nursing Ghana study abroad program evolved from a foundation in traditional international service learning to one based on concepts of community engaged learning and current ethical standards as described by Strait & Lima (2009) and Crump & Sugarman (2008). The program required a pre-trip class in global health for participants. Trip duration was four weeks, spent in observation of the Ghanaian health care system, as well as interaction with allopathic and traditional health care providers and nurses. Participants collaborated with and assisted community nurses in a community health project. Informal discussion of participant experiences, thoughts, and feelings occurred during and after the program. Theoretical Foundation Lewin's theory of change delineates the steps by which individuals make personal or professional changes in thoughts or actions. It is a stepwise process that involves the AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 25 incorporation of new ideas or information into existing ways of thinking or being. Those new ideas or experiences may occur in a formal or informal setting, and may evolve from an educational process or from a personal experience that may be transformative. The theory describes driving forces that push toward change and that are opposed by restraining forces. If these forces are in equilibrium, no change occurs. If driving forces are stronger, change will occur; if restraining forces are stronger no change will occur. Even when change has occurred, over time the presence of restraining forces may push back toward the previous status quo (Current nursing [Current], 2011). The 3 stages of Lewin's change theory are unfreezing, change or moving, and refreezing. Unfreezing involves an examination of how things are or have always been. During this stage driving forces must be in place for the next stage to occur. Change, or moving, is the stage when the driving forces overwhelm restraining forces and there is movement in the direction of change or outcomes. Refreezing is the stage when the change hopefully becomes permanent. This is the stage where desired outcomes may become long-term. However, it is also a stage when restraining forces may push back and desired outcomes may regress (Mitchell, 2013). Lewin's theory of change is applicable to this project in a number of ways. First, the assessment of change implies that change is not only possible, but is probable under the right conditions. Without the possibility of change, there is no point to outcomes assessment. Given Lewin's theory, study abroad has the potential to create change. Desired changes are reflected in the goals of the program, there may be changes desired by individual participants, or there may be potential changes that have not been articulated or considered. The purpose of this project is to make the assessment of change a part of the study abroad program. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 26 Additionally, although program evaluation is not a part of this project, the assessment of outcomes, by nature, implies that the feedback will be utilized to evaluate program components. Program theory and logic models, currently utilized in program evaluation, are often used in conjunction with an underlying change theory. While that theory is not of necessity a formal or research based theory (Funnell & Rogers, 2011), the use of Lewin's theory of change as the theoretical basis for this project, provides a foundational theory for the logical and iterative process of program evaluation and outcomes assessment. Implementation and Evaluation Project proposal was defended and passed (Appendix A). Objective 1. Describe the purpose, objectives, outcomes and assessment of study abroad experiences in nursing. This was accomplished through literature review using relevant key words. Project chair and content experts were provided with the literature review for review and acceptance. Objective 2. Identification of assessment tools for identified study abroad outcomes. An application was sent to the University of Utah Institutional Review Board and the project received exempt status (Appendix B). Literature review, and web search were completed and tools that assess outcomes in nursing study abroad were identified. Then, a survey was developed on Survey Monkey (Appendix C) to query a limited sample of directors of nursing SA programs regarding outcomes assessment in their programs. Identification of SA program directors was as follows. 1. Slips of paper with the names all US institutions of higher education who are members of CUGH were placed in a bag. 2. Names were drawn at random from 85 slips. 3. A web search of each institution drawn was completed to determine if the school had a college of nursing (CON) and if that college had a AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 27 study abroad program that was based within the college. 4. Study abroad directors and their emails were identified. This continued until 10 CON were identified. Brigham Young University and Westminster College were included for local benchmarking. While the original plan was to survey 12 schools, it was evident that many CON had more than one SA program. In those cases, each director was sent a survey. Surveys went to 12 CON, and 30 individuals. Six responses from 6 different institutions were received. This was a 20% overall response rate, and a 50 % institutional response rate. Response rate for the survey was computed as described above and compared to the goal of 60%. The tools identified were matched with outcome areas to see if all outcome areas had associated tools Objective 3. A Toolkit will be formulated for the University of Utah College of Nursing Ghana study abroad program. Tools that were identified from the literature search and survey were evaluated according to the inclusion criteria set for the project. Tools that met the criteria were described and the description was included in the toolkit document and appended to the final project (Appendix D). Identified outcomes were linked with a nursing theory. This was included in the toolkit to provide a theoretical foundation for the toolkit. The toolkit was provided to the Ghana study abroad director and content experts to identify barriers to use. The tool was provided to the study abroad director to identify preferences in formatting. The summary document was provided to the project chair and content experts for approval. Content experts approved the content. A poster presentation was presented as the final project defense (Appendix E). AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 28 Objective 4. Dissemination. The toolkit was provided to the director of Ghana study abroad program. The poster was presented at the April 2017 CUGH conference in Washington D.C. (Appendix E and F). Results A thorough literature review enabled description of study abroad programs. The vague definition of study abroad and the international nature of the studies were barriers. This made narrowing of the literature review necessary and may have led to missing some pertinent information. Identification of assessment tools was cumbersome. Literature search identified tools but typically did not supply them or did not identify how to access them. This made finding the tools a difficult task. The sheer number of tools identified in cultural competency alone was enormous and evaluation over 30 tools took a tremendous amount of time for little return. While many tools were successfully identified, few met the inclusion criteria. So, it was not possible to match all areas of outcomes with an appropriate tool. Survey responses were lower than desired, although the target was not realistic given the typical response rate of surveys. This may have been a result of the inexperience of the surveyor as well as the availability of faculty involved in SA. At times, it was difficult to find contact information for tool developers. This made obtaining tools and permission for use problematic. Individual tool developers were generally generous with their tools while corporations were not. Many tools that were identified had not undergone any psychometric testing. This limited the number of tools that fit criteria for inclusion in the toolkit. Linking each tool with a nursing theory was redundant and unnecessary. The theoretical foundation of the project was sufficiently AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 29 broad to provide a theoretical foundation for each area, particularly in light of the paucity of appropriate tools. The goals of dissemination have been impacted by changes that have occurred over the duration of the project and this was an unintended consequence. The Ghana study abroad project has been placed on indefinite hold. The time frame for completion of the project and application for poster presentation at the conference of the Transcultural Nursing Society did not align. However, an abstract was accepted by CUGH for poster presentation at their annual conference in Washington, D.C. in April 2017. In addition, four of the study abroad directors who were surveyed requested a copy of the completed toolkit to use in their programs. The toolkit will also be provided to UUCON faculty who were awarded a Global Learning Across Disciplines grant from the University of Utah Office of Global Engagement for their project, "Bridging Gaps in Global Learning and Leadership' to assist in strategizing a potential increase in global health nursing education in the CON. The primary limitation on the project was time. The sheer amount of information was unexpected and the time that it took to evaluate literature and tools in all areas of outcomes was onerous. The inclusion criteria, while necessary given the volume and variety of tools available, did exclude some tools that might be valuable to small SA programs. The final product or toolkit consists of three parts. The first section is the literature review pertinent to each tool. The second section is an easily used chart that includes description, psychometric data and contact information for access to each tool. The third section is a bibliography that includes articles and resources curated by the author that may be valuable for SA directors utilizing the toolkit. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 30 Recommendations The toolkit should be a resource that is available to future study abroad programs at any CON, particularly for new directors. The toolkit might be expanded to include commonly used tools, and tools that have reliability but not validity data. More of these tools are available at no cost, many of them have been widely used, and they may more practical for use by programs with financial limitations. In addition, further description of qualitative measures would be valuable. Study abroad as a concept is changing rapidly. Terminology, program structure, and best practices are rapidly evolving. Tools are being developed that reflect these changes and a wider range of outcomes are being evaluated. The toolkit will soon need to be updated for timeliness and accuracy and should include new and innovative assessment measures. These initiatives will require individuals and faculty with commitment to study abroad and to global health education. Given demands on faculty and clinician time this may be problematic. Support from administration at all levels will be needed so that development, implementation and evaluation of quality programs may continue and so that evidence based assessment may increase. DNP Essentials DNP Essential I. This project supported DNP Essential I: Scientific Underpinnings for Practice. Extensive inquiry uncovered advances in nursing science that were pertinent and foundational to the project and that will lead to new practice approaches and better patient outcomes. Outcomes evaluation is an expected skill of the DNP graduate and is the overarching purpose of the project. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 31 DNP Essential II. DNP Essential II: Organizational and Systems Leadership for Quality Improvement and Systems Thinking is supported by this project. The toolkit will guide the practice of nursing in the utilization of international experiences to encourage sensitivity to diverse populations. DNP Essential III. DNP Essential III: Clinical Scholarship and Analytical Methods for Evidence Based Practice is supported by this project. Clinical scholarship, analytical methods and information technology were utilized to discover the body of knowledge in nursing study abroad. Additionally, appraisal of existing literature identified gaps in knowledge and evidence. Analytic methods and information technology were utilized to develop the survey and to evaluate, assess and integrate results. The results of this inquiry justified the development of the toolkit. Substantial investigation and analysis was utilized in discovery and evaluation of tools. Inclusion criteria specified the incorporation of valid tools, thus supporting scholarship, evidence-based practice, and translation of research into practice. Conclusion International experiences and study abroad have been shown to impact nursing practice. Outcomes have been reflected and described in the literature for many years. Yet, evidencebased evaluation of these programs and of their outcomes has been under used. Often, outcomes have been measured by collection of anecdote, or by utilization of tools that have been developed by program faculty or that are program specific. This has limited the evidence base that should support the utilization of study abroad programs. Given that these programs impact nursing practice, this gap is significant and may decrease utilization of a unique tool. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 32 Evidence is the foundation of nursing practice. It supports practice and it should be a product of practice. Validated tools are necessary in this iterative process and should be accessible to practitioners. The purpose of this project was to discover, evaluate, and synthesize evidence and to discover validated tools and make them accessible. Outcomes may then be more easily measured and an evidence-based foundation will be developed for further growth and study. Hopefully this resource will impact the utilization of study abroad in nursing and impact nursing practice as a whole. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 33 References Adamshick, P., & August-Brady, M. (2012, May-June). Reclaiming the essence of nursing: The meaning of an immersion experience in Honduras for RN to Bachelor of Science students. Journal of Professional Nursing, 28, 190-198. http://dx.doi.org/10.1016/j.profnurs.2011.11.011 Allen, C. B., Smart, D. A., Odom-Maryam, T., & Swain, D. (2013). The value of communityfocused interprofessional care in Peru for developing cultural competency in health profession students. International Journal of Nursing Education, 10(1), 1-10. http://dx.doi.org/10.1515/ijnes-2012-0014 American Council on Education. (2012). Mapping internationalization on U.S. campuses:2012 edition. Retrieved from http://www.acenet.edu/newsroom/Documents/MappingInternationalizationonUSCampuses2012-full.pdf Bentley, R., & Ellison, K. J. (2007, September/October). Increasing cultural competence in nursing through international service learning experiences. Nurse Educator, 32, 207-211. Bolen, M. C. (2007). Introduction. In M. C. Bolen (Ed.), A guide to outcomes assessment in education abroad (pp. 1-6). Carlisle, PA: The Forum on Education Abroad. Bosworth, T. L., Haloburdo, E. P., Hetrick, C., Patchett, K., Thompson, M. A., & Welch, M. (2006, January/February). International partnerships to promote quality care: Faculty groundwork, student projects, and outcomes. The Journal of Continuing Education In Nursing, 37, 32-38. Brown, N. C. (2011). A 360- degree view of international service learning. In R. G. Bringle, J. A. Hatcher, & S. G. Jones (Eds.), International Service Learning; Conceptual Frameworks and Research, pp. 57-68). Sterling, Va.: Stylus Publishing. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 34 Butin, D. W. (2010). Service learning in theory and practice (1st ed.). New York, N.Y.: Palgrave Macmillan. Button, L., Green, B., Tengnah, C., Johansson, I., & Baker, C. (2005). The impact of international placement on nurse's personal and professional lives: literature review. Journal of Advanced Nursing, 50, 315-324. Caffrey, R. A., Neander, W., Markle, D., & Stewart, B. (2005, May). Improving the cultural competence of nursing students: Results of integrating cultural content in the curriculum and an international immersion experience. Journal of Nursing Education, 44, 234-239. Calhoun, J. G., & Spencer, H. C. (2011). Competencies for global health graduate education. Infectious Disease Clinics of North America, 25, 575-592. http://dx.doi.org/10.1016/j.idc.2011.02.015 Callister, L. C., & Cox, A. H. (2006). Opening our hearts and minds: The meaning of international clinical nursing electives in the personal and professional lives of nurses. Nursing and Health Sciences, 8, 95-102. http://dx.doi.org/10.1111/j.14422018.2006.00259.x Campinha-Bacote, J. (2003). The process of cultural competence in the delivery of healthcare services. In The process of cultural competence in the delivery of healthcare services: A culturally competent model of care (4th ed. (pp. 15-20). Cincinnati, OH: Transcultural C.A.R.E. Associates. Carpenter, L. J., & Garcia, A. A. (2012, March/April). Assessing outcomes of a study abroad course for nursing students. Nursing Education Perspectives, 33, 85-89. Clingerman, E. (2011). Social justice: A framework for culturally competent care. Journal of Transcultural Nursing, 22(4), 334-341. http://dx.doi.org/10.1177/1043659611414185 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 35 College of Nursing vision and mission statement. (n.d.). Retrieved from http://nursing.utah.edu/about/index.php Crigger, N. J., Brannigan, M., & Baird, M. (2006, January 28). Compassionate nursing professionals as good citizens of the world. Advances in Nursing Science, 29(1), 15-26. Crump, J. A., & Sugarman, J. (2008, September 24). Ethical considerations for short-term experiences by trainees in global health. Journal of the American Medical Association, 300, 1456-1458. http://dx.doi.org/1o.1001/jama.300.12.1456 Cultural competency in baccalaureate nursing education. (2008). Retrieved from http://www.aacn.nche.edu/leading-initiatives/education-resources/competency.pdf Cultural diversity in nursing practice [Position statement]. (1991). Retrieved from http://nursingworld.org/ Current nursing. (2011). Change theory: Kurt Lewin. Retrieved from http://currentnursing.com_theory/change_theory.html Currier, C., Lucas, J., & Saint Arnault, D. (2009). Study abroad and nursing. From cultural to global competence. In R. Lewin (Ed.), The handbook of practice and research in study abroad (pp. 133-150). New York, N.Y.: Routledge. Deardorff, D. K. (2009). Understanding the challenges of assessing global citizenship. In R. Lewin (Ed.), The handbook of practice and research in study abroad (pp. 346-364). New York, N.Y.: Routledge. DeDee, L. S., & Stewart, S. (2003, July/August). The effect of student participation in international study. Journal of Professional Nursing, 19, 237-242. http://dx.doi.org/10.1016/S8755-7223(03)00086-3 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 36 Edmonds, M. (2012, January/February). An integrative literature review of study abroad programs for nursing students. Nursing Education Perspectives, 33, 30-34. Education abroad glossary. (2011). Retrieved from http://forumea.org/EducationAbroadGlossary2ndEdition2011.cfm Establishing a culturally competent master's and doctorally prepared nursing workforce. (2009). Retrieved from http://www.aacn.nche.edu/education-resources/CulturalComp.pdf Evanson, T. A., & Zust, B. L. (2006, October). "Bittersweet knowledge": The long-term effects of an international experience. Journal of Nursing Education, 45, 412-419. Fast Facts Open Doors 2012. (2012). Retrieved from http://www.iie.org/ResearchandPublications/Open-Doors Fast Facts, Open Doors 2015. (2015). Retrieved from http://www.iie.org/Researchand Publications/Open-Doors Foronda, C. L., & Belknap, R. A. (2012). Short of transformation: American ADN students' thoughts, feelings, and experiences of studying abroad in a low-income country. International Journal of Nursing Education Scholarship, 9(1), 1-16. http://dx.doi.org/10.1515/1548-923X.2411 Funnell, S. C., & Rogers, P. J. (2011). Purposeful program theory (1st ed.). San Francisco: John Wiley & Sons. Grothus, U. (2009, May). Measuring impact: Evaluating program areas. Paper presented at the National Association for Foreign Student Advisors, Los Angeles, CA. Retrieved from http://www.iie.org/en/Research-and-Publications-and-Reports/IEE-Bookstore/MeasuringImpact AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 37 Harrowing, J. N., Gregory, D. M., O'Sullivan, P. S., Lee, B., & Doolittle, L. (2012). A critical analysis of undergraduate students' cultural immersion experiences. International Nursing Review, 494-501. http://dx.doi.org/ Herrin, C. (2004, Winter). It's time for advancing education abroad. International Educator, 13(1), 3-4.3. Kirkham, S. R., Hofwegen, L. V., & Pankratz, D. (2009). Keeping the vision: Sustaining social consciousness with nursing students following international learning experiences. International Journal of Nursing Education Scholarship, 6(1), 1-16. http://dx.doi.org/10.2202/1548-923X.1635 Kulbok, P. A., Mitchell, E. M., Glick, D. F., & Greiner, D. (2012). International experiences in nursing education: A review of the literature. International Journal of Nursing Education Scholarship, 9(1), 1-21. http://dx.doi.org/10.1515/1548-923X.2365 Larson, K. L., Ott, M., & Miles, J. M. (2010, Summer). International cultural immersion: En vivo reflections in cultural competence. Journal of Cultural Diversity, 17, 44-50. Leininger, M. M. (1988, 11/1/1988). Leininger's theory of nursing: Cultural care diversity and universality. Nursing Science Quarterly, 1, 152-160. Retrieved from Levine, M. (2009, May-June). Transforming experiences: Nursing education and international immersion programs. Journal of Professional Nursing, 25, 156-169. http://dx.doi.org/10.1016/j.profnurs.2009.01.001 Lewin, R. (2009). Introduction: The quest for global citizenship through study abroad. In R. Lewin (Ed.), The Handbook of Practice and Research in Study Abroad (pp. xiii-xxii). New York, N.Y.: Routledge. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 38 Lewin, R. (Ed.). (2009). Faculty beliefs and institutional values. Identifying and overcoming these obstacles to education abroad growth. The handbook of practice and research in study abroad (pp. 282-302). New York, NY: Routledge. Lipson, J. G., & Desantis, L. A. (2012). Current approaches to integrating elements of cultural competence in nursing education [Article]. Journal of Transcultural Nursing, 18(1), 10S20S. http://dx.doi.org/10.1177/1043659606295498 Mezirow, J. (2000). Learning as transformation: Critical perspectives on a theory in progress (1st ed.). [Kindle version]. Mitchell, G. (2013, April). Selecting the best theory to implement planned change. Nursing Management, 20(1), 32-37. National standards on culturally and linguistically appropriate services final report [Government Report]. (2001). Retrieved from http://www.hhs.gov Office of Global Engagement Mission and Vision statement. (n.d). Retrieved from global.utah.edu/oge/mission-and-vision.php Palmer, S., Wing, D., Miles, L., Heaston, S., & De la Cruz, K. (2011). Study abroad programs: Using alumni and graduate students as affiliate faculty. Nurse Educator, 38, 198-201. http://dx.doi.org/10.1097/NNE.0b013e3182a0e587 Purnell, L. (2002, July). The Purnell model for cultural competence. Journal of Transcultural Nursing, 13, 193-196. Read, C. Y. (2011). Semester Abroad Opportunities In Baccalaureate Nursing Programs. Journal of Professional Nursing, 27, 221-226. http://dx.doi.org/10.1016/j.prof.nurs.2011.03.005 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 39 Ryan, M., & Twibell, R. (2002, January). Outcomes of a transcultural nursing immersion experience: Confirmation of a dimensional matrix. Journal of Transcultural Nursing, 13, 30-39. http://dx.doi.org/10.1177/104365960201300106 Ryan, M., Twibell, R., Brigham, C., & Bennett, P. (2000, December). Learning to care for clients in their world, not mine. Journal of Nursing Education, 39, 401-408. Retrieved from Smedley, B. D., Stith, A. Y., & Nelson, A. R. (Eds.). (2003). Unequal treatment: Confronting racial and ethnic disparities in health care (full printed edition).: The National Academies Press. Smith, K., & Curry, K. (2011). Is it worth it? Measuring the long-term effects of an international experience for nursing students in Ecuador. Journal of Community Health Nursing, 28(1), 14-22. http://dx.doi.org/10.1080/07370016.2011.539080 Smith-Miller, C. A., Leak, A., Harlan, C. A., Dieckmann, J., & Sherwood, G. (2010, January/March). "Leaving the comfort of the familiar": Fostering workplace cultural awareness through short-term global experiences. Nursing Forum, 45, 18-28. Strait, J. R., & Lima, M. (Eds.). (2009). The future of service learning (1st ed.). Sterling, VA.: Stylus Publishing. The NASULGC Commission on International Programs. (2007). A national action agenda for internationalizing higher education. https://www.aplu.org/NetCommunity/Document.Doc?id=471 The non-negotiable nature of the ANA code for nurses with interpretive statements [Revised position statement]. (1994). Retrieved from http://nursingworld.org University of Utah mission statement. (n.d.). http://admin.utah.edu/office_of_the_president/university-mission-statement AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 40 VandeBerg, M. (n.d.). A research based approach to education abroad classification. Retrieved from http://www.forumea.org/research-position_papers.cfm Veras, M., Pottie, K., Welch, V., Labonte, R., Eslava-Schmalbach, J., Borkhoff, C. M., ... Tugwell, P. (2013). Reliability and validity of a new survey to assess global health competencies of health professionals. Global Journal of Health Science, 5(1), 13-27. http://dx.doi.org/10.5539/gjhs.v5n1p13 Walsh, L. V., & DeJoseph, J. (2003, June). "I saw it in a different light": International learning experiences in baccalaureate nursing experiences. Journal of Nursing Education, 42, 266272. Wanner, D. (2009). Study abroad and language. In R. Lewin (Ed.), The handbook of practice and research in study abroad (pp. 81-98). New York, N.Y.: Routledge. Warren, N., Breman, R., Budhathoki, C., Farley, J., & Wilson, L. L. (2016, April). Perspectives of nursing faculty in Africa on global health nursing competencies. Nursing Outlook, 64, 179-185. http://dx.doi.org/10.1016/j.outlook.2015.11.016 Wells, M. I. (2000). Beyond cultural competence: A model for individual and institutional cultural development. Journal of Community Health Nursing, 17, 189-199. http://dx.doi.org/10.1207/S15327655JCHN1704_1 What is cultural and linguistic competence? [Government Report]. (2003). Retrieved from http://www.ahrq.gov/populations/cultcompdef.htm Wilson, L., Harper, D. C., Tami-Maury, I., Zarate, R., Salas, S., Farley, J., ... Ventura, C. (2012, July-August). Global health competencies for nurses in the Americas. Journal of Professional Nursing, 28, 213-222. http://dx.doi.org/10.1016/j.profnurs.2011.11.021 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix A Proposal Defense 41 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix A Proposal defense 42 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 43 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 44 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 45 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix B IRB Exemption 46 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix B IRB Exemption IRB: IRB_00069976 PI: Susan Kirby Title: An Assessment Toolkit for the University of Utah Ghana Study Abroad Program Thank you for submitting your request for approval of this study. The IRB has administratively reviewed your application and a designated IRB member has determined that your study is exempt from further IRB review, under Exemption Category 2. Note the following delineation of categories: • Categories 1-6: Federal Exemption Categories defined in 45 CFR 46.101(b) Categories 7-11: Non-Federal Exemption Categories defined in University of Utah IRB policy at http://irb.utah.edu/_pdf/IGS - Exempt Research 090113.pdf • You must adhere to all requirements for exemption described in University of Utah IRB policy (http://irb.utah.edu/_pdf/IGS - Exempt Research 090113.pdf). This includes: • All research involving human subjects must be approved or determined exempt by the IRB before the research is conducted. • All research activities must be conducted in accordance with the Belmont Report and must adhere to principles of sound research design and ethics. • Orderly accounting and monitoring of research activities must occur. Ongoing Submissions for Exempt Projects • Continuing Review: Since this determination is not an approval, the study does not expire or need continuing review. This determination of exemption from continuing IRB review only applies to the research study as submitted to the IRB. You must follow the protocol as proposed in this application • Amendment Applications: Substantive changes to this project require an amendment application to the IRB to secure either approval or a determination of exemption. Investigators should contact the IRB Office if there are questions 47 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH • about whether an amendment consists of substantive changes. Substantive changes include, but are not limited to o Changes that increase the risk to participants or change the risk: benefit ratio of the study o Changes that affect a participant's willingness to participate in the study o Changes to study procedures or study components that are not covered by the Exemption Category determined for this study (listed above) o Changes to the study sponsor o Changes to the targeted participant population o Changes to the stamped consent document(s) • Report Forms: Exempt studies must adhere to the University of Utah IRB reporting requirements for unanticipated problems and deviations: http://irb.utah.edu/submit-application/forms/index.php Final Project Reports for Study Closure: Exempt studies must be closed with the IRB once the research activities are complete: http://irb.utah.edu/submitapplication/final-project-reports.php If you have questions about this, please contact our office at 581-3655 and we will be happy to assist you. Thank you again for submitting your proposal. Click IRB_00069976 to view the application. Please take a moment to complete our customer service survey. We appreciate your opinions and feedback. 48 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix C Survey 49 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 50 Appendix C Survey SURVEY SUMMARY Outcomes measured Tool used Other measures Satisfaction; Cultural competence; Understanding of the challenges in a developing country; Diversity, Self; Clinical enhancement; Language skill; Transformation of thinking; Skills and knowledge; Reflection of experience; Population health assessment Course evaluation; Faculty developed tool; Host country tool; Journaling; Global Perspectives Inventory; Caffrey Healthcare Cultural Competence Instrument, for Assessing the Process of Cultural Competence Among Healthcare ProfessionalStudent Version Essay prompts; Review of student submissions Weakness of method Strength of method Time consuming; Not specific to experience; Does not capture socio-emotional experience; Poor results on statistical factor analysis and tool discarded Richness of qualitative data; University distributes it; Captures themes not obtained from quantitative analysis; Journals capture more depth and self- reflection; Students can share experience; "At least we try". AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Survey Monkey #1 Tuesday, February 16, 2016 5:44:22 AMTuesday, February 16, 2016 5:44:22 AM Tuesday, February 16, 2016 5:46:46 AMTuesday, February 16, 2016 5:46:46 AM 00:02:2300:02:23 [e-mail]@rutgers.edu 68.81.117.18368.81.117.183 PAGE 3: Screener do you measure outcomes Q1: Do you measure student outcomes in your study Yes abroad program? PAGE 4: Don't Measure Student Outcomes Q2: Is there a reason why you do not measure student skipped this outcomes? question Respondent PAGE 5 Q3: What outcomes do you measure in your program? Satisfaction PAGE 6: Do Measure Outcomes Q4: Do you use a tool to measure student outcomes? PAGE 7: Do you use a tool? Q5: What tool do you use to measure outcomes? Yes 51 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 52 Course evaluation PAGE 8 Q6: If you do not use a tool, how do you measure Respondent skipped this outcomes? question PAGE 9: What problems? Q7: What weaknesses have you identified with your method of measuring outcomes? If you use a tool, what are the weaknesses of the tool? Not specific to experience PAGE 10: What strengths? Q8: What strengths have you identified with your method of measuring outcomes?If you use a tool, what are the strengths of the tool? University distributes PAGE 11: End Q9: If you would like to receive a copy of the results of [e-mail] this survey or of the toolkit that is developed, please leave your email contact information and I will be happy to share it with you. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH #2 Tuesday, February 16, 2016 6:58:19 AMTuesday, February 16, 2016 6:58:19 AM Tuesday, February 16, 2016 7:05:06 AMTuesday, February 16, 2016 7:05:06 AM 00:06:4700:0 6:47 stone.3@osu. edustone.3@ osu.edu 104.230.130.98104.230.130.98 PAGE 3: Screener do you measure outcomes Q1: Do you measure student outcomes in your study Yes abroad program? PAGE 4: Don't Measure Student Outcomes Q2: Is there a reason why you do not measure student skipped this outcomes? question Respondent PAGE 5 Q3: What outcomes do you measure in your program? Cultural competency understanding of health care challenges in a developing country clinical enhancement Spanish language enhancement PAGE 6: Do Measure Outcomes Q4: Do you use a tool to measure student outcomes? PAGE 7: Do you use a tool? Yes 53 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 54 Q5: What tool do you use to measure outcomes? faculty developed assessment tool and In country host tool PAGE 8 Q6: If you do not use a tool, how do you measure outcomes? Respondent skipped this question PAGE 9: What problems? Q7: What weaknesses have you identified with your method of measuring outcomes? If you use a tool, what are the weaknesses of the tool? Does not really capture the socio-emotional aspects of an out of country experience in a developing country. We are in Honduras in the southern region with many rural poor with no access to health care. PAGE 10: What strengths? Q8: What strengths have you identified with your method of measuring outcomes?If you use a tool, what are the strengths of the tool? We at least attempt to capture the experience PAGE 11: End Q9: If you would like to receive a copy of the results of [e-mail] this survey or of the toolkit that is developed, please leave your email contact information and I will be happy to share it with you. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 55 #3 Tuesday, February 16, 2016 7:37:41 AMTuesday, February 16, 2016 7:37:41 AM Tuesday, February 16, 2016 7:41:13 AMTuesday, February 16, 2016 7:41:13 AM 00:03:3100:03:31 phillipslm@missouri.e duphillipslm@mi ssouri.edu 162.104.171.205 162.104.171.205 PAGE 3: Screener do you measure outcomes Q1: Do you measure student outcomes in your study Yes abroad program? PAGE 4: Don't Measure Student Outcomes Q2: Is there a reason why you do not measure student skipped this outcomes? question Respondent PAGE 5 Q3: What outcomes do you measure in your program? We used a qualitative process to compare pre and post responses to essay prompts. We were looking for themes related to transformation in thinking and progression through the stages of cultural competency. PAGE 6: Do Measure Outcomes Q4: Do you use a tool to measure student outcomes? PAGE 7: Do you use a tool? No AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Q5: What tool do you use to measure outcomes? 56 Respondent skipped this question PAGE 8 Q6: If you do not use a tool, how do you measure outcomes? Again, qualitative responses to essay prompts. I would not consider that a "tool". We utilized NVivo to analyze the narratives of the students. PAGE 9: What problems? Q7: What weaknesses have you identified with your method of measuring outcomes? If you use a tool, what are the weaknesses of the tool? Time consuming to analyze. PAGE 10: What strengths? Q8: What strengths have you identified with your method of measuring outcomes?If you use a tool, what are the strengths of the tool? Qualitative data is so rich and captures themes you would not get from a likert scale type of tool. PAGE 11: End Q9: If you would like to receive a copy of the results of [e-mail] this survey or of the toolkit that is developed, please leave your email contact information and I will be happy to share it with you. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH #4 Tuesday, February 16, 2016 9:46:30 AMTuesday, February 16, 2016 9:46:30 AM Tuesday, February 16, 2016 9:48:41 AMTuesday, February 16, 2016 9:48:41 AM 00:02:1100:02:11 [e-mail]@westminstercollege.edu 67.2.13.21267.2.13.212 PAGE 3: Screener do you measure outcomes Q1: Do you measure student outcomes in your study Yes abroad program? PAGE 4: Don't Measure Student Outcomes Q2: Is there a reason why you do not measure student skipped this outcomes? question Respondent PAGE 5 Q3: What outcomes do you measure in your program? Understanding of diversity, Understanding of self PAGE 6: Do Measure Outcomes Q4: Do you use a tool to measure student outcomes? PAGE 7: Do you use a tool? Q5: What tool do you use to measure outcomes? Journaling Yes 57 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 58 PAGE 8 Q6: If you do not use a tool, how do you measure Respondent skipped this outcomes? question PAGE 9: What problems? Q7: What weaknesses have you identified with your method of measuring outcomes? If you use a tool, what are the weaknesses of the tool? It may not exactly capture what we are looking for. PAGE 10: What strengths? Q8: What strengths have you identified with your method of measuring outcomes?If you use a tool, what are the strengths of the tool? Journals provide opportunities (especially using guided questions) to have the students go in to more self reflection depth. PAGE 11: End Q9: If you would like to receive a copy of the results of Respondent skipped this this survey or of the toolkit that is developed, please question leave your email contact information and I will be happy to share it with you. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 59 #5 Monday, February 22, 2016 5:34:35 PMMonday, February 22, 2016 5:34:35 PM Monday, February 22, 2016 5:41:33 PMMonday, February 22, 2016 5:41:33 PM 00:06:5700:06:57 sheri_palmer@byu. edusheri_palm [e-mail] 128.187.112.1 0128.187.112. 10 PAGE 3: Screener do you measure outcomes Q1: Do you measure student outcomes in your study Yes abroad program? PAGE 4: Don't Measure Student Outcomes Q2: Is there a reason why you do not measure student skipped this outcomes? question Respondent PAGE 5 Q3: What outcomes do you measure in your program? Cultural Competency, skills and knowledge PAGE 6: Do Measure Outcomes Q4: Do you use a tool to measure student outcomes? PAGE 7: Do you use a tool? Yes AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 60 Q5: What tool do you use to measure outcomes? have used Global Perspectives Inventory, Campinha-Bacote, and Caffrey GPI and Campinha are copyrighted. See http://scholarcommons.usf.edu/cgi/viewcontent.cgi?article=4307&context=etd PAGE 8 Q6: If you do not use a tool, how do you measure Respondent skipped this outcomes? question PAGE 9: What problems? Q7: What weaknesses have you identified with your method of measuring outcomes? If you use a tool, what are the weaknesses of the tool? Caffrey tool we did statistical factor analysis with results being poor. We have not used that tool again. PAGE 10: What strengths? Q8: What strengths have you identified with your method of measuring outcomes?If you use a tool, what are the strengths of the tool? we compare between groups (where they do their study abroad), and between years. This helps us determine if there is large variance between experiences (teachers, locations, etc.) Also measuring outcomes can provide positive data for sustainability of programs. PAGE 11: End Q9: If you would like to receive a copy of the results of [e-mail] this survey or of the toolkit that is developed, please leave your email contact information and I will be happy to share it with you. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH #6 Monday, June 06, 2016 8:20:57 AMMonday, June 06, 2016 8:20:57 AM Monday, June 06, 2016 9:39:35 AMMonday, June 06, 2016 9:39:35 AM 01:18:3801:18:3 8 am.linares@uky .eduam.linares @uky.edu 128.163.7.1501 28.163.7.150 PAGE 3: Screener do you measure outcomes Q1: Do you measure student outcomes in your study Yes abroad program? PAGE 4: Don't Measure Student Outcomes Q2: Is there a reason why you do not measure student skipped this outcomes? question Respondent PAGE 5 Q3: What outcomes do you measure in your program? experience reflexion, population Health assesment PAGE 6: Do Measure Outcomes Q4: Do you use a tool to measure student outcomes? PAGE 7: Do you use a tool? No 61 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Q5: What tool do you use to measure outcomes? 62 Respondent skipped this question PAGE 8 Q6: If you do not use a tool, how do you measure outcomes? review material submitted by the students PAGE 9: What problems? Q7: What weaknesses have you identified with your method of measuring outcomes? If you use a tool, what are the weaknesses of the tool? no much, students are able to share their experience with peers in the Public Health course PAGE 10: What strengths? Q8: What strengths have you identified with your method of measuring outcomes?If you use a tool, what are the strengths of the tool? It is a great way student share their experience PAGE 11: End Q9: If you would like to receive a copy of the results of Respondent skipped this this survey or of the toolkit that is developed, please question leave your email contact information and I will be happy to share it with you. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix D Assessment Toolkit for a Nursing Study Abroad Program 63 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix D Assessment Toolkit for a Nursing Study Abroad Program ASSESSMENT TOOLKIT for a NURSING STUDY ABROAD PROGRAM 64 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 65 Introduction This purpose of this toolkit is to provide information about tools that are available to measure outcomes in nursing study abroad programs. Five outcomes were identified in the nursing literature as resulting from study abroad programs in nursing. 1. Cultural competence. 2. Global citizenship and global health competencies. 3. Personal and professional growth. 4. Impact on practice. 5. Transformative learning. Tools were evaluated prior to inclusion by the following criteria: 1. The tool is validated and psychometric data is available for review. 2. The tool measures identified outcomes. 3. The tool evaluates students. 4. The tool is available for review prior to use at no cost. Theoretical foundation Lewin's theory of change provides the theoretical foundation for use of the tools in this toolkit. Lewin's theory of change delineates the steps by which individuals make personal or professional changes in thoughts or actions. It is a stepwise process that involves the incorporation of new ideas or information into existing ways of thinking or being. Those new ideas or experiences may occur in a formal or informal setting, and may evolve from an educational process or from a personal experience that may be transformative. The theory AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 66 describes driving forces that push toward change and that are opposed by restraining forces. If these forces are in equilibrium, no change occurs. If driving forces are stronger, change will occur; if restraining forces are stronger no change will occur. Even when change has occurred, over time the presence of restraining forces may push back toward the previous status quo (Current nursing [Current], 2011). The 3 stages of Lewin's change theory are unfreezing, change or moving, and refreezing. Unfreezing involves an examination of how things are or have always been. During this stage driving forces must be in place for the next stage to occur. Change, or moving, is the stage when the driving forces overwhelm restraining forces and there is movement in the direction of change or outcomes. Refreezing is the stage when the change hopefully becomes permanent. This is the stage where desired outcomes may become long-term. However, it is also a stage when restraining forces may push back and desired outcomes may regress (Mitchell, 2013). Literature Review: Assessment tools Cultural Competence The National Center for Cultural Competence promotes self-assessment of cultural competence "as a vehicle to measure outcomes for personnel, organizations, population groups and the community at large" (National Center for Cultural Competence [NCCC], n.d.). A web search for self- assessment tools results in dozens of tools. However, while there are many selfassessment tools available, most are very specific to the goals of organizational assessment and to the specific assessment of health care providers within an organization. While some indicate validity and reliability, many do not. In 2007, Gozu et al conducted an extensive search for and review of tools for self-evaluation of cultural competence of health care professionals. Forty-five tools were identified. Only 15 had confirmed reliability or validity, and only 6 had confirmed AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 67 reliability and validity. Shen (2015) identified 15 cultural competence assessment instruments in the nursing literature. Ten were identified with reliability and validity. However, many were from the international literature or had other limitations. Over 30 instruments and websites were evaluated for this project. Many tools are available for purchase but were not available on request, without cost, due to their proprietary nature. As a result of these limitations, most tools or websites identified are not appropriate for outcomes evaluation in study abroad students. The number of tools that are appropriate for this purpose and that are available according to the criteria of this project, is actually quite small. The Inventory for Assessing the Process of Cultural Competence Among Healthcare Professionals-Student Version In 2007 Dr. J. Campinha-Bacote developed the Inventory for Assessing the Process of Cultural Competence Among Healthcare Professionals-Student Version (IAPCC-SV). As indicated in the title, Dr. Campinha-Bacote stresses that cultural competence is a process, not a defined goal. The constructs of this model, and of the process of cultural competence are: cultural desire, cultural knowledge, cultural awareness, cultural skill and cultural encounters (Campinha-Bacote, 2003). The IAPCC-SV tool is designed for health professions students and measures the five constructs in the model. This is accomplished by a paper and pencil self-assessment, using a 4point Likert scale. The tool reportedly takes 10-15 minutes to complete, and scores correlate with a described level of cultural competence i.e.: cultural incompetence, cultural awareness, cultural competence, or cultural proficiency. Psychometric testing of the tool showed "a moderate amount of range and variability" and "adequate" internal consistency reliability for the overall scale (Fitzgerald, Cronin, & AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 68 Campinha-Bacote, 2009, p. 67). Cronbach's alpha for the individual subscales which reflect the five constructs ranged from .189 to .684, indicating low internal consistency. Due to this, the original subscales were re-conceptualized into two larger subscales. This resulted in "acceptable" internal consistencies of the two scales with alpha co-efficient of .735 (Fitzgerald et al., 2009, p. 27). Construct validity was evaluated utilizing comparison of two sub-groups of nursing students; traditional and second degree. No significant difference was found between groups, except where difference was expected with t= -.02, df= 84, p=.986 (Fitzgerald et al., 2009, p. 27). Interestingly, construct validity was also evaluated by comparison of white vs non-white students. Non-white students scored significantly higher. It was hypothesized that non-white students have life experience that generates self- examination, openness to the experience of others, and an overall more diverse cultural experience. To evaluate this concept, the authors generated a cumulative cultural experience score for each student, based on life experiences such as travel, military experience etc., and including study abroad and cultural immersion experiences. Higher cumulative cultural experience scores were strongly correlated with higher cultural competence scores with r= .309, and p= .004 (Fitzgerald et al., 2009, p. 27). This would seem to support the literature that describes study abroad as a source of cultural competence development. A copy of the tool is included for review in the book "The Process of Cultural Competence in the Delivery of Health Care Services: The Journey Continues" by Dr. CampinhaBacote (2003). However, permission and fee are required for other than personal use. Cost is $8.00 each when personally hand administered and $20.00 when non-hand administered. A AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 69 formal letter of request for permission to use the tool must be included. The tool and book may be ordered from the website: transcultural care.net. The Transcultural Self-Efficacy Tool The Transcultural Self-Efficacy Tool (TSET) measures the degree of cultural selfefficacy in nursing students. This tool has been utilized to measure the development of cultural competence in study abroad nursing students (Allen et al., 2013, Amerson, 2010). Transcultural self-efficacy is defined as the confidence to learn or perform transcultural nursing skills. The TSET measures and evaluates the confidence that nursing students have for performing transcultural nursing skills in diverse populations (Jeffreys, 2000). The tool has 83 questions that utilize a 10-point rating scale from not confident to totally confident. Three subscales: Cognitive, Practical and Affective, reflect transcultural nursing skills that were identified in the literature and confirmed in a two-phase evaluation study. The Cognitive domain uses 25 questions to measure confidence of an individual's cultural knowledge and how that knowledge impacts health care. The Practical domain uses 28 questions to measure confidence in interviewing clients of different cultural backgrounds, and the Affective domain measures attitudes, values and beliefs with 30 questions (Jeffreys, 2016). Estimated completion time is 20 minutes. The TSET has been used extensively since its development in 1994. It has been translated into multiple languages, and has undergone repeated psychometric testing. Six doctorally prepared nurses certified in transcultural nursing evaluated the tool, establishing content validity (Jeffreys & Smodlaka, 1996). Construct validity was established through multiple studies that utilized a contrasted group approach, generally comparing novice and advanced nursing students. A cross-sectional study and a longitudinal study showed similar results, indicating statistically significant differences between groups. However, differences were found in 2 of the 3 sub-scales AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 70 in the cross-sectional study (t-test, p<.05) and in all 3 of the sub-scales in the longitudinal study (p<.001), (Jeffreys, 2000, p 133). While differences were found in all 3 of the sub-scales in the cross-sectional study, they were not statistically significant in the Affective domain. Factor analysis established validity of items and subscales. Interim correlation matrix showed correlation between .30-.70; intercorrelations between subscale means showed correlation ranging between .53-.68 on individual scales with p<.001; and principal component analysis indicated that the 3 dimensions measured are distinct from each other, that the tool measures all 3 domains of learning (Jeffreys, 2000; Jeffreys & Smodlaka, 1998), and that it also "assesses the multi-dimensional nature of cultural self-efficacy" (Jeffreys, 2000, p. 133). Factor analysis was repeated with updated tools, specifically common exploratory factor analysis (CEFA). This showed that the tool does measure transcultural self-efficacy and that it differentiates between the three subscales of learning (Jeffreys & Dogan, 2010). Criterion related validity was measured using examination of predictive validity. This was accomplished through a longitudinal study that found statistically significant differences of transcultural self-efficacy (TSE) perception at two points in time, specifically the difference between TSE perception in the first and the fourth clinical semester (Jeffreys, 2016). Internal consistency of the TSET has been measured with split-half reliability and Cronbach's alpha. In early studies split-half reliability coefficients of .76-.92 were found for subscales and for the total instrument. These coefficients exceeded acceptability for new instruments and established instruments. Cronbach's alpha for subscales and total instrument also consistently indicated excellent internal consistency with alpha co-efficients of .92-.98 (Jeffreys, 2000). Updated studies utilizing CEFA continued to support internal consistency as well as instrument reliability (Jeffreys & Dogan, 2010). Internal consistency and stability have been AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 71 extensively evaluated and have demonstrated instrument and subscale reliability that persists even when the tool is translated (Jeffreys, 2016). The TSET is available as a portion of The Cultural Competence Education Resource Toolkit. This can be accessed for purchase on www.springer.com/cctoolkit. The cost is $350.00 for a one year renewable license, and allows the user to administer the TSET to up to 500 people. The toolkit is comprehensive and includes 12 questionnaires (including TSET), the founding model and transcultural efficacy pathway, and 9 self- assessment and planning tools. Content of the toolkit is available for review on the website and in her book Teaching Cultural Competence in Nursing and Healthcare. Global Health Competencies Many assessment tools are available for purchase for the evaluation of global competence and may be administered as pre-test and post-test to students who participate in study abroad. The Global Competence Inventory, the Global Competence Aptitude Assessment and the Global Perspective Inventory are examples of generalized assessment tools, but they do not address global health competencies. A request to review these tools was denied due to their proprietary nature and so they not available for review without cost. Global health competency models have been published for various medical disciplines and for public health education. In 2009, two of these models were adapted to develop a list of potential global health competencies for undergraduate nursing students. These tools were utilized to develop a survey for nursing faculty. Six subscales and 30 competencies were evaluated by nursing faculty from the US, Canada, the Caribbean and Latin America to develop a list of appropriate global health competencies for nurses. All items surveyed were approved for inclusion with a mean value of 3 for all (scale 1-4 from strongly disagree to strongly agree) AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 72 (Wilson et al., 2012). At this time, no evaluation tool for measurement of these particular competencies has been developed. Further development of global health competencies for nurses has been completed (Clark, Raffray, Hendricks, & Gagnon, 2016; Warren, Breman, Budhathoki, Farley, & Wilson, 2016). In 2013, a CUGH subcommittee began work on the development of multi-disciplinary global health core competencies. Two levels of competency were delineated. The Global Citizen level incorporates 8 domains and 13 competencies and the Basic Operational Program-Oriented level incorporates 11 domains and 39 competencies (Jogerst et al., 2015). Again, no evaluative tools have yet been developed for these competencies. The Global Health Competency Survey is a unique self -assessment tool, developed in 2013 for the evaluation of global health competencies in multiple health professions, including nursing. Item development utilized existing instruments developed for health professionals, existing global health competency tools, a validated questionnaire to measure resident physician knowledge of underserved populations and expert consultation (Veras et al., 2013). 465 Students from medicine, nursing, physiotherapy and occupational therapy from 5 universities in Ontario, Canada were surveyed to determine reliability and validity of the questionnaire. The final tool contained 22 competency items and 8 demographic questions. Content validity was established with expert opinion, and evaluation of floor and ceiling effects for each category. Only 1 variable showed a floor effect and 7 performed best. The overall scale showed no floor or ceiling effects. Face validity was established by expert analysis and was determined to be adequate. Exploratory factor analysis was conducted to determine retained items. Internal consistency judged reliability of the tool to be good with Cronbach's alpha coefficient of 0.862 for the 22 items (Veras et al., 2013). The questionnaire is appended to "Reliability and Validity AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 73 of a New Survey to Assess Global Health Competencies of Health Professionals, Veras et al, 2013. The tool is available by permission of the lead author. Personal and Professional Growth Measurement of personal and professional growth has utilized reflective journals (Edmonds, 2012; Walsh & DeJoseph, 2003), semi-structured interviews (Green, Comer, Elliott, & Neubrander, 2011; Levine, 2009; Bosworth et al., 2006; Walsh & DeJoseph, 2003), focus groups, and questionnaires with narrative response (Evanson & Zust, 2006; Ryan & Twibell, 2002; Ryan et al., 2000). These were specific to individual programs and are not available or appropriate for generalized use. Impact on Practice Similar qualitative approaches have been used to measure impact on practice; some studies measured both outcomes (Bosworth et al.; 2006 Ryan & Twibell, 2002; Ryan et al., 2000). Focus groups (Adamshick & August-Brady, 2012), reflective journals, interview, and written response to questions (Adamshick & August-Brady, 2012, Carpenter & Garcia, 2012), journals, and reflection papers (Smith-Miller et al., 2010; Conroy & Taggart, 2016) have all been utilized. These have the same limitations discussed above in terms of generalized use. The International Education Survey (IES) is a unique quantitative tool developed in 1996 to measure the long-term effect of international study (Zorn, 1996). While alpha coefficients have been measured for application of the survey, it was not evaluated prior to use and validity has not been measured. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 74 Transformative Learning The breadth and depth of literature on transformative learning reflects the growth in interest in since the inception of the original theory. While Mezirow's theory has been expanded and changed, the means of measurement remains the same. Again, reflective journaling or writing, narrative inquiry, interviews, and story-telling are employed in the evaluation of transformative learning in nursing study abroad (Kirkham et al., 2009; Levine, 2009; Adamshick & August-Brady, 2012). These are subject to the same limits with regards to generalizability as the methods described above. In essence, "as is characteristic of qualitative research, the researcher was the instrument for data collection and analysis" (Kear, 2013, p. 1084). AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 75 Quick Tool Reference Inventory for Assessing the Process of Cultural Competence Among Healthcare Professionals-Student Version (IAPCC-SV) What it measures How it measures Psychometric properties Information Measures the 5 constructs of the CampinhaBacote model of cultural competence Paper and pencil self-assessment 20 questions 4 point Likert scale Scores correlate with a described level of cultural competence "moderate amount of range and variability" Two subscales reflect 5 constructs: Internal consistency/reliability: Cronbach's alpha: 0.735 Construct validity: Compared 2 subgroups: no significant difference T=-.02 df: 84 p=.986 Construct validity: 2 steps White v non-white and comparison of levels of cumulative cultural experience r=.309 p=.004 Copy for review in The Process of Cultural Competence in the Delivery of Health Care Services: The Journey Continues Transculturalcare.net Copyrighted. Permission and fee required $8.00 each when hand administered $20.00 each when non-hand administered Formal letter of request for permission required AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 76 Transcultural Self-Efficacy Tool (TSET) What it measures How it measures Psychometric properties Information Measures the degree of cultural self-efficacy in nursing students i.e. the confidence to learn/perform transcultural nursing skills AND the confidence to perform them in diverse populations Cognitive, Affective and Practical domains Paper and pencil self-assessment 83 questions 3 subscales 10 point Likert scale "not confident" - "confident" Repeat psychometrics Content validity Construct validity. Multiple studies: contrasted group approach, X-sectional, longitudinal. Showed differences between groups. Factor analysis: showed validity of items and subscales: - interim correlation matrix .30-.70 -inter-correlations between subscale means .53.68 -principal component analysis: showed 3 distinct dimensions, measures all 3 domains of learning and assesses dimensions of self-efficacy Updated with Common Exploratory Factor Analysis (CEFA): showed that tool measures selfefficacy and distinguishes 3 domains Criterion related validity: showed ss difference at 2 points in time Internal consistency/reliability: Cronbach's alpha: total instrument and subscales .92-.98 Split ½ reliability coefficients .76-.92 Updated with CEFA: shows internal consistency, instrument reliability, stability even when translated Available as a portion of The Cultural Competence Education Resource Toolkit. Copyrighted. $350.00 for 1 year renewable license allows administration to 500 people. Copy for review or purchase at www.springer.com/cctoolkit or in Teaching Cultural Competence in Nursing and Healthcare AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 77 Global Health Education Survey What it measures How it measures Psychometric properties Information Global health competencies Paper and pencil self-assessment 22 Global health competency items, 8 demographic questions 4 parts: 1. Knowledge in global health and health equity3 point Likert scale: "not at all confident"- Very confident" 2. Global health skills for working with different language, educational, socioeconomic and cultural backgrounds5 point Likert scale: "Strongly disagree"- "Strongly agree" 3. Global health learning needs5 point Likert scale: "not at all important""Extremely important" 4. Demographics Face Validity: expert panel Content Validity: Floor and ceiling effects: For individual items 1 floor effect (36.1%) 7 ceiling effects (34%-59.6%). For the overall rating score no participants with floor or ceiling effects. Factor analysis: Eigenvalues and cumulative variance confirmed factor structure. 5 factors retained with multiple subscales. Internal consistency Cronbach's alpha: >0.862 for entire scale including 5 factors. Available with permission from lead author: [e-mail]; [e-mail] AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 78 References Adamshick, P., & August-Brady, M. (2012, May-June). Reclaiming the essence of nursing: The meaning of an immersion experience in Honduras for RN to Bachelor of Science students. Journal of Professional Nursing, 28, 190-198. http://dx.doi.org/10.1016/j.profnurs.2011.11.011 Allen, C. B., Smart, D. A., Odom-Maryam, T., & Swain, D. (2013). The value of communityfocused interprofessional care in Peru for developing cultural competency in health profession students. International Journal of Nursing Education, 10(1), 1-10. http://dx.doi.org/10.1515/ijnes-2012-0014 Amerson, R. (2010). The impact of service learning on cultural competence. Nursing Education Perspectives, 31, 18-22. Retrieved from journals.lww.com/neponline/pages/default.aspx Bosworth, T. L., Haloburdo, E. P., Hetrick, C., Patchett, K., Thompson, M. A., & Welch, M. (2006, January/February). International partnerships to promote quality care: Faculty groundwork, student projects, and outcomes. The Journal of Continuing Education In Nursing, 37, 32-38. Campinha-Bacote, J. (2003). The process of cultural competence in the delivery of healthcare services. In The process of cultural competence in the delivery of healthcare services: A culturally competent model of care (4th ed. (pp. 15-20). Cincinnati, OH: Transcultural C.A.R.E. Associates. Carpenter, L. J., & Garcia, A. A. (2012, March/April). Assessing outcomes of a study abroad course for nursing students. Nursing Education Perspectives, 33, 85-89. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 79 Clark, M., Raffray, M., Hendricks, K., & Gagnon, A. J. (2016). Global and public health core competencies for nursing education: A systematic review of essential competencies. Nurse Education Today, 40, 173-180. http://dx.doi.org/10.1016/j.nedt.2016.02.026 Conroy, S. F., & Taggart, H. M. (2016, August 1). The impact of a cultural immersion study abroad experience in traditional Chinese medicine. Journal of Holistic Nursing, 34(3), 229-235. http://dx.doi.org/10.1177/0898010115602995 Current nursing. (2011). Change theory: Kurt Lewin. Retrieved from http://currentnursing.com_theory/change_theory.html Edmonds, M. (2012, January/February). An integrative literature review of study abroad programs for nursing students. Nursing Education Perspectives, 33, 30-34. Evanson, T. A., & Zust, B. L. (2006, October). "Bittersweet knowledge": The long-term effects of an international experience. Journal of Nursing Education, 45, 412-419. Fitzgerald, E. M., Cronin, S. N., & Campinha-Bacote, J. (2009). Psychometric testing of the Inventory for Assessing the Process of Cultural Competence Among Healthcare Professionals- Student Version (IAPCC-SV). The Journal of Theory Construction and Testing, 13(2), 64-68. Retrieved from tuckerpub.com/jtct.htm Gozu, A., Beach, M. C., Price, E. G., Gary, T. L., Robinson, K., Palacio, A., ... Cooper, L. A. (2007). Self-administered instruments to measure cultural competence of health professionals: A systematic review. Teaching and Learning in Medicine, 19, 180-190. http://dx.doi.org/10.1080/10401330701333654 Green, S. S., Comer, L., Elliott, L., & Neubrander, J. (2011, September/October). Exploring the value of an international service learning experience in Honduras. Nursing Education Perspectives, 32, 302-307. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 80 Jeffreys, M. R. (2000). Development and psychometric evaluation of the transcultural selfefficacy tool: A synthesis of findings. Journal of Transcultural Nursing, 11, 127-136. Retrieved from tcn.sagepub.com Jeffreys, M. R. (2016). Transcultural self-efficacy tool. In Teaching Cultural Competence in Nursing and Health Care (3rd ed. (pp. 91-139). New York, NY: Springer Publishing Company. Jeffreys, M. R., & Dogan, E. (2010). Factor analysis of the transcultural self-efficacy tool (TSET). Journal of Nursing Measurement, 18, 120-139. http://dx.doi.org/10.1891/10613749.18.2.120 Jeffreys, M. R., & Smodlaka, I. (1996). Steps of the instrument design process: An illustrative approach for nurse educators. Nurse Educator, 21(6), 47-52. Retrieved from http://journals.lww.com/nurseeducator/pages/default.aspx Jeffreys, M. R., & Smodlaka, I. (1998). Exploring the factorial composition of the transcultural self-efficacy tool. International Journal of Nursing Studies, 35, 217-225. Retrieved from http://www.journalofnursingstudies.com Jogerst, K., Cellender, B., Adams, V., Evert, J., Fields, E., Hall, T., ... Wilson, L. L. (2015, March-April). Identifying interprofessional global health competencies for 21st-century health professionals. Annals of Global Health, 81, 239-247. http://dx.doi.org/10.1016/j.aogh.2015.03.006 Kear, T. M. (2013). Transformative learning during nursing education. Nurse Education Today, 33, 1083-1087. http://dx.doi.org/10.1016/j.nedt.2012.03.016 Kirkham, S. R., Hofwegen, L. V., & Pankratz, D. (2009). Keeping the vision: Sustaining social consciousness with nursing students following international learning experiences. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 81 International Journal of Nursing Education Scholarship, 6(1), 1-16. http://dx.doi.org/10.2202/1548-923X.1635 Levine, M. (2009, May-June). Transforming experiences: Nursing education and international immersion programs. Journal of Professional Nursing, 25, 156-169. http://dx.doi.org/10.1016/j.profnurs.2009.01.001 National Center for Cultural Competence. (n.d.). Self Assessments. Retrieved from nccc.georgetown.edu/resources/assessments.html Ryan, M., & Twibell, R. (2002, January). Outcomes of a transcultural nursing immersion experience: Confirmation of a dimensional matrix. Journal of Transcultural Nursing, 13, 30-39. http://dx.doi.org/10.1177/104365960201300106 Ryan, M., Twibell, R., Brigham, C., & Bennett, P. (2000, December). Learning to care for clients in their world, not mine. Journal of Nursing Education, 39, 401-408. Shen, Z. (2015). Cultural competence models and cultural competence assessment instruments in nursing: A literature review. Journal of Transcultural Nursing, 26, 308-321. http://dx.doi.org/10.1177/1043659614524790 Smith-Miller, C. A., Leak, A., Harlan, C. A., Dieckmann, J., & Sherwood, G. (2010, January/March). "Leaving the comfort of the familiar": Fostering workplace cultural awareness through short-term global experiences. Nursing Forum, 45, 18-28. Veras, M., Pottie, K., Welch, V., Labonte, R., Eslava-Schmalbach, J., Borkhoff, C. M., ... Tugwell, P. (2013). Reliability and validity of a new survey to assess global health competencies of health professionals. Global Journal of Health Science, 5(1), 13-27. http://dx.doi.org/10.5539/gjhs.v5n1p13 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 82 Walsh, L. V., & DeJoseph, J. (2003, June). "I saw it in a different light": International learning experiences in baccalaureate nursing experiences. Journal of Nursing Education, 42, 266272. Warren, N., Breman, R., Budhathoki, C., Farley, J., & Wilson, L. L. (2016, April). Perspectives of nursing faculty in Africa on global health nursing competencies. Nursing Outlook, 64, 179-185. http://dx.doi.org/10.1016/j.outlook.2015.11.016 Wilson, L., Harper, D. C., Tami-Maury, I., Zarate, R., Salas, S., Farley, J., ... Ventura, C. (2012, July-August). Global health competencies for nurses in the Americas. Journal of Professional Nursing, 28, 213-222. http://dx.doi.org/10.1016/j.profnurs.2011.11.021 Zorn, C. R. (1996, March/April). The long-term impact on nursing students of participating in international education. Journal of Professional Nursing, 12, 106-110. AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix E Poster Presentation 83 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix E Poster Presentation 84 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH Appendix F Abstract Acceptance 85 AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 86 Appendix F Abstract Acceptance Abstract number: 659 Dear Susan, As previously informed your abstract 'An Assessment Toolkit for Measuring Outcomes in Nursing Study Abroad Programs' was accepted for POSTER PRESENTATION at the Eighth Annual Global Health Conference, April 7-9, 2017. We have received confirmation that you (or one presenter) will attend and present your poster. Posters will be assigned to dates at the end of January. Poster presentations will be assigned to either Friday, April 7 or Saturday, April 8 (not as below mentioned, Saturday or Sunday!) Over the next week, we will start checking if registrations and payments of presenters were received. If you (or the presenter) have not yet registered, please do so by December 18, 2016 (https://www.eiseverywhere.com/ereg/index.php?eventid=191756&). If, for some reason, you find out that you cannot participate, we appreciate if you let us know as there is a long waiting list for poster presentations and you can help another presenter to attend and present their work. Thank you for your cooperation. regards, Doris Steinbach CUGH 2016 Conference Manager ##################################################################### Your preliminary abstract number: 659 Dear Susan, On behalf of the Scientific Committee I am pleased to inform you that your abstract 'An Assessment Toolkit for Measuring Outcomes in Nursing Study Abroad Programs' was accepted for POSTER presentation at the CUGH 2017 Conference which will be held in Washington, DC, April 7 to 9, 2017. CONFIRMATION OF PRESENTATION: To secure your poster presentation slot you must confirm your presentation no later than December 18, 2016. If we do not hear from you by then, the slot will automatically be offered to another presenter and your poster will be removed from the program. The exact presentation date (Saturday or Sunday - wrong!!! ->> correct: Friday or Saturday!!!) will be emailed to you by the end of January. ONLINE CONFIRMATION PROCESS: link: https://esociety.netkey.at/isid/cugh/abstractsubmission/ --->> then use your existing login information that you created when you submitted your abstract (otherwise click on 'FORGOT YOUR LOGIN DATA' to obtain your login information) AN ASSESSMENT TOOLKIT FOR THE UNIVERSITY OF UTAH 87 Login to either CONFIRM or DECLINE the presentation. After you login your abstract title(s) will appear, click on the 'click here to confirm/decline' button in the lower right corner AND on the next page on the 'accept' or 'decline' button on the very bottom of the abstract. Please be advised of the following poster guidelines: • MAXIMUM SIZE OF POSTER: max. width 44'', max. height 45'' Format: Portrait • Posters will be presented either on Saturday, April 8 or on Sunday, April 9, 2017 during lunch time. Information with your exact presentation DATE will be emailed at the end of January! • The poster must be mounted on the day of your presentation between 08.30am and 9.30am at the latest and removed in the afternoon of the same day. Posters not removed by 6.30pm will be dismantled by the congress staff and we cannot guarantee the poster's condition. • At least ONE OF THE AUTHORS must be registered to present the poster during the poster viewing period during the lunch break. • The final POSTER NUMBER will be indicated in the FINAL PROGRAM which will be available onsite. • Posters can be affixed with pins which will be available in the poster area onsite. • As a reminder, all poster abstracts will be published by the 'Annals for Global Health' journal unless you indicated in the abstract submission form that permission is not given for publication. Abstracts will be published as entered by the submitters and will not be edited! If you have not yet registered (early registration fee deadline: February 13, 2017), please send your completed registration form and full payment as soon as possible. Online registration is available through our web site https://www.eiseverywhere.com/ereg/index.php?eventid=191756& . Presenters not registered by December 18, 2016 will be removed from the program. If you know that you will not be able to attend the congress we urge you to PLEASE send an e-mail to [e-mail] as soon as possible BUT NO LATER than December 18, 2016! Thank you for your cooperation as you will help another presenter to attend and present their work. Sincerely, Doris Steinbach CUGH 2017 Conference Manager