Navigating elementary teacher education tuning in utah

This exploratory study examines the faculty’s purposeful and creative work aimed at developing student learning outcomes and competencies for Elementary Teacher Education (ETE) in Utah. The research uses a single-case study design as a qualitative method to gain insight into the ETE Tuning process that was launched in Utah in 2011. Tuning is a college discipline level methodology of writing student learning outcomes and demonstrable competencies wherein faculty’s role is critical. The qualitative content analysis was conducted on data gathered through interviews, documents, field notes, and observations. The study explores the purposive work of actors to create, maintain, or disrupt institutions through the lens of institutional work analysis. The dissertation is organized in five chapters: introduction, literature review, methodology, findings, and discussion. Chapter 1 introduces the research: its context, purpose, research questions, and significance of the research. Chapter 2, literature review, consists of three sections: the research on quality models in higher education, the literature on Tuning as a current higher education reform, and the research on organizational change in higher education with special attention to institutional theory as a form of organizational analysis. Chapter 3, methodology, presents the rationale for the study, research design, the researcher’s positionality, data collection methods and data analysis plan, ethical and political considerations, and issues of trustworthiness of the proposed study. Chapter 4 contains the findings on ETE Tuning organized in three major sections: (1) developmental, dynamic, complex complicated, iterative process, (2) tenacious and tenuous collaboration, and (3) interrogating faculty practice. Chapter 5 discusses the research findings applied to leadership and the faculty Tuning work through institutional work. It also presents implications of the study for practice, policy and further research.

NAVIGATING ELEMENTARY TEACHER EDUCATION TUNING IN UTAH by Natalia Vladimirovna Ralyk A dissertation submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Educational Leadership and Policy The University of Utah December 2016 Copyright © Natalia Vladimirovna Ralyk 2016 All Rights Reserved Th e Un iversity of Utah Grad u ate S ch ool STATEMENT OF DISSERTATION APPROVAL The dissertation of Natalia Vladimirovna Ralyk has been approved by the following supervisory committee members: , Chair Andrea K. Rorrer 04/19/2016 Date Approved , Member Phyllis C. Safman 04/19/2016 Date Approved , Member Amy A. Bergerson 04/19/2016 Date Approved , Member Sharon Aiken Wisniewski 04/19/2016 Date Approved , Member Irene H. Yoon 04/19/2016 Date Approved and by Gerardo R. Lopez the Department/College/School of , Chair/Dean of Educational Leadership and Policy and by David B. Kieda, Dean of The Graduate School. ABSTRACT This exploratory study examines the faculty's purposeful and creative work aimed at developing student learning outcomes and competencies for Elementary Teacher Education (ETE) in Utah. The research uses a single-case study design as a qualitative method to gain insight into the ETE Tuning process that was launched in Utah in 2011. Tuning is a college discipline level methodology of writing student learning outcomes and demonstrable competencies wherein faculty's role is critical. The qualitative content analysis was conducted on data gathered through interviews, documents, field notes, and observations. The study explores the purposive work of actors to create, maintain, or disrupt institutions through the lens of institutional work analysis. The dissertation is organized in five chapters: introduction, literature review, methodology, findings, and discussion. Chapter 1 introduces the research: its context, purpose, research questions, and significance of the research. Chapter 2, literature review, consists of three sections: the research on quality models in higher education, the literature on Tuning as a current higher education reform, and the research on organizational change in higher education with special attention to institutional theory as a form of organizational analysis. Chapter 3, methodology, presents the rationale for the study, research design, the researcher's positionality, data collection methods and data analysis plan, ethical and political considerations, and issues of trustworthiness of the proposed study. Chapter 4 contains the findings on ETE Tuning organized in three major sections: (1) developmental, dynamic, complex complicated, iterative process, (2) tenacious and tenuous collaboration, and (3) interrogating faculty practice. Chapter 5 discusses the research findings applied to leadership and the faculty Tuning work through institutional work. It also presents implications of the study for practice, policy and further research. iv TABLE OF CONTENTS ABSTRACT ....................................................................................................................... iii LIST OF FIGURES ......................................................................................................... viii LIST OF TABLES ............................................................................................................. ix Chapters 1 INTRODUCTION ...................................................................................................1 Research Context .....................................................................................................2 Research Purpose .....................................................................................................5 Research Questions ..................................................................................................6 Significance of the Study .........................................................................................6 Conclusion ...............................................................................................................7 2 LITERATURE REVIEW ........................................................................................9 Introduction ..............................................................................................................9 Quality Models in Higher Education .......................................................................9 What is Tuning? .....................................................................................................13 Tuning Context ......................................................................................................16 Tuning Content ......................................................................................................24 Organizational Change...........................................................................................34 Analytical Framework ...........................................................................................40 3 RESEARCH DESIGN AND METHODS .............................................................51 Rationale for Qualitative Research ........................................................................52 Research Paradigm.................................................................................................54 Researcher's Positionality ......................................................................................56 Researcher as an Instrument ..................................................................................60 Design of the Study ................................................................................................63 Sampling ................................................................................................................67 Data Collection ......................................................................................................70 Data Management ..................................................................................................77 Data Analysis Strategies ........................................................................................79 Trustworthiness ......................................................................................................83 Ethical and Political Considerations ......................................................................87 Limitations .............................................................................................................89 Conclusion .............................................................................................................91 4 FINDINGS .............................................................................................................94 Introduction ............................................................................................................94 Developmental, Dynamic, Complex, Complicated, Iterative Process ...................96 Collaboration: Tenacious and Tenuous ...............................................................144 Tuning Requires Interrogating Faculty Practice ..................................................168 Conclusion ...........................................................................................................178 5 DISCUSSION ......................................................................................................189 Introduction ..........................................................................................................189 Leadership and ETE Tuning: Agents of Change .................................................199 The Utah ETE Tuning Faculty Expert Work .......................................................218 The Implications of ETE Tuning: A Mechanism of Enhancing Higher Education .............................................................................................................244 Conclusion ...........................................................................................................261 Appendices A: ENGAGEMENT THEORY OF PROGRAM QUALITY ..........................................267 B: INSTITUTIONS .........................................................................................................269 C: THE TUNING AND BOLOGNA PROCESS RELATIONSHIPS ............................271 D: NATIONAL QUALIFICATIONS FRAMEWORKS ................................................273 E: DYNAMIC RELATIONSHIPS AMONG TUNING STAKEHOLDERS ................275 F: INTERVIEW PROTOCOL .........................................................................................277 G: INFORMED CONSENT STATEMENT: TUNING ELEMENTARY TEACHER EDUCATION IN UTAH .................................................................................................282 H: TUNING REPORTS ...................................................................................................287 I: UTAH TUNING REPORTS ........................................................................................289 J: ETE TUNING MEETING NOTES LIST ....................................................................291 vi K: ELEMENTARY EDUCATION MAJOR'S MEETINGS NOTES LIST ..................293 REFERENCES ................................................................................................................295 vii LIST OF FIGURES Figures 1 Key Elements of Tuning Methodology...........................................................................92 2 Utah ETE Tuning Holistic Single-Unit Case Study .......................................................92 3 Convergence of Evidence. ..............................................................................................93 4 Developmental, Dynamic, Complex, Complicated, and Iterative Process. ..................181 5 The Utah ETE Tuning Project: Key Points and Timeline. ...........................................182 6 The Bases for the ETE Student Learning Outcomes. ...................................................183 7 Tuning is Discipline and Degree Specific. ...................................................................184 8 Tuner's Whiteboard Exercise. ......................................................................................185 9 Tuning is Learning-Driven Process. .............................................................................186 10 The ETE Tuning as a Complex and Complicated Process. ........................................264 11 Tuning the ETE Discipline at the State Level as Institutional Work. .........................265 12 Engagement Theory of Program Development ..........................................................268 13 Institutions...................................................................................................................270 14 The Tuning and the Bologna Process Relationships...................................................272 15 Dynamic Relationships Among Tuning Stakeholders ................................................276 LIST OF TABLES Tables 1 Engaged Leaders. ...........................................................................................................48 2 Key Points of Four Higher Education Quality Models. ..................................................49 3 The Base Model of Tuning. ...........................................................................................50 4 School Principals' Expectations About ETE Graduates' Abilities and Skills. .............187 5 ETE Faculty Understanding of Tuning. ........................................................................188 6 ETE Faculty Frequency of Discussions. .......................................................................188 7 Comparison Sample of UPTLOs With UETS. .............................................................266 8 National Qualifications Frameworks ............................................................................274 CHAPTER 1 INTRODUCTION The quality of higher education is dependent upon educators' ability to transform existing institutional models in order to meet the needs of college students. Today's political imperatives are driven by concerns about student outcomes (Introduction to LEAP, 2007; Hart Research, 2013). Accordingly, faculty should reexamine learning models and reform the institutions to present transparent college pathways for students, families, larger communities, and whole society. Research has increased our understanding of the roles and responsibilities of faculty in ensuring high quality of college degrees for students. However, the faculty's role remains underrepresented in research (Adelman, 2008b; Adelman, 2009; Bowden & Marton, 1998; Harvey & Knight, 1996; Haworth & Conrad, 1997; Tierney, 1998; Zemsky, 2009b). The lack of research exists in spite of the fact that faculty are vital framers of current higher education reform (Adelman, 2008b; Adelman, 2009). If colleges and universities genuinely intend to improve the quality of higher education through articulating measurable and assessable learning outcomes and competencies for students to be qualified for the workforce, then the faculty's role in this process must be studied and understood. The lack of current qualitative data on faculty's work at developing college discipline outcomes and competencies is a serious gap in efforts to develop systematic educational indicators of 2 college degree quality. The lack of research on faculty's work at developing student learning outcomes ignores the systemic nature of higher education. Consequently, I examined and analyzed, using qualitative methods, the faculty's purposeful and creative work aimed at developing student learning outcomes in Utah. Research Context It is critical that colleges and universities be accountable for the quality of the degrees they offer. Recently, boosting college degree production has become a common subject in higher education. However, the pressure toward helping many more students earn degrees has not been grounded in any consistent public understanding of what these degrees ought to mean. While it is obvious that more degrees are being awarded, it is not clear that these degrees are of the quality that will enable their holders to perform at the level needed by society. Quality in higher education has a range of interpretations. For instance, Harvey and Knight (1996) relate quality in higher education to the way in which the "educational experience enhances the knowledge, ability and skills of graduates" (p. 8). The Western Association of Schools and Colleges (WASC) defines quality of higher education as "a rich, coherent, and challenging educational experience, together with assurance that students consistently meet the standards of performance" (Handbook of Accreditation, 2013, p. 29). More important, employers want from their future employees a broader set of skills and higher level of learning and knowledge to meet increasingly complex demands and contribute to today's economy (Hart Research, 2013). Overall, there is a crucial need to place more emphasis on learning outcomes to increase graduates' potential to be successful and fruitful members of society. Nationally, at issue 3 is the improvement of student learning. Recently, there have been studies at the federal level in many countries (Adelman, 2008b; 2009; OECD, 1998; 1999) on how higher education systems respond to changes, both structurally and academically, as well as research on various issues of institutional response in universities worldwide (Bowden & Marton, 1998; Duke, 1992; Harvey & Knight, 1996; Haworth & Conrad, 1997; Srikanthan & Dalrymple, 2002; Tierney, 1998). At the same time, the United States of America is searching for ways to improve the quality of higher education. There is an explicit call for highly-qualified, competent graduates from American universities and colleges (Zemsky, 2009a; 2009b). State legislatures, professional associations, philanthropic organizations, and other stakeholders are increasingly concerned with the performance of American colleges and universities. The Association of American Colleges and University's Liberal Education and America's Promise (LEAP) initiated a "public debate about the quality of college learning for the twenty-first century and the learning outcomes that are essential for all students" (Introduction to LEAP, 2007). The private Lumina Foundation aims, as a part of its Big Goal, to ensure that 60% of Americans have high quality degrees by 2025 (Lumina Foundation Strategic Plan 2013-2016, n.d., p. 1). As a result of these concerns, competency-based education is currently a hot topic in higher education. Dr. Robert W. Mendenhall, the President of the Western Governors University, emphasized the fundamental premise of competency-based education in which faculty "define what students should know and be able to do, and they graduate when they have demonstrated their competency … which means they prove that they have mastered the knowledge and skills" (http://www.huffingtonpost.com/dr-robert- 4 mendenhall/competency-based-learning-_b_1855374.html). Schneider (2013) underlined that "competency frameworks also provide needed opportunities for faculty to work- together-in mapping competency expectations and related assignments across their educational programs. If students learn what they practice, then faculty should collectively ensure that their programs sufficiently emphasize the needed practices" (p. 5). The Lumina Foundation is one of the main funding vehicles for contemporary higher education. It initiated and currently co-funds "Tuning USA." This is defined as a faculty-led, discipline-level process to articulate what a student should know and be able to demonstrate in a given discipline at each degree level. Its objective is "to better establish the quality and relevance of degrees in various academic disciplines" (http://tuningusa.org/About/What_is_Tuning.aspx). Tuning is a higher education reform that focuses on learning outcomes and competencies. The Tuning methodology suggests any college discipline needs systematic and systemic analysis and improvements to meet the requirements of high quality. Colleges and universities must articulate "clear and discrete criteria for learning and thresholds of performance" (Adelman, 2009, p. 193). Faculty must learn how to collaboratively articulate criteria for student learning. Faculty-driven efforts in Tuning are critical, and their active roles in developing criterion-referenced learning outcomes and competencies are crucially important. Accordingly, my research presents a singlecase study of Utah Tuning and explores its faculty's purposive, creative and practical work aimed at identifying the discipline's learning outcomes and competencies utilizing the concept of institutional theory as a guiding analytical framework. In my research institutions are not physical organizations. Institutions were defined as "regulative, 5 normative, and cognitive structures and activities that provide stability and meaning for social behavior" (Scott, 1995, p.33). Lawrence and Suddaby (2006) in the framework of institutional theory introduced the concept of institutional work as "the purposive action of individuals and organizations aimed at creating, maintaining and disrupting institutions" (p. 215). The creation of new institutions (rules, scripts, schemas, and cultural accounts) requires institutional work on the part of a wide range of actors, both those with the resources and skills to act as entrepreneurs and those whose role is supportive or facilitative of the entrepreneur's endeavors (Leblebici, Salancik, Copay, & King, 1991). In the case of Utah Tuning the main actors are the faculty members of Utah colleges and universities and the Utah Tuning leadership team. Research Purpose The purpose of this study was using qualitative methods to examine the Tuning process for baccalaureate Elementary Teacher Education (ETE) in Utah. Specifically, I aimed to explore the role of faculty in the process of developing the student learning outcomes and competencies for Elementary Teacher Education at the discipline level. In my research, I focused on learning how those who were involved in this process were developing a student-centered, learning-centered, and competency-based Elementary Teacher Education college discipline, and how institutional theory was guiding my research process. Given the complexity of Tuning as an iterative process, the major emphasis was on how this process looked like for Elementary Teacher Education in Utah. A single-case study analysis devoted to understanding the development of studentcentered and learning-centered ETE college discipline in Utah examined this issue. 6 Research Questions This research focused on current higher education reform known as Tuning and faculty's critical role in this process (Adelman, 2008b; Adelman, 2009; Evenson, 2012; Jones, 2012; McInerney, n.d.; McKiernan & Birtwistle, 2010), and incorporated new institutional approach for analysis - institutional work (Clark, 2004; Dacin, Goodstein & Scott, 2002; DiMaggio, 1991; Lawrence, Leca & Zilber, 2013; Lawrence & Suddaby, 2006; Lawrence, Suddaby & Leca, 2009). Therefore, the following questions guided this research: 1. How have faculty been engaged in the Utah Tuning project? 2. How has the Tuning project influenced Elementary Teacher Education in Utah? 3. a. Who provides the leadership direction for tuning Elementary Teacher Education in Utah? b. What factors have been used to advance the Utah Tuning project? Significance of the Study This study is significant for several reasons. First, it focuses on how faculty tune a college discipline, that is, work together to develop student learning outcomes and competencies that Elementary Teacher Education students need to know and demonstrate to strengthen their teaching. Therefore, the study contributes to the practice of student learning and faculty professional development through expanding the faculty's knowledge on the best practices of developing student-centered, learning-centered, outcome- and competency-based disciplines. Second, this study makes a contribution to the research by focusing on faculty's interpretations of their experiences in Tuning ETE 7 college discipline, incorporating faculty's voices into the research. Finally, a thoughtful and detailed qualitative analysis of the Tuning process across the state provides more validated and authenticated evidence of the Tuning methodology and the deliverables. Conclusion The need and demand for highly-qualified, competent graduates from American universities and colleges will continue to grow. Consequently, educators must be able to transform existing institutional models in order to meet the needs of college students and the society. They must be able to learn how to work together in order to innovate and provide organizational reforms. In Chapter 2, the literature review aims to contextualize the shift in higher education to the increasing demands for the quality of higher education based on the student learning outcomes and competencies. The literature review chapter consists of three sections. First, I critically synthesize the research on the quality in higher education and examine the organizational implications for universities. I specifically examine four models: a transformative model, an engagement model, university of learning, and responsive university model. Second, I examine the literature on the current higher education reform known as Tuning, with its central message about the harmonic alignment of all educational structures and content of the programs of studies focusing on the student learning outcomes and competencies, and faculty's critical role in tuning college disciplines. Third, I review the research on organizational change in higher education, and pay special attention to institutional theory as a form of organizational analysis, and its new shift - institutional work - the individuals and organizations' purposeful work to create, maintain, and disrupt institutions. Institutional 8 theory is utilized as an analytical framework for this study to address the lack on faculty research. Chapter 3 explains the research methods utilized in the study including the rationale for the study, research design, the researcher's positionality, data collection methods, and data analysis plan, ethical and political considerations, and issues of trustworthiness of the proposed study. Chapter 4 presents the findings of this qualitative study providing the content analysis of the interviews, observations, field notes, and document data. Chapter 5 discusses the findings through institutional work analysis and provides the implications of this study for practice, policy, and future research. CHAPTER 2 LITERATURE REVIEW Introduction For the last two decades, researchers have reexamined the fundamental educational processes and proposed various new models for educational quality in universities. Among a huge amount of publications in higher education the recent literature has suggested some well-articulated comprehensive models on quality in higher education (Bowden & Marton, 1998; Harvey & Knight, 1996; Haworth & Conrad, 1997; Tierney, 1998). This chapter provides a critical review of these models in terms of their definitions of quality in higher education, specific examples, key players and activities, and benefits and criticisms. The chapter gives special attention to the current educational reform known as Tuning including its historical context and national peculiarities; tuning content specifies actors, politics, processes, and outcomes. Additionally, the chapter presents an analytical framework of the proposed study - institutional theory as a form of organizational analysis. Quality Models in Higher Education Harvey and Knight (1996) identified the five "approaches to quality" (Harvey & Knight, 1996, p. 5). First, quality as exceptional, that is, exceptionally high standards of 10 academic achievement; second, quality as perfection or consistency, which focuses on processes and their specifications and is related to zero defects and quality culture; third, quality as fitness for purpose, which judges the quality of a product or service in terms of the extent to which its stated purpose is met; fourth, quality as value for money, which assesses quality in terms of return on investment or expenditure and is related to accountability; and finally, quality as transformation, which defines quality as a process of qualitative change with emphasis on adding value to students and empowering them. They further prioritized quality as transformation: The transformative notion of quality presupposes a fundamental purpose of higher education. It assumes that higher education must concern itself with transforming the life experiences of students, by enhancing or empowering them. The transformative conception is, in effect, a meta-quality concept. Other concepts, such as perfection, high standards, fitness for purpose and value for money, are possible operationalizations of the transformative process rather than ends in themselves. (Harvey & Knight, 1996, pp. 25-26) In Haworth and Conrad's (1997) engagement theory, high-quality programs were defined as those in which students, faculty, and administrators as principal stakeholders "invested significant time and efforts in mutually supportive teaching and learning" and contributed "to enriching learning experiences for students that have positive effects on their growth and development" (p. xii). The scholars grounded the engagement theory in a substantial amount of empirical research: "the perspectives of 781 people representing 47 master's programs in 11 fields of study" (Haworth & Conrad, 1997, p. xii). Then, they grouped the stakeholders' perspectives in five separate clusters of program attributes. (Appendix A provides a visual representation of the engagement theory). The study's qualitative data are rich and impressive. Within each cluster (a total of five) each attribute (a total of 17) is presented in a matrix through actions, consequences for 11 learning experiences, and effects on students. Table 1 "Engaged Leaders" (p. 48) presents an example of one attribute, Engaged Leaders, of Cluster One Diverse and Engaged Participants. The researchers systematically identified and knitted together program attributes into a unified theory of program quality, and explained how and why specific program attributes enhanced students' learning. As a result, they proposed a framework to assist faculty, students, and administrators in learning about assessing and improving the quality of higher education programs. For instance, their template for assessing program quality included questions to guide assessment, criteria and indicators of attributes, and methods of assessment. The University of Learning: Beyond Quality and Competence in Higher Education (Bowden & Marton, 1998) strongly supported the definition of quality in higher education offered by Ball (1985) as fitness for purpose, that is, "quality can only be defined in relation to articulated values, purposes, and desired processes, experiences and outcomes" (Bowden & Marton, 1998, p. 219). Following the competency movement, they proposed to define learning "in terms of expected and achieved outcomes, rather than in terms of educational inputs" (Bowden & Marton, 1998, p. 11). William Tierney (1998), editor and author of the book The Responsive University: Restructuring for High Performance, gathered the views of a number of the leading higher education scholars on a renewal kind of higher education - the responsive university aimed to "restructure for high performance" (Tierney, 1998, p. 3). The researchers' broad model was based on the premise that higher education would "need to focus on outcomes" (Keith, 1998, p. 164) because the public "will judge the university in terms of the quality of their relationships with the university, and the quality of the 12 outcomes of those relationships" (Keith, 1998, p.163). Accordingly, "to survive and thrive, colleges and universities will have to be responsive … and service-oriented" (Keith, 1998, p.164) for students, parents, governments, businesses, nonprofit organizations, and other stakeholders. Key Points of the Models Each reviewed model has its own unique perspective on the quality of higher education. However, there emerged two principal points from these models: the shift to student learning and outcomes and dynamic relationships around it. Table 2 "Key Points of Four Higher Education Quality Models" (see p. 49) compares the four models through these key points. All of the models clearly contribute to the transformative approach to the quality of higher education emphasizing the participants' enhancement. Harvey and Knight (1996) identified this by "a range of interactions at the teaching interface" (p. 2). In Haworth and Conrad (1997), the notion of enhancement was determined to be the "growth and development" of students (p. 29), and was identified by a range of characteristics at the teaching program design and delivery levels. Bowden and Marton (1998) gave a subtle pedagogic interpretation of enhancement as learners' ability to "discern relevant aspects of variation" (p. 7). The responsive university subsumed enhancement within the notions "quality of outcomes" when they were "student centered" (Tierney, 1998, p. 163). Overall, these models claimed that the whole system of higher education had to contribute to meet the requirements of student learning and outcomes integrating the efforts of the stakeholders, including faculty, students, academic leaders, alumni, businesses, employers, and so forth, and aligning all elements of the 13 educational process. Higher education is asked to be transformative, engaging, and responsive to an ever changing environment. The four models, critically reviewed above, provide several key insights into quality assurance process in higher education by, first, describing the what of the process: focusing on student learning and outcomes, and a dynamic collaboration of faculty and administrators around this center. Second, as these scholars examined the quality of higher education, their concepts became critical points of concern in their analyses. They argued for the need for higher education reform, and described the forces and sources for changes in higher education, that is, the why of change. However, these models for higher education reform lack the explicit how of change - how the reform process could be unfolded. The scholars became advocates of their models, rather than providing detailed, informative explanations to readers on how to apply their approaches. The literature review below contextualizes the current higher education reform under the name Tuning, which explicitly explains the what, why and how of the reform process. The literature review on Tuning provides the background for the study to enable readers to understand the context utilized in my dissertation research. I specifically discuss the nature of the Tuning reform, its historical background, the stakeholders, politics, processes, and anticipated outcomes. What is Tuning? Tuning is "a program of academic reform that scholars and administrators around the nation ought to consider for its practical - and perceptive - approach to the evaluation 14 of higher education." (McInerney, n.d., p. 2). Clifford Adelman defined: Tuning is a methodology, including a consultation phase with recent graduates and employers, that produces "reference points" for faculty writing criterionreferenced statements of learning outcomes and competencies in the disciplines, providing a common language for (1) academic-subject specific knowledge, and (2) generic competencies or shared attributes. (Adelman, 2009, p. xi) Evenson (2012) characterized Tuning as "a process of quality improvement for academic disciplines" (p. 18). The Tuning approach represents a methodology which aims "to (re-) design, develop, implement and evaluate study programs" (EAC, 2010, p. 2) for subject areas in higher education. Conceptually, the content of Tuning as a learning-centered framework focuses on learning outcomes and competencies. Student learning is at the heart of the Tuning process and the Tuning dynamics that centres around it, and Tuning aims to create or redesign educational structures and content of studies. In my literature review I use the name Tuning as a generic term to speak about this higher education reform currently taking place in many parts of the world. I also use the name Tuning process to emphasize its ongoing nature, and I follow Evenson (2012) to use the name Tuning project in the sense "of an initiative to set the process in motion" (p. 19). Since competencies and learning outcomes, and other features of any discipline need to be reexamined, it is not relevant to think about a discipline as "having been Tuned" (Evenson, 2012, p. 19). As the Tuning goals are students' learning, quality assurance, transparency, and accountability, "Tuning is an on-going process, not a discrete project" (Evenson, 2012, p. 19). Additionally, the central notions of Tuning are learning outcomes and competencies. Learning outcomes are "statements created by discipline faculty, with student input, of what students are expected to know, understand, and be able to do to receive a degree in the discipline" (Tuning Glossary, 2013). 15 "Competencies represent a dynamic combination of cognitive and meta-cognitive skills, knowledge and understanding, interpersonal, intellectual and practical skill, and ethical values" (Tuning Glossary, 2013). Learning outcomes are formulated by academic staff but competencies are obtained by the student (Gonzalez, 2004). Tuning started as a two-year pilot project which intended "to tune the different educational structures in Europe and to develop professional profiles and desired learning outcomes, in terms of knowledge, skills, and competencies" (EUA Trends II, 2001, p. 53). The name ‘Tuning' was chosen for the process to reflect the idea that universities do not and should not look for uniformity in their degree programs or any sort of unified perspective or definitive European curricula but simply look for points of reference, convergence and common understanding (McKiernan and Birtwistle, 2010). It has developed into a process offering a new approach to (re)design, implement, assess, and enhance the quality of higher education cycles. Gonzalez (2004) stated that the main objectives of Tuning were intended: To identify common reference points from a discipline and university perspective; to develop professional profiles and comparable and compatible learning outcomes; to facilitate employability by promoting transparency in educational structures (easily readable and comparable degrees), and to develop a common language which is understood by all stakeholders (higher education sector, employers, and professional bodies). (p. 3) The focus of the Tuning process is on student learning, namely, what each student must know, understand, and be able to do to qualify for an associate's or bachelor's degree in the discipline. Additionally, it includes what each student must know, understand, and be able to do to qualify for a master's degree. How do the competencies and learning outcomes "ratchet up" (Adelman, 2008b, p. 113) from associate's degree to bachelor's degree to master's degree? Furthermore, Tuning considers what each student must know, 16 understand, and be able to do at the transfer point from community colleges to a bachelor's program. As a result, Tuning is "making the implicit explicit" (McKiernan & Birtwistle, 2010, p. 511) regarding the knowledge, understanding and skills required for a degree in the discipline. Overall, Tuning is a methodological faculty-led process that aims to determine subject specific learning outcomes and competences that students are expected to possess and demonstrate to earn a degree in a particular discipline. Tuning Context Historical Background Tuning has European roots. The Tuning pilot project officially started in December 2000 and was actually launched May 4, 2001 (Wagenaar, 2002). By the launch date a total of 75 colleges and universities from 15 countries were selected to participate in the Tuning pilot project and joined this initiative. The selection was done with the support of a number of the national Rectors' Conferences and the European University Association (Wagenaar, 2002). The Tuning project was a response by the main European universities to the challenge posed by the Bologna process which started a year earlier. In 1999, the ministers of 29 European countries signed the Bologna Declaration - "a set of principles for mutual recognition of educational credentials from grade school to graduate school" (Birtwistle & McKiernan, 2008, p. 317). The Bologna Declaration is an agreement to create the European Higher Education Area (EHEA) in order to build a competitive knowledge-based economy (Bologna, 2009). The Tuning project began in five selected subject areas: business administration, education, geology, history, and mathematics. From 2001 to 2002 five planned meetings 17 in these five disciplines were held. Additionally, two thematic networks, physics and chemistry, began working closely together with the project as groups six and seven making up a total of around 100 institutions (Wagenaar, 2002). The initial aim of the Tuning project was to adopt a system of easily readable and comparable degrees in all Bologna (signatory) countries. Tuning is funded by the Socrates-Erasmus education and training programs of the European Commission. Two universities - the University of Deusto (Bilbao) in Spain and the University of Groningen in the Netherlands - are responsible for the overall coordination of Tuning Educational Structures in Europe (an official name of this process in Europe). All Tuning documents can be found on its main website (http://www.unideusto.org/tuningeu/). Currently, Tuning has its recognizable logo which has been designed by a Spanish designer and is used on the Tuning websites and in all presentations. Tuning and the Bologna Process Relationships Adelman (2009) characterized the Tuning and Bologna Process relationships in this way: "the winds of Bologna changed the atmosphere for higher education reform in Europe. They came early, scattering seeds that were picked up, planted, and nurtured outside the formal proceedings. The most notable of these is the ‘Tuning project,' designed by faculty not ministers" (p. 48). Tuning was spreading "until it was unofficially embraced as a component of the Bologna agenda" (Adelman, 2009, p. 48). The Bologna Process as a higher education reform structurally has "a three-stage platform" (Gaston, 2010, p. 151) of qualifications. (Appendix C provides a visual 18 representation of the Tuning and Bologna Processes relationships). The first level - the European qualifications framework - offers an overarching structure for the degree cycles. The framework of qualifications for the European Higher Education Area describes the general categories such as depth of knowledge and understanding, the ability to manage scholarly protocols, capability in the use of data, and scholarly independence (Framework, 2005). These categories suggest "a cumulative vision of higher education from one level to the next" (Gaston, 2010, p. 152). Essentially, the European framework "offers a broad but secure scaffold on which the Bologna nations" attempt to construct "more focused and detailed national curricular structures" (Gaston, 2010, p. 152). The second level, which interrelates with the first, is the national qualifications framework, and tends to incorporate discrete national approaches to higher education accountability. The continent-wide European qualifications framework (EQF) sets " ‘outer limits' within which national frameworks should be situated," accommodates diversity "within those limits," "ensures compatibility between national frameworks," and offers for European higher education a "common face" (Bologna 2009, p. 7). In sequence, national qualifications frameworks (NQF) tend to facilitate movement "within the system," determine "what qualifications [degrees] learners will earn," and define how the different qualifications are related (Bologna, 2009, p.8). Adelman (2009) emphasized that "creating and obtaining consensus on a NQF is a time-consuming challenge" (p. xv). Every country goes its own way taking into account its national higher education needs (see Appendix D for the main features of NQFs from seven countries). The third level - the disciplinary qualifications or outcomes frameworks - 19 involves the disciplines under the name of Tuning. The first two levels focus on educational systems, which are mostly the responsibility of governments (Adelman, 2009). They represent a top-down approach. Tuning focuses on educational structures and content of studies, which are the responsibility of higher education colleges and universities (Tuning, 2004). This level assembles faculty members to produce learning outcomes, levels of learning, and desired competencies for specific disciplines. Tuning, as a bottom-up approach, has engaged members of the academic community in a broad discussion of "educational structures and content" (Tuning, 2004). The Bologna Process is comprehensive reform, which has brought into closer comparability the higher education systems of Europe. Tuning is a particular reform, connected with "the comparability of curricular in terms of structures, programs and actual teaching" (Tuning, 2004). Tuning Goes Worldwide The Tuning initiatives were being implemented in Europe from 2000 to 2006 in three phases, each with its own action lines. The action lines included: 1) generic competences and 2) subject-specific competencies (skills, knowledge and content), 3) the role of ECTS as an accumulation system, 4) the role of learning, teaching, assessment and performance in relation to quality assurance and control, and 5) the role of quality enhancement and assurance in the educational process (based on a system of an internal institutional quality culture at program level) (Gonzales, 2004). As Evenson (2012) noticed Tuning appeared "to have global appeal" (p. 19) because it offered to the world of higher education the example of its process, one of "joint reflection and debate 20 …involving thousands of colleagues, students, graduates, employers and other stakeholders" (Gonzales and Wagenaar, 2008, p. 1). At present, Tuning is being implemented in Africa, Australia, Japan, India, Latin America, Russia, and the USA. There are a total of 58 countries taking part in it with publications in more than 17 languages (http://www.unideusto.org/tuningeu/ http://www.unideusto.org/tuningeu/publications.html). The process is spreading under composite names, where the first name is Tuning and the last name can represent whole continents (Africa, Australia) or whole regions (Latin America, Central Asia), or countries (Georgia, Japan, Russia, USA). The spectrum of the Tuning names mirrors the worldwide process of reforming the higher education structures and content of studies. It includes Tuning Educational Structures in Europe, Tuning Latin America, Tuning Africa, Tuning Russia, Tuning Georgia, Tuning Central Asian Higher Education Area [TuCAHEA], and Tuning USA. In November 2013, the first issue of the Tuning Journal for Higher Education (TJHE) was published. This "biannual peer reviewed journal facilitates the collaborative efforts of hundreds of international researchers who are working to develop Tuning in global higher education" (http://www.tuningjournal.org/index.php/tuning). Its first issue focused on the impact of quality educational programs on societal developments. The journal discussed the building of new profiles and new generations of graduates in different countries, including Ireland, Spain, the Netherlands, Canada, Russia, and African countries. Tuning "grew out of the ferment of the Bologna Process and has become, worldwide, the major effort focusing on academic quality and transparency at the discipline level" (http://utahtuning.weebly.com/bologna-process-relationship-to- 21 tuning.html). In view of that, Tuning has "sufficient momentum to become the dominant global higher education model within the next two decades" (Adelman, 2009, p. viii). Tuning USA "Tuning USA" was launched by the Lumina Foundation for Education in 2009 and, since 2010 has been cosupported by the William and Flora Hewitt Foundation, emerging as a way to find a solution for the issues of the quality of higher education. The Tuning USA pilot projects initially involved three states (Indiana, Minnesota and Utah) in six disciplines, including biology, chemistry, education, history, physics, and graphic design. The project embraced a mix of two-year, four-year, public and private colleges and universities. When the initial pilot project was completed in August 2010, the states' teams issued their final reports (http://tuningusa.org/Library/ProjectReports.aspx), which have been discussed, studied, and thus have informed the evaluation work and context. The second wave of Tuning USA involved a number of other states and organizations: Kentucky, Texas, and the Midwest Higher Education Compact: Montana, Illinois, and Missouri. Texas began to tune two disciplines of civil and mechanical engineering, and then added biomedical and industrial engineering, biology and chemistry (THECB, 2011). In 2010, Kentucky started its Tuning project in five disciplines: business, biology, elementary education, nursing, and social work (http://cpe.ky.gov/committees/tuning/). Each of the states' Tuning teams produced a report in each of their discipline areas. The reports raised many serious issues, including the initiatives to strengthen and align subject area programs and align curricula within majors and across colleges and universities, the transparent transferability from one college to another, the increasing 22 demands for career-ready graduates, and so forth. Various aspects of the Tuning process and practices were critically analyzed and synthesized by the participants of the Tuning teams. For example, the Utah Tuning teams emphasized that Tuning provided the opportunity for the faculty to rethink the definitions of the disciplines, and the process reinforced the faculty ownership and responsibility for their disciplines. The project furthered the state system-wide discussion that took place through the previous different types of meetings. Tuning also provided more perspectives on international higher educational processes. The Indiana Tuning teams underlined the main strength of the process as the "collaborative opportunities" for the faculty. Specifically, "crossinstitutional and cross-sector" (two-year and four-year) meetings were very valuable and built trust between the faculty and colleges (ICHE, 2010, p. 10). All reports pointed out that the Tuning teams went from faculty's skepticism and fears to their awareness, and understanding and then to enthusiasm and appreciation of the Tuning methodology. Among the weaknesses of the project were a huge amount of information at the beginning of the process and creating time constraints, but the faculty were able to overcome these obstacles (ICHE, 2010; MNOHE, 2010; USHE, 2009). Tuning was a good match for those processes that had been taking place in the states. For example, the Indiana faculty teams saw their Tuning work as building on the 2008 Strategic Plan by the Indiana Commission for Higher Education which specifically called for "ever-improving quality and accountability, particularly in the arena of learning outcomes, and for international benchmarking" (ICHE, 2010, p. 6). Also, the Commission had called for participation in the Voluntary System of Accountability (VSA) because the VSA included an emphasis on developing student learning outcomes. 23 The Utah higher education system had had a long history of faculty discipline majors' meetings and defined general education among colleges and universities. The report (USHE, 2009) stated that "Utah established a Regents' Task Force on General Education in 1997" and "the What is an Educated Person? Conference series" (p. 7). The Tuning USA project appeared to complement the states' higher education system efforts. However, Tuning gave a new momentum for the faculty to focus on student learning outcomes and competencies, that is, what students must know and be able to do in the discipline to graduate with a college degree, and content of the programs of studies. As Adelman (2009) observed, the accomplishments of Tuning in Europe challenged the United States to "focus on what is directly taught, i.e. subject matter that reflects the training and organization of our faculties" (p. 54). The Tuning USA pilot project, in its turn, gave birth to the "concept of formulating a set of evaluation instruments and processes that could be used across the boundaries of national, state, and regional higher education systems" (EAC, 2010, p. 2). In spring 2011, the Lumina Foundation launched the EU-US study aimed to produce a methodology to evaluate the application of the Tuning approach. Representatives of the two academic communities - American and European - worked together exchanging ideas, building academic trust, investigating the respective contexts of higher education, negotiating and navigating their way through terminological and linguistic differences. The results of the study revealed a robust methodology, based on quantitative and qualitative parameters to measure the effects of applying Tuning to degree programs, teaching staff, students, and graduates (EAC, 2010). Started as a project, Tuning has developed into a process offering a new approach for (re)designing, implementing, assessing, and enhancing the quality of higher 24 educational structures and programs on the basis of diversity and autonomy. Tuning Content Tuning Actors The Tuning process engages many stakeholders, including faculty, students, alumni, advisors, employers, college and university administrators, K-12 teachers who prepare students for college, librarians, policy makers, and any others involved in this process (Adelman, 2009; Evenson, 2012; Holliday, 2011; McInerney, n.d.). However, Tuning clearly recognizes the faculty as "the experts on what students need to know, understand, and be able to do to develop professionally in the discipline" (Evenson, 2012, p. 18). In Utah, the Tuning faculty teams (Physics, History, General Education Mathematics, and Elementary Teacher Education) involved representatives from all colleges and universities: two research universities, four comprehensive state or regional universities, two community colleges, and private institutions. The team work was facilitated by the Office of the Commissioner of Higher Education under the leadership of Assistant Commissioner for Academic Affairs, Dr. Phyllis Safman (Evenson, 2012; USHE, 2009). Other stakeholders also play their essential roles in the process. Students are very important members of any Tuning teams. Ideally, every college and university should bring its students "to accomplish the learning outcomes in its own way" (Evenson, 2012, p. 19) as in the example of Utah Tuning Physics team (Evenson, 2012; USHE, 2009). Students are expected to express what they as learners of the discipline experience in their studies. Their role is to help and support the faculty in seeing the real picture of 25 what actually happens in classrooms. Additionally, Holliday (2011) emphasized librarians' "strong role" to support faculty "to think about and articulate information literacy learning goals and activities in their own language and within their own experience" (p. 197). Norm Jones, Chair of History from Utah State University, "found that librarians working with particular courses could greatly improve student mastery of research skills" (Jones, 2012, p. 14). In Utah State University, librarians and historians together built web portals for research projects, and the librarians taught the history major students how to use the tools in them (Jones, 2012). Additionally, alumni, employers, and professional organizations help the Tuning teams to map the employability of their graduates. The faculty work with the field stakeholders or "disciplinary communities" in terms of Jones (Jones, 2012, p. 3) through consultations, surveys, focus groups, personal interactions, and other communication channels. The communications go two ways within the faculty and across the stakeholders (Appendix E provides a visual representation of stakeholders' communication). The purpose of the consultations, surveys, focus groups is not to have stakeholders define a discipline for the faculty "but to understand more clearly what those outside the academy expect and value in higher education" (McInerney, n.d., p. 4). For instance, Evenson (2012) accentuated "focus groups with students and employers and group or individual discussions with faculty colleagues not on the team were very productive" (p. 22) for the Utah Tuning Physics team. He stressed that consultations with alumni, employers, colleagues did not "dictate the programs defined by the faculty" (Evenson, 2012, p. 22). On the contrary, they provided rich insight and kept "the team grounded in the realities of their larger context" 26 (p. 22). The faculty's interactions with the stakeholders gave "snapshots in time, emphasizing the need to work through from competencies to learning outcomes, consultations, employment maps, and degree profiles repeatedly in an ongoing process of strengthening and updating the program" (Evenson, 2012, p. 22). Special attention is given to the faculty's communications during the Tuning meetings. Faculty get together to develop clear common reference points in a discipline. They are asked "to explain the fundamental questions that inform all of the work in their field: what must students in their discipline know, understand, and be able to do in order to receive a degree?" (McInerney, n.d., p. 4). All Tuning teams reported that their teams' work began with in-depth discussion of their disciplines (ICHE, 2010; MNOHE, 2010; USHE, 2009). Evenson (2012) also described the process: Several sessions of discussions were required before teams took ownership of the process. They needed to understand the process and how their work related to prior efforts to define learning outcomes and establish requirements. They needed to understand that Tuning is not standardization! They needed to understand that administrators who may have facilitated the establishment of the Tuning teams did not have preconceived outcomes, but that the outcomes of this work were the responsibility of the faculty/student teams themselves. Once the teams reached that understanding of the process, they agreed rather quickly on common sets of general and discipline-specific competencies that are central to the discipline. (p. 20) Additionally, they build up a list of appropriate generic competencies through consultations with all stakeholders in the field. Along the way the faculty gain a sense of the scope to which students can achieve competencies in their studies and formulate competencies specific to the discipline (Evenson, 2012; Gaston, 2010; Holliday, 2011; Jones, 2012; McInerney, n.d.). The minutes of the Utah Elementary Teacher Education Tuning team meetings registered the complexity of the process. Through reviewing and refining multiple drafts of learning outcomes the faculty had "a big ‘ah ha' moment … 27 that many of the outcomes … were not assessable … urged us to think about wording … that would allow us to assess them earlier in the program" (USHE, 2012, p. 90). Consequently, in tuning a discipline, "faculty develop a shared language for competencies and learning outcomes, making degree expectations transparent" (Evenson, 2012, p. 19). As a result, the transparency is extended to students and ultimately to all higher education stakeholders. Politics of Tuning Norm Jones, Chair of History and director of General Education at Utah State University, analyzed three considerations that made Tuning unique. They were "visions," "trust," and "mechanics" (Jones, 2012, p. 1). By "visions" he meant "the right of the disciplinary faculty to enact their professional values" (Jones, 2012, p. 9). Under "trust" he accentuated that the deliberations about learning outcomes, their formulation and articulation "must be valued and supported by academic leadership" … "The faculty must be heard, and their vision must be allowed" (Jones, 2012, p. 7). The politics of Tuning is to identify "the people and organizations that have the power to convince the faculty that they will not be harmed if they articulate their values and base their assessments upon them" (Jones, 2012, p. 7). Even though Tuning is a faculty-driven process, there is a great need for administration support during its development and implementation stages on different levels. Tuning at its development stage requires an in-depth open and honest conversation about learning outcomes that includes many colleges and universities across the discipline. Jones (2012) underlined the importance for colleges and universities to 28 "stand together," in this case "their degrees articulate better, and their professional outcomes are easier to impose" (p. 10). At the implementation stage there is the danger for the faculty not to be heard, and academic leaders and administrators can "reject the implementation of professional outcomes if they cost money, or if they introduce complications in the larger curriculum" (p. 11). Jones's (2012) third consideration is the "mechanics" of Tuning: "the processes of implementation" (p. 12) within colleges and universities. A discipline has to think about what it can achieve within the structures of its college or a university, namely, the faculty and upper administration "must be very aware of how a discipline nestles within the larger curriculum" (Jones, 2012, p. 12). Tuning a discipline has implications across the curriculum. In the best scenario, Tuning methodology should be applied to whole colleges. For example, the College of Humanities and Social Sciences at Utah State University is designing a college curriculum using Tuning methodology. The aim is to ensure that students "have common literacy and skills, functioning as a springboard into the majors" and "the majors get the benefit of knowing what their students should know, understand, and be able to do and can plan the curriculum accordingly" (Jones, 2012, p. 18). The researchers and practitioners (Kolb, Kalina, & Chapman, 2013) presented the Tuning project in Texas. The faculty and state leaders "viewed Tuning as a way to create a transferrable engineering curriculum" (Kolb et al., 2013, p. 61) and worked together to produce it. The work was led by the Texas Higher Education Coordinating Board. The Tuning process in Texas identified the importance of academic leaders meeting with key stakeholders prior to the faculty Tuning work. The interactions between the academic 29 leadership and employers "created time and political space on campus for faculty to execute their work" (ibid). Moreover, this process emphasized "the need for champions, internal to the key institutions, in order to drive the work and ensure fidelity and success" (Kolb et al., 2013, p. 62). The object of Tuning is student learning. Consequently, the researchers (Adelman, 2008b, 2009; Birtwistle & McKiernan, 2008; Evenson, 2012; Gaston, 2010; McInerney, n.d.) stressed a huge need to shift the focus of faculty members, departments, colleges and universities, professional associations, and accrediting organizations from what is taught to what students learn. Accordingly, the shift from teaching to student learning outcomes demands culture change in academic departments. This, in turn, needs and demands rethinking of existing leadership, which could empower faculty members, and could be done only through trust from local administration and "political support from the upper administration" (Jones, 2012, p. 21). "A discipline standing together and blessed by the system in which it is embedded is powerful, but professional organizations, professional accreditors, and regional accreditors must be recognized as having authority over the future of the outcomes" (Jones, 2012, p. 11). The whole system of higher education must express its support for the faculty's efforts tuning their disciplines. Tuning Processes In 2012, the Institute for Evidence-Based Change (IEBC) presented the report Tuning American Higher Education: The Process. According to the report, the base model of Tuning "consists of five discrete processes, each with its own constituent 30 elements" (IEBC, 2012, p. 5). Table 3 "The Base Model of Tuning" (p. 50) demonstrates the main processes and basic elements of Tuning. The base model serves: Like an interstate road map, but it is not a road. It is a highway with exits that faculty might take as they return to their campuses to try out ideas with colleagues or in classrooms to continually inform the work of the Tuning group. (IEBC, 2012, p. 6) The exact configuration of these processes can vary depending on the faculty's needs engaged in Tuning. Additionally, IEBC (2012) offered four variations of the base model. The faculty work groups can begin Tuning the disciplines with gathering data, or consulting stakeholders, or multiple consultations and revisions. Moreover, the process can revise "the core discipline multiple times as a result of both consultation and trial within departments" (IEBC, 2012, p. 34). Overall, the flexible Tuning methodology brings the potential for periods of searching for approaches that can best advance the faculty's work. Discipline Specific Tuning is performed at the discipline level. Holliday (2011) emphasized that "the Tuning methodology requires the active participation of faculty in the disciplines" (p. 192). Faculty meet "by discipline to work through the Tuning process" (Kolb, Kalina, & Chapman, p. 58), where "faculty specialists are asked to explain the fundamental questions that inform all of the work in their field" (McInerney, n.d., p. 4). Evenson (2012) described how discipline Tuning started in Utah: The work of the Tuning teams began with in depth discussion of their discipline by the teams of faculty and students: How do we define what it is that students need to qualify for a degree in the central discipline? What competencies are essential that are taught in other departments (general competencies)? What discipline-specific competencies are essential? (p. 20) 31 From the very beginning, the tuners were differentiating between discipline specific competencies and general competencies so that the Tuning could focus on the discipline specific. State Tuning reports (ICHE, 2010; USHE, 2013; 2014) connected with elementary education accentuated numerous standards and requirements that guide elementary education, and, consequently, college and university programs have to thoroughly consider them in their teaching and include them in their Tuning process. Nationwide, the Elementary Education has to follow the Interstate Teacher Assessment and Support Consortium (InTASC) Model Core Teaching Standards. In 2011, the Council of Chief State School Officers (CCSSO), a national nonpartisan organization (http://www.ccsso), through its Interstate Teacher Assessment and Support Consortium, offered a set of Model Core Teaching Standards. This Model outlined "what teachers should know and be able to do to ensure every K-12 student reaches the goal of being ready to enter college or the workforce in today's world" (CCSSO, 2011, p. 3). The InTASC Model Core Teaching Standards articulated what effective teaching and learning looks like in a transformed public education system - one that empowers every learner to take ownership of their learning, that emphasizes the learning of content and application of knowledge and skill to real world problems, that values the differences each learner brings to the learning experience, and that leverages rapidly changing learning environments by recognizing the possibilities they bring to maximize learning and engage learners. A transformed public education system requires a new vision of teaching. (CCSSO, 2011, p. 3). Like most vision statements, this contains lofty language that can provide inspiration, but must be made substantial, practical, and workable by use 32 of supporting goals and standards in order to be useful in program design. Statewide, all Utah teacher education programs have to follow the Utah Effective Teaching Standards (http://www.schools.utah.gov/CURR/educatoreffectiveness/ Standards/Teaching/TeachingStandards.aspx; http://www.schools.utah.gov/CURR/ educatoreffectiveness/Observation-Tools/BecomingFamiliar.aspx), which are "a general set of standards for practicing teachers" (USHE, 2014, p. 3). The Utah Effective Teaching Standards (UETS) were "adapted from" the InTASC Model Core Teaching Standards. All Utah teacher education programs had to follow the requirements and standards for accrediting and professional associations including Teacher Education Accreditation Council (TEAC) (http://www.teac.org/), National Council for Accreditation of Teacher Education (NCATE) (http://ncate.org/), and Council for the Accreditation of Educator Preparation (CAEP) (http://caepnet.org/). On July 1, 2013 the Council for the Accreditation of Educator Preparation became "fully operational as sole accrediting body for educator preparation providers" (http://caepnet.org/) by consolidating the National Council for the Accreditation of Teacher Education (NCATE) and Teacher Education Accreditation Council (TEAC). On August 29, 2013 "the CAEP Board of Directors approved new accreditation standards" (http://caepnet.org/about/ history). The CAEP set standards for elementary teacher preparation. Overall, this array of standards and requirements informs the faculty as they design the discipline and the course work that defines what students are expected to learn. But defining course work implies concentration on what is taught, whereas: Tuning brings faculty together to build clear common reference points in a discipline … Neither curriculum nor pedagogy nor assessment is prescribed by Tuning, only outcomes. I think of Tuning as learning to sing in the same key but not in unison, discipline by discipline. (Evenson, 2012, p. 19) 33 This sets Tuning apart from activities, such as curriculum development, that faculty have traditionally engaged in when designing a program. Tuning occurs at the discipline level and requires specificity, not generalities. While outcomes are not foreign to faculty thinking, they have not, prior to Tuning, been the exclusive focus of discipline specific processes. Tuning Anticipated Outcomes Tuning explicitly identifies what to reform in higher education, who does this reform, and how to reform the current higher education system. Tuning is a studentcentered approach to education. More specifically, the "what to reform" theme focuses on transparent academic and profesional profiles in study programs with emphasis on generic and subject specific student learning outcomes and competencies (Adelman, 2008a, 2008b, 2009). Additionally, the Tuning methodology and models are appropriate for mono-disciplinary, inter- and multidisciplinary, integrated and joint degree programs (Gonzalez, 2004). Organizationally, if asked how to tune the programs and the content of studies, the answer would be, together (Jones, 2012; Schneider, 2013). Tuning "enables faculty to better establish the quality and relevance of degrees in various academic disciplines and professional fields" (IEBC, 2012, p. 3). In the framework of the "who reforms" theme, the scholars (Evenson, 2012; Jones, 2012; Kolb et al., 2013; McInerney, n.d.) also identified benefits for the Tuning participants and broader public. For the faculty, Tuning provides professional development, space for innovation, valuable "discussions about student learning across institutions and sectors," establishes "meaningful relationships among faculty members across" colleges and 34 universities, and "a defense against accountability imposed from outside the institution" (McInerney, n.d., p. 6). For students and their parents, Tuning offers a clear expectation for college readiness, clear path through their college degree, simplifies the credit transfer process, and states what students "can expect to gain in terms of personal development, knowledge, skills, and competencies" (McInerney, n.d., p. 5). Dr. Julia Gonzalez, a codirector of the European Tuning Project, has found that Tuning has considerably changed approaches in teaching, learning and in assessment (Gonzalez, 2004). Overall, Tuning "offers a means of strengthening American higher education" (IEBC, 2012, p. 3). McInerney (n.d.) recognized Tuning's overarching trend: Tuning contributes to a profound shift in the informing culture of higher education. Our attention focuses more on learning, on a student-centered academic environment, on the "outcomes" of higher education (more than academic "inputs"), and on continuous and reform-minded evaluations of our work (rather than sporadic and ineffective self-studies). Tuning reflects on what we do and projects new paths for higher education to follow. (p. 6) Educational reforms need organizational changes. Therefore, a critical analysis of research literature on the processes of organizational change at the institutional level is needed. The next section of my literature review is devoted to the research from organizational change/reform in higher education. Organizational Change A modern university is a big, complex, demanding, competitive, bureaucratic, and multiversity organization (Altbach, 2005). Universities and colleges struggle to respond with "re" strategies: "restructuring, reducing, reallocating, and refocusing" (Eckel, 2000, p. 15). Kezar (2001) made a reasonable assumption that in order to develop a distinctive model of organizational change in higher education, one must take into account the 35 unique features of higher education. Her list included: interdependent organization, relatively independent of environment, unique culture of the academy, institutional status, values-driven, multiple power and authority structures, loosely coupled system, organized anarchical decisionmaking, professional and administrative values, shared governance, employee commitment and tenure, goal ambiguity, and image and success. (Kezar, 2001, p. vi) Bolman and Deal's (2003) multidimensional model of organizational change or a four-cornered frame (structural, human resource, political, and symbolic) sought to explain organizational behavior across institutional types, and in various institutional activities. The scholars focused on the role of managers and leaders in implementing reframing strategies and techniques in their organizations, that is, examining the same situation from multiple vantage points (loosely coupled vs. tightly coupled) to develop a holistic picture. Successful change requires an ability to frame issues, build coalitions, and establish arenas, which provide rules, referees, and spectators. Ritual is also essential in change, which helps minimize conflicts. Overall, restructuring, recruiting, and retraining of those who are involved in the change processes are simultaneous actions to effective reframing. Kezar (2001) conceptualized recent research on organizational change in higher education. Her assumption was that institutional change could be better facilitated by a deep understanding of the process of change from multiple perspectives. Based on the extensive research on organizational change from several disciplines, Kezar developed the typology of major change theories: evolutionary, lifecycle, teleological, dialectical/political, social-cognition, and cultural models. She then applied this typology to the literature on change in the field of higher education. Kezar (2001) claimed that organizational change could "best be explained through political, social-cognition, and 36 cultural models" (p. vii). Cultural models (symbolism, history and traditions, and institutional culture), political processes (persuasion, informal negotiation, mediation, and coalition-building), and social-cognition models (altering mental models, learning, constructed interaction) appeared to be very important and powerful strategies for creating change. Additionally, she presented a complex set of research-based principles (totaling 17) which emerged from her extensive review of the research. Among others, these principles included: "institutional culture affecting change," "opportunities for interaction to develop new mental models," "shared governance and collective decisionmaking," "helping people in changing belief systems," "strategies for change vary by change initiative," "combining models or approaches" (Kezar, 2001, p. 113). Research on organizational change is closely connected with studies on organizational learning (Argote, 2011). Argyris and Shon (1996) studied organizational learning at depth, and stated that the organization learns through culture change as the central process, and organization itself could be a learning subject. Furthermore, research on organizational change has widely investigated participatory learning. Research literature (Barr, 1996; Boreham & Morgan, 2004; Horn, 2005; Lave & Wenger, 1991; Wenger, 2009) used participatory learning as a descriptive term that encompasses a broad range of learning systems wherein the learner is an active participant in the process rather than a recipient of direct instruction. Participatory learning, communities of practice, and social learning systems are related and partially overlapping terms that also describe group learning. Participatory learning occurs in structured environments, and often among peers. It occurs in both the private and public sectors through communities of practice and especially in "education communities of practice are increasingly used for professional 37 development, but they also offer a fresh perspective on learning and education more generally" (Wenger, 2009, p. 7). Communities of practice are also often viewed as learning partnerships, which create high learning potential even though "a learning partner is not someone who agrees with you or disagrees with you or even shares your background necessarily" (ibid). Broad participation is often characteristic of communities of practice, and a fully functional community could include members whose experience ranges from novice to "old timer" and the full range of intermediate levels of experience. According to Wenger (2009), the function and success of communities of practice as learning partnerships is firmly rooted in "a mutual recognition as potential learning partners" (p. 12). The researcher (Wenger, 2009) further presented learning partnerships as consisting of four disciplines, such as the discipline of domain, the discipline of community, the discipline of practice, and the discipline of convening. The discipline of domain addresses the purpose of the community of practice or learning partnership. The key questions that should be asked here: "What is our partnership about? Why should we care? Are we likely to be useful to each other? What is our learning agenda? What specific set of issues does it entail?" (Wenger, 2009, p. 12). Boreham and Morgan (2004) emphasized that in order "for an organization to be able to learn as an organization [emphasis in original], there must be a common object of its collective activity, without which it would cease to be the kind of unitary entity that could be identified as a learning subject" (p. 321). The discipline of community addresses the composition of a community of practice, namely, the issue of who should be in the community, and who should not, in 38 order to maximize the potential for success. Communities of practice are built on many factors including expertise of participants, representation of interest groups or individuals, and group dynamics such as trust and reliability. The key questions that should be asked here: "Who should be at the table so the partnership can make progress? What effects will their participation have on the trust and dynamics of the group? How do we manage the boundaries of the community?" (Wenger, 2009, p. 12). For examples of community composition, we can look at Horn (2005) who described two efforts at reform of the teaching of mathematics. In both cases, the principal participants in the projects were exclusively classroom teachers. Excluded from the community were students, administrators and parents, groups who were often included in school oriented activities. Also excluded were teachers from other subject areas. In both studies, the communities of practice were deliberately selected to include specific, desired expertise, and exclude others. The discipline of practice addresses how the community of practice could learn together. It addresses the "how" and "what to do" of group interactions. It is the actual dynamics of group interaction necessary for the participants to learn and benefit from the community of practice. The key questions that should be asked here: " How can the practice become the curriculum? How can the practice be made visible and inspectable? What should participants do together to learn and benefit from the partnership?" (Wenger, 2009, p. 12). When successfully implemented, the discipline of practice lead to trust among the community that the members "make contributions that are a very likely to be relevant to practice. It is trust in the learning capability of a partnership" (Wenger, 2009, p. 12). The discipline of convening covers much of the mechanics and support 39 structure for a community of practice. It addresses such issues as providing a learning space for the community and resources available to the community. It also addresses defining roles and interaction with external stakeholders. The key questions that should be asked here: Who will take leadership in holding a social learning space for this partnership? How can we make sure that the partnership sustains a productive inquiry? Who are the external stakeholders, and what are their roles? What resources are available to support the process? (Wenger, 2009, p. 12) These things are vital to initiating and sustaining a productive partnership. One of the most important features of how communities of practice actually learn is talking, debating, dialogue, or conversation (Bakhtin, 1981; Senge, 1994). Horn (2005) emphasized that "teachers learn about classroom practice by the ways it is (and is not) rendered in collegial conversations" (p. 225). According to Boreham and Morgan (2004), "the collective capacity to enact dialogical transactions appropriately" (p. 315) is critical to collective learning. Companion to dialogue is the need for common understanding of terms. Lave and Wenger (1991) in their work on situated learning, made the point that language was an important part of practice that could not be overlooked. When observing teachers in the midst of long-term communities of practice, Horn (2005) found the conversations impenetrable because of a lack of understanding of local language and shorthand language developed by the community to expedite conversation and convey complex notions in short phrases. A common understanding of terms is critical to the learning partnership. This does not mean a simple dictionary definition level of understanding, but a deeper, nuanced understanding of meaning in the context of the community of practice's subject. Terms and their meaning become a social activity and not a matter of individual interpretation (Lave & Wenger, 1991). 40 Learning, in communities of practice, involved both organizational learning and individual learning (Boreham & Morgan, 2004; Brown & Duguid, 1991; Horn, 2005; Lave& Wenger, 1991), and these two types of learning were mutually supportive. Boreham and Morgan (2004) stated, "concept of learning implies the simultaneous transformation of social practices and the individuals who participate in them, and thus, the social and individual dimensions of learning are mutually constitutive" (p. 309). Brown and Duguid (1991) reassessed learning, work, and innovation "in the context of actual communities and actual practices" (p. 40) and suggested the composite concept of learning-in-working that viewed "learning as the bridge" between work and innovative change. Research on organizational change has also paid considerable attention to institutionalization and its core - institutional theory. Analytical Framework Institutional Theory Institutional theory, a form of organizational analysis, focuses on the relationships between organizations and the fields in which these organizations function. More specifically, institutional theory emphasizes the role of "rational formal structures in enabling and constraining organizational behavior" (Lawrence & Suddaby, 2006, p. 215). Scott (1995) comprehensively defined institutions as: Cognitive, normative, and regulative structures and activities that provide stability and meaning for social behavior. Institutions are transported by various carriers - cultures, structures, and routines - and they operate at multiple levels of jurisdiction. In this conceptualization, institutions are multifaceted systems incorporating symbolic systems - cognitive constructions and normative rules - and regulative processes carried out through and shaping social behavior. Meaning systems, monitoring processes, and actions are interwoven. (p. 33-34) 41 (Appendix B provides a visual representation of this definition). Jepperson (1991) defined institutions as "an organized, established procedure" that reflects a set of "standardized interaction sequences" (p. 143-145). In his view, institutions (rules, schemas, shared meanings) were a product of specific purposive actions taken to reproduce, alter, and destroy them. Fligstein (2001) also emphasized regulation and human cognition in defining institutions as "rules and shared meanings ... that define social relationships, help define who occupies what position in those relationships and guide interaction by giving actors cognitive frames or sets of meanings to interpret the behavior of others" (p. 108). Background The key milestones of institutional studies are connected with organizational sociology (Selznik, 1957), old institutionalism during the 1970s and 1980s (DiMaggio & Powell, 1983; Meyer & Rowan, 1977), and neoinstitutionalism in 1990s (Greenwood & Hinings, 1996). Traditionally, institutional theory studied the processes through which institutions ruled actions, and the emphasis was on the explanation of organizational similarity based on institutional conditions - institutional isomorphism (DiMaggio & Powell, 1983; Greenwood & Hinings, 1996). Theoretical institutional studies of Meyer and Rowan (1977), DiMaggio and Powell (1983, 1988), and Greenwood and Hinings (1996) conceptually outlined the key notions and relationships, and connected institutional structures and logics to organizational forms and behavior. Additionally, many empirical institutional studies have been conducted: Keohane (1989), Leblebici, Salancik, Copay, and King (1991), Meyer, Boli, Thomas, and Ramirez (1997), Rorrer 42 (2001), Zilber (2002). They practically committed to an institutional understanding of organizational actions, and documented the connections between institutions, fields, and organizations in deferent spheres, locations, and levels. Moreover, the reviews by Powell and DiMaggio (1991), Tolbert and Zucker (1996), Scott (1995; 2001), and Schneiberg and Clemens (2006) synthesized and summarized the major studies on institutionalism in organizations and coherently presented institutional frameworks. At the center of all institutional approaches to organizational research is the concept of an institution: "enduring elements in social life - institutions - have a profound effect on the thoughts, feelings and behavior of individual and collective actors" (Lawrence & Suddaby, 2006, p. 216). Another central concept of institutional theory is diffusion (Greenwood, Suddaby, & Hinings, 2002; Tolbert & Zucker 1996; Zucker 1988). Diffusion happens when individuals, being exposed to some innovations, accept them, and then adopt and implement them. Many researchers (Baron, Dobbin, & Jennings, 1986; Davis 1991; Haveman 1993; Hinings & Greenwood 1988; Tolbert & Zucker 1983) examined the diffusion of some organizational structure or practice, and attempted to explain the factors that led organizations to take on that structure or practice. Institutional scholars (Czarniawska & Sevon, 1996; Greenwood & Hinings, 1996) acknowledged old institutionalism which focused more on external factors of legitimacy and new institutionalism with more focus on interpretation, adoption, and rejection by the individual organization of change ideas. 43 New Emphasis: Institutional Work Over the past two decades growing awareness of institutional scholars about institutions as a product of human actions, "motivated by both idiosyncratic personal interests and agendas for institutional change or preservation" (Lawrence, Suddaby, & Leca, 2009, p. 6) has shifted from a traditional emphasis of research on isomorphism of organizational structures to a new emphasis on examining the role of actors towards institutions. This new emphasis got the name of institutional work in Lawrence and Suddaby (2006). These scholars defined institutional work as "the purposive action of individuals and organizations aimed at creating, maintaining and disrupting institutions" (Lawrence & Suddaby, 2006, p. 215). The definition analysis shows that institutional work conceptually emphasizes three key elements. First, institutional work centers on actors' actions as the core of institutional dynamics. Second, it represents institutional actors as goal-oriented, skillful, capable, and reflexive. Structure, agency, and their interrelations are implicit (Battilana, Leca, & Boxenbaum, 2009). Third, institutional work focuses on micropractices: three categories of institutional work - creating, maintaining, and disrupting institutions. Institutions are understood as "constituted in the more or less conscious action of individual and collective actors" (Lawrence & Suddaby, 2006, p. 219). The concept of institutional work stemmed from institutional theory (DiMaggio, 1988; 1991; DiMaggio & Powell, 1983; Jepperson, 1991; Oliver, 1991; 1992), and reflected a shift in focus on the role of actors in institutionalization (Dacin, Goodstein, & Scott, 2002; Dacin, Munir, & Tracey, 2010). Additionally, deterministic effects of structures shifted to actors' power and agency in manipulating and even transforming the 44 institutional order (Battilana & D'Aunno, 2009; Greenwood, Oliver, Suddaby, & Sahlin, 2008). Moreover, large scale macro-level inquiries, which concentrated on structures and processes, shifted to an interest in microlevel ideational dynamics (Zilber, 2008). Institutional Work: Foci Institutional work is based on a bottom-up approach (Zilber, 2013), the theorization of institutional work is still developing. In general, current research in institutional work with an emphasis on the role of actors focuses on three issues: how institutional work occurs, who does institutional work, and what constitutes institutional work (Lawrence, Leca, & Zilber, 2013). The first issue - how institutional work occurs - is examined in terms of three practices: creating, maintaining and disrupting institutions. Lawrence and Suddaby's (2006) concept of institutional work followed in the practice tradition. Schatzki, Knorr, Cetina, and Von Savigny (2001) defined practices as "embodied, materially mediated arrays of human activity centrally organized around shared practical understanding" (p. 2). Brown and Duguid (2000) contrasted practice theory and research with process-oriented studies: Practice focuses on the ‘internal life of process' … a process-oriented theory articulates a sequence of events that leads to some outcome, a practice theory describes the intelligent activities of individuals and organizations who are working to effect those events and achieve that outcome. (p. 95) Essentially, the practice perspective in institutional theory locates the concept of a "field" as central to all things social. Summarizing this issue, Schatzki et al., (2001) argued that "practice approaches promulgate a distinct social ontology: the social is a field of embodied, materially interwoven practices centrally organized around shared practical understandings" (p. 3). For example, Greenwood and Hinings (1996), and Tolbert and 45 Zucker (1996) focused on procedural descriptions of institutionalization: what happened to institutions; how they were transformed; what states they took on and in what order. In contrast, a practice orientation in Brown and Duguid (2000), and Whittington (2003) focused on the world inside the processes: the work of actors as they attempted to shape those processes, as they worked to create, maintain, and disrupt institutions. Lawrence and Suddaby (2006) analyzed and synthesized the practices of institutional work into three categories, and provided a list of practices used to create, maintain, and disrupt institutions. The second issue - who engages in institutional work - presents professionals, leaders, and actors without the expertise of professionals (Lawrence et al., 2013). The creation of new institutions (rules, scripts, schemas, and processes) requires institutional work on the part of a wide range of actors, both those with the resources and skills to act as entrepreneurs and those whose role is supportive or facilitative of the entrepreneur's endeavors (Leblebici et al., 1991). The scholars mostly examine the relationship between institutional work and professions (Empson, Cleaver, & Allen, 2013; Singh & Jayanti, 2013; Suddaby & Viale, 2011). Research on leaders in institutional work stems from Selznick's (1957) classical Leadership in Administration: a theory of the institution with a central role of an institutional worker - the leader or "statesman." For example, Rojas (2010) explored how a college president engaged in institutional work that reshaped the organization's structure and norms and provided him with "extensive powers" (p. 1264). Dorado's (2013) case study showed how group dynamics of actors without professional expertise motivated, inspired and enabled individuals to engage in institutional work. The third issue - what constitutes institutional work - tends to focus on the 46 relationship of institutional work to agency: the agent's capacity to act in a world, the agent's intentional actions (Bandura, 2001). Battilana and D'Aunno (2009) examined agency and institutional work relationships in detail. They drew on a relational, multidimensional understanding of agency that included "habit, imagination and practical evaluation" (p. 47), and argued that "intentions … will vary considerably depending on the dimension of agency that dominates the instances of institutional work one considers" (Battilana & D'Aunno, 2009, p. 49). The relationship between agency and institutional work was also explored by Zundel, Holt, and Cornelissen (2012) who argued that the study of institutional work faced a "double bind", that is, "agents' activities and thoughts are observed and acknowledged in conceptual juxtaposition to their institutional context," but "the closer any study gets to the ‘rough ground' of the phenomena, the less apparent [the juxtaposition] becomes: just where does the individual stop and institution start?" (pp. 102-103). The reproduction and continuation of institutions cannot be taken for granted; even the most highly institutionalized technologies, structures, practices and rules require the active involvement of individuals and organizations in order to maintain them over time (Lawrence et al., 2001). On the one hand, institutions (rules, scripts, schemas, and shared meanings) affect organizational actions. On the other hand, what is the effect of individual and collective actors on institutions? How do actors accomplish the social construction of rules, scripts, schemas, and shared meanings? I believe the Tuning process is a unique exploration site for this purpose. Tuning as a methodological process is completely based on how faculty change old and create new structures, rules, schemas, shared meanings. Consequently, institutional theory is employed as an analytical lens to 47 research Tuning. In the case of Utah Tuning for Elementary Teacher Education the main actors of the institutional work are the faculty members of Utah colleges and universities and the Utah Tuning leadership team. Additionally, there are other actors who are involved in this process, such as, ETE department faculty members, department chairs, the Utah State Office of Education. Institutional theory permitted me to explore the purposive, creative and knowledgeable work of the faculty members and its effect on the learning outcomes and competencies of Elementary Teacher Education. 48 Table 1. Engaged Leaders. (Source: Haworth & Conrad, 1997, p. 32) Engaged Actions Leaders Consequences for Effects on Students Learning Experiences Faculty and Leaders effectively Leaders who administrators recruit promote their provide students department or program program to internal with program chairs who will and external leadership "champion" their audiences and are opportunities help program adept at securing them become more resources to sustain highly-skilled, self- it. confident leaders. 49 Table 2. Key Points of Four Higher Education Quality Models. Principal points of the models The transformative model (Harvey & Knight, 1996) The engagement model (Haworth & Conrad, 1997) The university of learning model (Bowden & Marton, 1998) The responsive university model (Tierney, 1998) Focus on student learning and outcomes Quality policies must result in a clear focus on student experience The clusters of program attributes contribute to enriching the learning experiences for students Quality in the university context relates strongly to quality of learning Studentcentered in programs, communitycentered in outreach and nation-centered in research Dynamic relationships/ collaboration in the education delivery Students' learning experience must be based on a dialogue between the learners and teachers about the nature, scope and style of their learning, and among the teachers about the teaching and learning process Learning and teaching must be based on critical dialogue, mentoring and cooperative peer learning Faculty should be involved in a course/research team and develop a holistic view of student competencies and a collective consciousness of what is common and what is complementary New internal and external relationships must be created across and within faculty and administrators, and universities through communication and partnerships 50 Table 3. The Base Model of Tuning. 1 Processes Constituent elements Defining the Draft general degree profile discipline core Identify core concepts Draft competency statements Draft measurable student learning outcomes Mapping career Research student career destinations pathways Develop career pathways map Consulting Identify stakeholders stakeholders Draft survey instruments or focus group protocols Gather stakeholder input Honing core Review stakeholder feedback competencies and Review discipline core in light of feedback learning outcomes Implementing Identify departmental assets/priorities/missions results locally & Emphasize departmental distinctiveness writing degree Write degree specifications for each degree level specifications 1 Note. Reprinted from Tuning American higher education: The process, by the Institute for EvidenceBased Change, 2012, p. 6. Copyright 2012 by the Institute for Evidence-Based Change. Reprinted with permission. CHAPTER 3 RESEARCH DESIGN AND METHODS This chapter presents the research design, specific procedures, and analytical framework utilized for conducting this study. As stated earlier, research examined the Tuning process for baccalaureate Elementary Teacher Education in Utah colleges and universities. Specifically, I explored the process of developing the Elementary Teacher Education discipline student learning outcomes and competencies and the role of faculty and leadership in ETE discipline tuning. The research was guided by the following questions: 1. How have faculty been engaged in the Utah Tuning project? 2. How has the Tuning project influenced Elementary Teacher Education in Utah? 3. a. Who provides the leadership direction for tuning Elementary Teacher Education in Utah? b. What factors have been used to advance the Utah Tuning project? In this chapter, I explain the course and logic of the decision-making throughout the research process. I describe the rationale for the qualitative research, the research site and sample, and the type of information needed for the case study, the design of the study and methods for data collection, analysis, and synthesis. I consider ethical and political issues involved in the study, issues of trustworthiness, and limitations of the study. I 52 include various figures, and visuals, which appear as working tools for the methods of this research. Following is the section on the rationale for this study. Rationale for Qualitative Research The section offers a rationale for this research approach. This research study focused on understanding how faculty members work together to develop criterionreferenced statements of learning outcomes and competencies for the Elementary Teacher Education discipline and how they make meaning from their Utah Tuning experiences. The focal point of this explorative study was to determine the key elements of this endeavor. For instance, what was the level of understanding of the faculty on tuning the discipline's content over time? What strategies did they use, if any, to articulate student learning outcomes and competencies for the discipline? What challenges did the faculty members face in developing and implementing the discipline's learning outcomes and competencies? What support did the faculty members need to tune the discipline? This exploration has required a method that goes beyond an attempt to quantify such characteristics for the purpose of attaining the insights of the whole process, and investigates the complexities of the Tuning process and practice. Overall, the research goal, objectives, and research questions of this study have determined the type of the research to be conducted. Taking into account that the study has focused on the real professional experiences of the real people - the Elementary Teacher Education faculty - in real setting - the Tuning process and practice of the ETE discipline in Utah colleges and universities, the study has required a qualitative research method for delving into the complexities of the process (Marshall & Rossman, 2011). 53 I have been interested in exploring, analyzing, and describing the meanings faculty used to understand the Tuning process and practice. Accordingly, the power of Utah ETE Tuning positioned "in the words of the participants and the analysis of the researcher" (Morrow & Smith, 2000, p. 200). Qualitative studies are about understanding the meanings that individuals construct in order to participate in their social lives (Bogdan & Biklen, 1998; Erickson, 1986; Hatch, 2002; Lincoln & Guba, 1985). Hatch (2002) emphasized that qualitative research centered on the importance of meaning: (a) human beings act towards things on the basis of the meaning that the things have for them; (b) the meaning of such things is derived from, or arises out of, the social interaction that one has with one's fellows; and (c) these meanings are handled in, and sometimes modified through, an interpretive process used by individual in dealing with the things they encounter. (p. 8-9) Adelman's (2009) definition of Tuning adopted for this study as a working one given on p. 14 of this study conceptually highlights four key elements: who, what, how, and why. Figure 1 "Key Elements of Tuning Methodology" (see p. 91) shows the fundamental elements of the definition of Tuning. First, who: faculty are the critical actors. Second, what: faculty's main activity is to write student learning outcomes and competencies in the disciplines. Third, how: faculty consult with and get input from students, employers and other stakeholders. Through their consultations they develop ‘reference points' for writing learning outcomes and competencies. Fourth, why: the main goal of Tuning methodology is to produce generic and subject-specific learning outcomes and competencies providing a common language for them. The core of my research is faculty's purposeful and creative work at writing student learning outcomes and competencies. I explored the Utah ETE Tuning's unique, dynamic, and complex existence. I 54 systematically examined this social context as a whole, without breaking it down into isolated, incomplete, and disconnected variables. In particular, I was looking at both context and the Utah ETE Tuning process and practices. This qualitative study provided complex, detailed narratives, comprehensive descriptions including the voices of the participants being studied. I believe, this study included enough details and actual data to take the readers inside the social situation under investigation (Bogdan & Biklen, 1992; Erickson, 1986; Hatch, 2002; Lincoln & Guba, 1985). Research Paradigm The research paradigm guiding this study is rooted in social constructivism focusing on how individuals act towards social structures and how they construct the meanings based on their understanding of the experiences. In particular, the emphasis is on how faculty work together to tune the ETE college discipline - develop clear outcomes and competencies: what each student must know and do to get a college degree. By doing this, the faculty create transparent pathways for the students to earn college degree, which in turn, contributes to and enhances learning, teaching, assessment, and performance in relation to quality assurance and control in the educational process. In the constructivist paradigm the nature of reality is relative: there exist local and specific realities (Guba & Lincoln, 1994). These multiple realities "are inherently unique because they are constructed by individuals who experience the world from their own vantage points" (Hatch, 2002, p. 15). The constructivists acknowledge also that people share their perspectives in social groups, and their perspectives and meanings arise out of their social interactions. Accordingly, the objects of inquiry are individual perspectives 55 or constructions of reality. As a result, realities are ontologically apprehendable in the form of abstract mental constructions that are experientially based, local, and specific (Guba & Lincoln, 1994; Hatch, 2002). Epistemologically, the constructivist researchers are interested in individual constructions of reality, what can be known. They assert that "knowledge is symbolically constructed and not objective; that understandings of the world are based on conventions; that truth is, in fact, what we agree it is" (Hatch, 2002, p. 161). In this paradigm, "the knower and the known are taken inseparable" (Hatch, 2002, p. 10). From this perspective, the researcher and the participants of the study join together in the process and "construct the subjective reality" (Hatch, 2002, p. 10) that is under examination. The researcher is a part of the world she studies. Accordingly, the constructivist approach provides a method of inquiry. Knowledge is gained through naturalistic qualitative research: the data collection and analytic tools of the constructivist (Lincoln & Guba, 1985). The researcher spends extended periods of time interviewing participants of the study and observing them in their natural settings in order to "reconstruct the constructions participants use to make sense of their worlds" (Hatch, 2002, p. 15). This research paradigm emphasizes an inductive and triangulated approach to the construction of knowledge, systematic inductive guidelines for collecting and analyzing data to build middle-range theoretical frameworks that explain the collected data. The strategies include simultaneous collection and analysis of data, a two-step data coding process, comparative methods, memo writing aimed at the construction of conceptual analyses, sampling to refine the researcher's emerging ideas, and integration of the theoretical framework (Charmaz, 56 2006). The principles of content and discourse analysis, and hermeneutics are used to guide the researcher's interpretive coconstructions of the participants' perspectives (Guba & Lincoln, 1994). In this framework, produced knowledge is often presented in the forms of case studies or rich narratives. Case studies describe the interpretations constructed as part of the research process. Accounts include sufficient contextual details and necessary representation of the participants' voices. The quality of the findings based on criteria of "credibility, transferability, dependability, and confirmability" (Denzin & Lincoln, 1994, p. 14). These criteria "replace the usual positivist criteria of internal and external validity, reliability, and objectivity" (Denzin & Lincoln, 1994, p. 14). Working in the constructivist paradigm, I analyzed the case of Utah Elementary Teacher Education discipline tuning to explore this particular phenomenon. Researcher's Positionality My research interests have been shaped by my roles as a faculty member, administrator, and international master and doctoral student. They are derived from my teaching and administrative practice and have gradually evolved from being focused on the discipline of applied linguistics to the more complex questions of university education connected with the significance and quality of a college degree. The path from where I was ten years ago to where I am now has been a long one, with many unexpected turns and stops. At the time Russia joined the Bologna Process (September 2003 in Berlin, Germany) I worked as both a director of a language training center and an English Language department chair. I began questioning what students could get out of college. 57 Do students see their college experiences as cohesive, interrelated, connected, and additive? Are their college experiences meaningful, tangible for them? What do they acquire besides credits to earn a degree certificate? Can students see their day-to-day experiences in relation to the ends? Can they assess their own progress toward the ends? I also questioned if it were necessary that the curriculum and programs offered by colleges and universities should be improved and upgraded by all faculty in a discipline or program together. This collaboration could provide faculty's professional development, and, in turn, better teaching, and better learning. With these questions in my mind, I searched the possibility of learning how the Bachelor-Master degree system functioned from within. In 2006, my educational path led me to the master's degree program in the Educational Leadership and Policy department at the University of Utah. Being a Muskie Scholarship Fellow gave me a unique opportunity to study theoretically and in practice different aspects including methods, procedures, and principles of American higher education. However, I questioned the value of my degree and where I could apply it. I was not exposed to clear pathways or student learning outcomes and competencies. I faced many academic challenges. Very often syllabi did not specify learning outcomes or competencies. Even less the faculty's expectations for the students were not articulated in operational verbs: what student must demonstrate that could be assessed at the end of the course. My Muskie fellow experience was a turning point in my thinking about quality of higher education system in both regional and global context. Reflecting and comparing two higher education systems - Russia and the United States - greatly changed my perspectives on education. 58 My experience as a master's degree student followed by two years of work as an adviser to the first-vice president in the Moscow High School of Economics - Research University in Russia - further motivated my interests in transparent content of a college degree. At this position, I mostly worked with faculty members providing systematic professional development on best teaching practices to Language Department staff. In collaboration with other faculty we carried out a needs assessment of language teaching and provided recommendations to restructure and improve language teaching, and I acted as liaison between the Language Departments, other departments, and external institutions to provide coordination and assure that vocational and departmental needs were met. Overall, I was responsible for leading the development of the English Language teaching policy, including curriculum, syllabi and materials design, and standardized assessment practices. I was also teaching English for academic purposes. Teaching students and getting to know their academic needs and expectations, and listening to their perceptions of their college experiences contributed to the developing question of the meaning of a college degree. The repeated questions were about the value of a college degree, and what organizational institutional changes needed, what kind of work needed to organizationally change the system that did not meet the society's requirements. At the same time, I tracked the changes in higher education systems in Europe and the USA. The desire to research and understand these processes drove me to a doctoral program in education. I further questioned if a college or university exist for the sake of a student or for the sake of a professor. Faculty commitment is critical. How do faculty commit to students' college experiences? How do faculty develop student learning outcomes and 59 competencies? How could faculty intentionally and purposefully work together to create clear pathways for students to earn a higher education degree? Do faculty do their best? If yes, what do they do exactly? Do universities provide their students with the education, environment, or service that emphasize adequate global, international, and intercultural perspective to meet the social, academic, and professional needs of both the domestic and foreign students in order to assist them to function appropriately in this increasingly multicultural and diversified world? Over time, my teaching and collaboration with my colleagues shaped my understanding of colleges and universities, and I began thinking more focused at how faculty work together to develop clear pathways for our students through earning a college education. My academic experience of moving from developing syllabi and programs for an English Language Department to designing curriculum for a language center to leading of a language teaching policy development for the whole university presupposed my research on Tuning. One experience in particular provided a powerful affirmation I had made the right decision to explore the Tuning process. Early in my Ph.D. program I took a class on writing research proposals. At the outset there were no outcomes prescribed, but I waited for several classes before asking about them. The response was nonresponsive. Then after a few more class sessions, my fellow students began asking whether we could see samples of research proposals. The response was affirmative, but then none was ever made available. Later in the semester, I sought input from the instructor on Tuning as a research topic. The immediate response was "oh, it's about standardization." Once this utterance passed the instructor's lips, there was nothing I could say to change her perception. My classmates and I muddled through the course, and based on my 60 discussions with them, none was satisfied that they had learned even the slightest thing about writing research proposals. What I have learned from this experience is that students need and deserve transparent expectations and outcomes for their classes and for their degree. Learning outcomes must be clearly stated to help students realize what they must know, understand and demonstrate at the end of the course. My academic experiences have situated me in a unique position that has given me a possibility to explore the phenomenon "disciplinary tuning" from different angles and see different aspects of the reform in higher education in the United States and other countries. As I have been investigating the field of higher education, I have gradually developed my own understanding that Tuning has identified the shift in higher education, and focuses on discipline-specific content, skill development, and student learning outcomes and competencies. This process helped me to narrow down the research, articulate the final research questions, identify the research territory, and my role as a researcher. Accordingly, my experiences shared above - personal and professional - influenced how I collected, interpreted, and presented the findings of the research. I focused on how faculty purposefully and creatively worked at developing clear pathways for students studying for their college degree. I explored the insights of the ETE Tuning process and practices of how faculty articulated student learning outcomes and demonstrable competencies. Researcher as an Instrument The previous section presented my positionality as a researcher: experiences, values, philosophical, and theoretical assumptions that have impacted what questions 61 have been asked and how they could be answered (Guba & Lincoln, 1994). In this section, I discuss my approaches to how the act of studying a social phenomenon influences the enactment of that phenomenon (Hatch, 2002). The researcher-asinstrument approach means that a qualitative researcher directly gathers the principle data for her study. The researcher is "the tool of the investigation" (Morrow & Smith, 2000, p. 219). Even when specific computer programs are used to support qualitative work, "data take on no significance until they are processed using the human intelligence of the researcher" (Hatch, 2002, p. 7). Smith (1980) emphasized, "we cannot with assurance separate in the written account the characteristics of what is studied and the characteristics of the researcher" (p. 1). My research aimed to understand the participants' perspectives in natural contexts. Consequently, I spent enough time with the participants in those contexts to feel confident that I was capturing what I claimed (Bogdan & Biklen, 1998; Erickson, 1986; Hatch, 2002; Lincoln & Guba, 1985; Walsh, Tobin & Graue, 1993). The qualitative researchers "emphasize, describe, judge, compare, portray, evoke images, and create, for the reader or listener, the sense of having been there" (Guba & Lincoln, 1981, p. 149). My qualitative research revealed how all the parts of Utah ETE Tuning worked or did not work together to form a whole. The meaning is embedded in people's experiences and this meaning is mediated through the investigator's own perception (Hatch, 2002; Smith, 1980). Patton (1985) described: [Qualitative research] is an effort to understand situations in their uniqueness as part of a particular context and the interactions there. This understanding is an end in itself, so that it is not attempting to predict what might happen in the future necessarily, but to understand the nature of that setting - what it means for participants to be in that setting, what their lives are like, what's going on for them, what their meanings are, what the world looks like in that particular setting - and in the analysis to be able to communicate that faithfully to others who are interested in that setting… The analysis strives for depth of understanding. (p. 1) 62 Being a qualitative researcher, I am really interested in inner states as well as outer forms of human activity (Hatch, 2002). Since the inner states cannot be observed directly, I "must rely on subjective judgements to bring them to light" (Hatch, 2002, p. 9). The qualitative researcher cannot escape subjective judgements in description, analysis, and interpretation. Accordingly, the researcher must be aware of and address her biases and subjectivity. Bias can be found in all research projects whether qualitative or quantitative. Hatch (2002) stated that the stance of a qualitative researcher is "to concentrate on reflexivity" applying her "own subjectivities in ways that make it possible to understand the tacit motives and assumptions" (p. 7) of the study's participants instead of pretending to be objective. Reflexivity, "the process of personally and academically reflecting on lived experiences in ways that reveal deep connections between the writer and his or her subject" (Goodall, 2000, p. 137), is crucial to the integrity of qualitative research. Being reflexive places me in a distinctly different position than that of the "objective scientist" in quantitative research. "The capacities to be reflective, to keep track of one's influence on a setting, to bracket one's biases, and to monitor one's emotional responses are the same capacities that allow researchers to get close enough to human action to understand what is going on" (Hatch, 2002, p. 10). I agree that subjectivity "can be seen as virtuous, for it is the basis of researchers making a distinctive contribution, one that results from the unique configuration of their personal qualities joined to the data they have collected" (Peshkin, 1988, p. 55). The desire to understand the Tuning process and practice and gain insights from the faculty's Tuning experiences on transforming institutions of higher education reflects the constructivist research paradigm guiding this study. Several assumptions drove my 63 approaches to this research. First, I have assumed that there are faculty who intentionally and creatively attempt to develop and implement big changes in higher education to student-centered, learning-centered, competency-centered college disciplines. They understand the great need for changes in this direction to work on a discipline level across the colleges and universities. They are pioneers in this educational endeavor. Second, I have assumed that institutional structures must be changed to meet the current requirements of the society for higher education: the increasing demands for career-ready graduates. At a micro level, I believe the institutional work - work aimed to change institutions or work on institutions - is crucially important: how the key agents, faculty, as the most close to students, work together to create, maintain and disrupt institutions. Finally, the national and global context of higher education plays a significant role in the current demands for the changes in higher education. This study incorporated an understanding of this context and considered how it could continue to influence the higher education reform. This research was through these assumptions. Design of the Study This section explains why a case study design has been chosen and how it matches the research purpose and questions. My study of Utah Tuning for Elementary Teacher Education presents a single-case study design which is "analogous to a single experiment" (Yin, 2014, p. 52). Merriam (1988) defined a qualitative case study in terms of its end product as "an intensive, holistic description and analysis of a single instance, phenomenon, or social unit" (p. 21). According to Miles and Huberman (1994), a case is "a phenomenon of some sort occurring in a bounded context" (p. 25). They graphically 64 portrayed a case as a circle with a heart in the center, where the heart was the focus of the study, while the circle "defines the edge of the case: what will not be studied" (Miles & Huberman, 1994, p. 25). Their visual image as a case of a unit of analysis correlates with Yin's (2014) scheme of holistic single-unit case study (p. 50). Adopted to my Tuning research, the Utah ETE Tuning holistic single-unit case study has a formal design presented in Figure 2 Utah ETE Tuning Holistic Single-Unit Case Study (see p. 92). The Utah ETE Tuning project itself is contextualized in Utah Tuning (for other three disciplines: history, physics, general education mathematics), and in Tuning USA as a bigger context. The figure shows that this design includes the analysis of the context in relation to the "case." The dashed lines between the contexts and the case indicate that the boundaries between them are not clearly obvious. Generally, "the case is an integrated system" (Stake, 1995, p. 2). Yin (2014) outlined "the holistic (single-unit of analysis)" case study as one that centers "on the global nature of an organization or a program" (p. 53). Accordingly, in my holistic single-case study I focused comprehensively on Utah Tuning for Elementary Teacher Education. The qualitative single-case study method immersed me in the real-world case Utah ETE Tuning and permitted me to "deal with the reality behind appearances, with contradictions, and dialectical nature of social life" (Sjoberg, Williams, Vaughan, & Sjoberg, 1991, p. 39). Case study methods required me to choose a particular unit of analysis as "each case study and unit of analysis either should be similar to those previously studied by others or should deviate in clear, operationally defined ways" (Yin, 2014, p. 34). The unit of analysis for this study was the college discipline level for the Elementary Teacher Education in Utah. A review of the research literature demonstrated that discipline 65 Tuning was not studied as a unit of analysis in relation to its role in providing transparent pathways through clear learning outcomes for all ETE college students. I have chosen this case study site because what goes on in the case is critical for understanding of the Utah Tuning process or practice, which is unique, distinctive, and different from other discipline Tuning. The Utah ETE Tuning case study is a specific and complex functioning unit, which is intrinsically interesting from multiple perspectives: discipline Tuning and interorganizational work to develop transparent pathways and clear learning outcomes for ETE students, shift in higher education towards student learning outcomesbased, competency-based programs. Faculty's commitment is critical in this process. Even though eight reports on the Tuning pilot projects (ICHE, 2010; MHEC, 2014; MNOHE, 2010; THECB, 2011; USHE, 2009; 2011; 2012; 2013) have been done so far, we still know very little about insights of the Tuning process and practice. There is still a need to explore the ETE faculty Tuning experience because of the Elementary Teacher Education discipline, in particular. Elementary Teacher Education is directly connected with public elementary education. So, how colleges and universities prepare future teachers determines the quality of teaching in elementary schools, and this impacts the whole educational system, and even more - the future of the country. As noted earlier, doing a case study on Utah ETE Tuning allowed me to focus on a "case" and retain a holistic and real-world perspective. Yin (1994) explained a case study in terms of the research process as "an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident" (p. 13). Yin (2014) summarized her previous approaches to study cases in a twofold definition. On the one 66 hand, the definition emphasized the scope of a case study "a case study is an empirical inquiry that investigates a contemporary phenomenon (the "case") in depth and within its real-world context, especially when the boundaries between phenomenon and context may not be clearly evident" (p. 16). On the other hand, the definition also covered the features of a case study: A case study inquiry copes with the technically distinctive situation in which there will be many more variables of interest than data points, and as one result relies on multiple sources of evidence, with data needing to converge in a triangulating fashion, and as another result benefits from the prior development of theoretical propositions to guide data collection and analysis. (Yin, 2014, p. 17) This twofold definition explicitly identified the main characteristics of a case study, including "empirical inquiry," "contemporary phenomenon," "in depth," "real-world context," "many more variables of interest than data points," "multiple sources of evidence," "data needing to converge in a triangulating fashion." Overall, the dual definition shows how case study research comprises a comprehensive method, and covers the logic of design, data collection techniques, and specific approaches to data analysis, and is not limited to a data collection tactic or a design feature alone. The research questions of this case are exploratory and require an extensive and "in depth" description of the Utah ETE Tuning process and practice. They are justifiable rationale for conducting exploratory study, where one of the goals is to develop pertinent propositions for further inquiry. My single-case study design permitted me to yield insights into the innovative process of the discipline's reform in higher education - Tuning. Utah ETE Tuning as a complex case in a complex context favors holistic approach. The holistic single-case design let "add significant opportunities for extensive analysis, enhancing the insights into the single case" (Yin, 2014, p. 56). Additionally, 67 this type of design provided insights into the purposive, creative, and knowledgeable work of the faculty members and leadership engaged in Tuning. The Utah ETE Tuning case as a contemporary phenomenon will need some fieldwork to study this case "in depth" in order to achieve as full understanding of the phenomenon as possible. The recent past of this contemporary phenomenon is presented in the literature review chapter, including historical roots of Tuning and how Tuning USA has become a current higher education reform in the country. Epistemologically, accommodating a constructivist perspective, I acknowledge there exist multiple realities and multiple meanings from the study participants' perspectives, and findings depend on observer's perspectives. However, the convergence of participants' perspectives have created categories, models, and even a continuum, a typology that could conceptualize different approaches to the study. Sampling This section provides a comprehensive description of the research sample, sampling strategies and criteria used for sampling selection. The research questions and analytical framework provided the foci and boundaries for sampling decisions. This study is consistent with Merriam's (1998) viewpoint in that it has utilized purposeful sampling to select the participants as the first strategy for the case sampling. Merriam (1998) highlighted "purposeful sampling is based on the assumption that the investigator wants to discover, understand, and gain insight and therefore must select a sample from which the most can be learned" (p. 61). First, in order to begin purposeful sampling, I determined what selection criteria were essential in choosing the participants to be 68 studied. The criteria are important because they directly reflect the purpose of the study and guide in the identification of the case. The criteria used to choose the participants for this study include the following: 1. Participants must be in the Utah ETE Tuning team from the very beginning of the project or may have joined the team in the last full year. 2. Participants must be a faculty, a department chair, or a state coordinator involved in Tuning. 3. Participants must have responsibility for ETE discipline outcomes (performance, teaching) in Utah colleges or universities. The second strategy - snowball or chain - involved asking each participant to refer me to other participants. As Patton (1990) described, this strategy involved identifying participants or "cases of interest from people who know people who know people who know what cases are information-rich, that is good examples for study, good interview subjects" (p. 182). ETE faculty named other ETE faculty members who exemplified the contribution to the collection of data for this study. The main aim was "to find people who are knowledgeable, reliable, and accurate in reporting" (Miles & Huberman, 1994, p. 29) Tuning process and practices. I have referred to two types of sampling - purposeful and snowball - which perfectly worked together for the proposed research. First, I purposefully selected a sample to start the research in order to discover, understand, and gain insights of Utah ETE Tuning. Then, using a snowball strategy, I dug deeper to contribute to the collection of data. The overall goal was to discover reliable and accurate data. This qualitative sampling has three key features. First, this case sampling is 69 nested: studying Tuning Elementary Teacher Education, specifically, faculty's purposive and creative work aimed at developing student learning outcomes and competencies for the ETE discipline within Utah context within Tuning USA context. The second major point is that this sampling is theoretically driven by institutional theory and its new focus - institutional work. The theory is prespecified as "theoretical sampling" (Glaser & Strauss, 1998). Miles & Huberman (1994) emphasized: To get to the construct the researcher needs to see different instances of it, at different moments, in different places, with different people. The main concern is with the conditions under which the construct or theory operates, not with the generalization of the findings to other settings. (p. 29) Choices of informants, settings, events, and interactions were driven by a conceptual question - how Elementary Teacher Education faculty tune their college discipline - not by a concern for representativeness. The third point is the investigative character of the sampling: I am, as a researcher, a cerebral detective, searching for answers to the research questions. Following researchers (Merriam, 1998; Miles & Huberman, 1994; Patton, 1990; Yin, 2014), I have interviewed, observed, talked to people, and picked up the artifacts and documents which led to new samples of informants and observations, and new documents. How I Contacted Participants I contacted the Tuning team chair and initial research participants who met the criteria for this purposeful sampling outlined above by phone and by email in November 2014. Afterward, I made personal contacts with them to schedule the interviews. Initially, this was a relatively small sample of the Tuning team members, nested in the Utah ETE context aimed to be studied in depth. It is important to remember that the 70 purpose of the study is not to produce largely generalizable results, but to understand and gain insights of the Utah ETE Tuning team members' experiences as they make meaning from their discipline tuning efforts. I maintained contacts with the participants via telephone and e-mail throughout data collection, analysis, and writing. Utah ETE Tuning is an information-rich case "from which one can learn a great deal about issues of central importance to the purpose of the research" (Patton, 1990, p. 169). The power and significance of this study contributed to and enhanced my understanding and awareness of how faculty members and state Tuning leaders worked together to develop the transparent pathways and clear learning outcomes for every ETE college student in Utah. Accordingly, I believe this research provided valuable information for researchers, practitioners, and policy makers on how faculty, specifically, Elementary Teacher Education discipline faculty frame the needs of their discipline and ensure transparent pathways and clear learning outcomes for every college student, how the faculty tune the discipline. Data Collection This section discusses main sources of data for this study, more specifically, the purposes of the data sources and collection steps. The study utilized several sources of data, including individual, group, and focus group interviews, observations and field notes, and documentary materials. The main intention was to achieve a more complete understanding of the phenomenon under scrutiny (Bogdan & Biklen, 1998) and strengthen the study's use in other settings (Marshall & Rossman, 2011). Additional sources of data included my self-reflective journal and analytic memos produced during 71 the process of data collection. Through multiple sources of data, I was intentionally creating a comprehensive database for the chosen case under study. Following Yin's (2014) recommendations, I assembled evidence containing all case study notes, documents, filed materials, narratives, and memos. Individual and Focus Groups Interviews Interviews have been chosen because they help provide the most flexible means of investigating many complex dimensions (Fontana & Frey, 2003) of the Tuning process. I conducted 16 individual, group, and focus group interviews between November 2014 and June 2015. I conducted 45-60 min individual interviews with 10 ETE faculty tuners and two interviews with the Utah State Office of Education (USOE) representatives to the faculty Tuning team. Additionally, I conducted three group interviews and one focus group interview with faculty tuners and faculty members of the ETE programs, and one group interview with USOE representatives. All interviews were recorded and transcribed for data analysis. The main purpose of interviews was to gather the participants' perceptions of the discipline tuning experience in their own words. I agree with Patton (1990) about "the purpose of interviewing is to find out what is in and on someone else's mind" (p. 278). The researcher and participants engage in purposeful dialogue (Patton, 1980). Through personal interaction between the researcher and participants interviews disclose how participants construct their experiences (Marshall & Rossman, 1999). Accordingly, I asked questions in a way "to obtain meaningful information" (Merriam, 1998, p. 23). For example, how was your level of understanding on Tuning the ETE discipline's content 72 changing over time? (Appendix F presents the questions I posed to the participants during the interviews). The semistructured interviews consisted of open-ended questions designed to probe experts' experiences and insights. On the one hand, semistructured interviews limit flexibility to some degree, however, this approach supports reproduction of the case methodology. Applying these techniques, the similar questions in the interviews emerged similar categories during the data analysis stage. On the other hand, unstructured emerging questions provided a chance to conduct further theoretical sampling (Strauss & Corbin, 1998). The interviews will contain a general set of questions for all participants. These questions focused on the descriptions of the Utah Tuning project, including its features, procedures, and structures that contributed to faculty involvement in Tuning; questions focused on descriptions of the Elementary Teacher Education college discipline. First, I followed this line of inquiry reflected by the case study protocol, second, I also asked "actual (conversational) questions in an unbiased manner that also serves the needs" (Yin, 2014, p. 110) of the line of inquiry. I conducted group and focus groups interviews as a supplement to one-to-one interviews. The target audience for group and focus groups interviews was ETE faculty directly engaged in Tuning and also ETE faculty working in the ETE departments or programs in Utah colleges and universities but not directly involved in Tuning. The reason to involve other faculty in focus groups was to explore, understand, and gain insights from the process of institutionalization of new structures and activities connected with Tuning. Namely, how faculty's institutional work is being institutionalized in normative rules and regulatory processes, and how they (rules and processes) provide 73 stability and meaning to social behavior. Focus groups allowed faculty to reflect and recall their experiences. And of being able to listen to others, the participants could spur memories and opinions, provide instances of interchanges between contrasting perspectives, and amend some initial accounts. Interviews are targeted, that is, focus on the case study topics, and insightful because they provide explanations and personal views (Yin, 2014). Interviews with the Tuning participants identified the complex interplay of theory and practice in order to uncover as yet unrecognized or undocumented innovations (Marshall & Rossman, 2011). After my second interview, I compared the first set of data with the second one. This comparison, in turn, informed the next data collected, and so on, throughout the study. In this research approach, I also discovered emerging best practices, and commonalities and differences among the participants' experiences that were then shared and incorporated into a set of recommendations for Tuning practice, policy and future research. Observation Observations are "a primary source of data in qualitative research" (Merriam, 1998, p. 94) together with interviews. Marshall and Rossman (1995) defined observation as "the systematic noting and recording of events, behaviors, and artifacts (objects) in the social setting chosen for study" (p. 79). Observations are immediate: they cover actions in real time, and they are contextual, that is, cover the case's context (Yin, 2014). As an educational researcher interested in how to develop transparent pathways for students through their college degree, I observed how faculty worked together in ETE discipline tuning. For my research project, I observed one ETE Tuning meeting, two ETE Faculty 74 Discipline Major's meetings whose participants were mostly faculty tuners, and the meetings were directly connected with student learning outcomes and alignment of ETE discipline courses in Utah. I also participated and observed the Tuning group discussions during two annual "What is an Educated Person?" higher education conferences, where faculty tuners from "tuned" college disciplines (physics, history, general education mathematics) shared their experiences about Tuning dissemination and implementation at the department level. Guba and Lincoln (1994) stressed that a good qualitative researcher "looks and listens everywhere" (p. 142). Observing and listening are very vital communication skills. It is only by listening "to many individuals and to many points of view that value-resonant social contexts can be fully, equitably, and honorably represented" (Guba & Lincoln, 1994, p. 142). I believe the observations effectively provided additional context for this case study. Following Sharan Merriam's recommendations I focused on the participants, activities and interactions, deliberation and conversation, physical setting, and subtle factors (e.g., informal and unplanned activities, nonverbal communication), and my own behavior (Merriam, 1998). Simultaneously, I took extensive notes on the participants' interactions and approaches observed while elaborating on those already described in interviews. The field notes together with interview transcripts, as forms of data collection, underwent content analysis with explicit and implicit questions, including who said what to whom, why, to what extent and with what effect. Another form of field notes was audio tapes of my reflections at each stage of data collection. Taping my responses immediately following an interview or meeting observation and transcribing them provided me with additional analytic and reflective summaries of each interaction 75 with the participants. What was written down from observation became the raw data, from which the findings of the study eventually emerged. All written details on what I observed, overall, contributed to the comprehensive picture of the case under study. Documentation For my case study, the documents included all Utah Tuning reports, Tuning meetings' minutes, reports of the events, announcements, news clippings, articles appearing in the mass media, and other documents that could ground an investigation in the context of the Tuning process in Utah. I continually reviewed all information accessible through the Utah Tuning website (www.utahtuning.weebly.com). Documents as ready-made resources also aim to mine data that specifically address the research questions. They are important sources of data; they are stable, specific, unobtrusive, and broad (Yin, 2014). Like the interviews and observation, the focus of the documents analysis was the faculty's purposeful and creative work aimed at developing student learning outcomes and competencies for college students. Similar to the information gathered during the interviews and observations, documents were context-embedded, which helped "to ground an inquiry in the milieu of the writer. This grounding in realworld issues and day-to-day concerns is ultimately what the naturalistic inquiry is working towards" (Guba & Lincoln, 1981, p. 234). For document analysis, I mostly applied two types of coding: initial and focused coding. In initial coding I looked for what I could define and discover in the data (Charmaz, 2006). I used an inductive approach to identify the themes emerging from the initial coding of documents. Basically, I used myself as an instrument of the research by 76 my interests, expertise, commitment, and knowledge of Tuning. A corpus of initial coding, when accumulated, was numerous and varied. At the next stage - focused coding - the initial corpus became itself an object of review in terms of broader topics and questions, overarching ideas and/or propositions. Some categories were elaborated, some were collapsed or dropped. The focused set of codes was then applied to an increasing array of data. The aim was "to attain a condensed and broad description of the phenomenon, and the outcome of the analysis should be concepts describing the phenomenon with the purpose for those concepts to build up a model, conceptual system, conceptual map or categories" (Elo & Kyngas, 2008, p. 108). These documents exist in the context of Elementary Teacher Education as an academic field of higher education, which is intertwined with the fabrics of educational reforms and leadership, and in the context of Utah Tuning. In order to properly understand the content of these materials, I placed the documents in a relevant academic context and analyzed them within this context and factual framework. I also reviewed the Tuning reports from other states: Indiana, Kentucky, Minnesota, Montana, Texas, and the Midwest Higher Education Consortium. The data collection from the documents was guided by the research questions of the study and emerging findings. My analysis of the documentation was systematic and open to new insights, and sensitive to the up-andcoming data similar to interviewing and observing. Field Notes and Researcher Responses Throughout the data collection process, I took detailed field notes. During observations, written notes were the primary means for recording data. I immediately 77 typed the notes following observations that allowed me to fill in gaps while events were still fresh in my mind. Another form of field notes were audio tapes of my reflections at each stage of data collection. Taping my responses immediately following an interview or meetings observation and transcribing the tape provided additional analytic and reflective summaries of each interaction with my research participants. The field notes helped me interrogate the information from interviews and documents, and compare the data from different sources which, in turn, provided the credibility of the findings. Additionally, I maintained a self-reflective journal throughout the research process. This journal provided a place for me to process what occurred in the field on both a personal and analytical basis. Information from this journal was included as data where appropriate. Also, I used analytic memos and summaries as additional data sources. Analytic memos are the beginnings of data interpretation as they make links between data and observer comments (Bogdan & Biklen, 1998). I wrote memos to myself about my reactions and reflections, tentative themes, methods of the study, hunches, ideas, and emerging findings. I also noted things I wanted to ask. The process of data collection stops when the researcher achieves exhaustion of sources, saturation of categories, emergence of regularities, and overextension (Lincoln & Cuba, 1985). The overall goal and strategy of the data collection is to explore and uncover meaning, and obtain insights relevant to the research problem in order to answer the research questions. Data Management Preceding section has focused on how to collect data for a qualitative study through interviews, observations, documents, and other sources. This section discusses 78 the ways I organized, documented, and managed the data collected for this case study. The managing data included the following steps. First, all data from the interviews, observations, documents and other materials collected from the field were built into the data corpus of the Utah ETE Tuning case study, which was a separate and orderly compilation. I have developed a portfolio which contains the full array of data and holds a mixture of folders with documents and materials. The data corpus' main function is "to preserve the collected data in a retrievable form" (Yin, 2014 p. 124), and a wellorganized data corpus made my own analysis easier later. I believe the creation of a data corpus markedly increased the reliability of the entire case study. Second, I have developed an annotated bibliography of the documents collected during the research. This bibliography served as an index and facilitated the documents' storage and retrieval. For example, for interviews I have assigned some shorthand labels so that one can easily retrieve specific pieces of data. Each interview was named with a role in Tuning (faculty or state leader), type of interview (individual, group or focus group), and the number of the interview (e. g., Faculty_individual interview_1). All individual interviews with faculty and USOE representatives were named as "faculty_individual interview" to preserve anonymity. The disposition of the documents was presented in the case study protocol. I transcribed all recorded interview and observation data by myself and entered them into a larger body of data. Documentation, field notes, and personal observations were also included in the data corpus. By transcribing the data myself, I conducted an initial phase of analysis. Following transcription of taped data, copies of transcripts were made available to all participants of the study to check for accuracy. I asked the 79 participants to evaluate their own words as well as initial analysis notes. The use of a computer-assisted data analysis program, HyperRESEARCH (the qualitative data analysis tool), helped me with data management by organizing the data, simplifying the movement of data, and developing visual data displays. Beyond these considerations, other activities contributed to the strength of the study. They included the self-reflective journal, the use of multiple data sources in order to provide a deeper understanding of faculty experiences in Tuning, and my full immersion in the field and data to ensure those empirical assertions and their key linkages were supported by the evidence. Additionally, an audit trail, which is a chronological list of all collected data and the analysis process, accompanied the final research report. The audit trail provided documentation supporting the use of the intended data collection and analysis methods. Data Analysis Strategies This section aims to fairly present the analysis strategies that I applied to analyze and synthesize a mass of collected data. The section explains how I, as a researcher, devoted myself to following a rigorous analytical path. It includes two main subsections on content analysis and discourse analysis, more specifically what and how they were done. Marshall and Rossman (1999) defined data analysis as "the process of bringing order, structure, and interpretation to the mass of collected data" (p. 150). In my research, the mass of collected data consists of interviews, observations, documentation, field notes provided by the research participants in verbal and written forms. Polkinghorne (1995) described a way to analyze qualitative data as a goal of identifying 80 "particulars as instances of general notions or concepts" (p. 13). The approach to the analysis of these data sets was thematic, that is, a process in which the developer seeks to expose recurrent concepts and ideas that will enhance understanding of how the Tuning process is construed in the context of higher education reforms. Coding for Interviews, Focus Groups, and Content Analysis Data analysis methods for this case study mainly included content and definition analysis utilizing descriptive-interpretive orientation. According to Merriam (1998) "all qualitative data analysis is content analysis in that it is content of interviews, field notes, and documents that is analyzed" (p. 160). Content analysis of texts is concerned with social reality and the results of analysis and their interpretations are correspondingly interconnected and dependent. Content analysis is technically connected with coding. The process of coding is basically one of selective reduction. Coding as a part of the inductive approach (Lincoln & Guba, 1985) for this study was interactional, not sequential. Coding is "assigning some sort of shorthand designation to various aspects" (Merriam, 1998, p. 164) of data so that the researcher can easily retrieve specific pieces of data. Saldana (2013) defined a code as: In qualitative data analysis, a code is a researcher-generated construct that symbolizes and thus attributes interpreted meaning to each individual datum for later purposes of pattern direction, categorization, theory building, and other analytic processes. Just as a title represents and captures a book, film, or poem's primary content and essence, so does a code represent and capture a datum's primary content and essence. (p. 4) Miles, Huberman & Saldana (2014) believed "coding is deep reflection… deep analysis and interpretation of the data's meanings" (p. 72). Saldana (2013) divided coding into 81 first cycle and second cycle coding major stages. During the first cycle coding the researcher initially assigns codes to the data chunks, while during second cycle coding the researcher generally works with the results of the first cycle. Throughout the first cycle coding I used descriptive coding techniques, that is, summarized data in a short phrase or word, condensing a basic topic of a discrete segment of qualitative data. As a result, this procedure provided a list of themes for categorizing the interviews' content. I also applied In Vivo coding, which is "words or short phrases from the participants' own language in the data record" (Miles, Huberman & Saldana, 2014, p. 74). Applying In Vivo coding I prioritized the participants' voices, which pointed to regularities or patterns in the study. Additionally, in order to denote observable actions in the data, I applied a process coding technique, which uses "-ing" words (gerunds). I believe this type of coding is especially useful for the Tuning case study because it could identify and make implicit processes and practices explicit as they emerge, change, occur in particular sequences or become strategically implemented when faculty and leadership purposefully work at creating, maintaining or disrupting institutions. Further, I compared one segment of data with another to determine similarities and differences, for example, one quote about faculty's work on developing student learning outcomes and competencies with another quote by the same or another participant. Then data were then grouped on a similar dimension, which tentatively got a name. Finally, step by step it resulted in a category. The overall goal of this analysis was to seek patterns in the data. These patterns were arranged in relationships to each other in the building bigger themes and concepts. During the second cycle coding I grouped the emerged summaries into a smaller 82 number of patterns, categories, or themes. Pattern codes are inferential codes, that identify an emergent theme, or configuration based on the material from first cycle coding and pooled into more meaningful and parsimonious units of analysis (Miles, et al., 2014). Through condensing a large amount of data into a smaller number of analytic units, pattern coding help the researcher elaborate an evolving, more integrated schema for understanding the Utah Tuning local interactions. I also applied pattern coding to interrogate the data categories to fully explore the application of the new institutional analytical framework. I applied the two-stage reading process, correlated with two-cycle coding method and took notes of any salient features of the texts. I went through the texts again and again (from text-level to sentence-level to word-level) going to deep microscopic levels of analysis (Huckin, 2002). Importantly, I illustrated my analyses by several extracts quoted from the collected data. I did not merely quote, summarize, and paraphrase participants' discourse, but provided a detailed, systematic, and theoretically-based analysis of all textual data. I have used some computer programs, which offer distinct advantages over elements of the manual process. The programs vary in their complexity and sophistication, but their common purpose is to assist researchers in organizing, managing, and coding qualitative data in a more efficient manner (Foster, 2004). The basic functions that are supported by such programs include text editing, note and memo taking, coding, text retrieval, and code/category manipulation (Tesch, 1990). Specifically, I have found the HyperRESEARCH program to be particularly helpful. It is essentially a "code and retrieve" program that allows researchers to go through the interview texts on the computer screen, to divide the texts into segments or chunks by a 83 particular category, and attach codes to the segments. Finally, the researchers can find and display all examples of these segments by the push of a button. The HyperRESEARCH is also sophisticated enough to allow for hierarchical or multilevel coding of data. In addition, it has "source tags" which allow the researcher to see where a retrieved segment has originally come from (the actual original interview, whom, when). There are also memo areas where the researcher can write extended reflections about specific data. Because of these advantages to retrieve data more accurately and rapidly by category, in my own research they accelerated my ability to organize and process findings. Trustworthiness This section demonstrates how this study aimed to be believable, accurate, and plausible to meet the criteria for evaluating trustworthiness of this study. The section describes strategies to enhance the trustworthiness of the study regarding credibility, dependability, and transferability. Lincoln and Guba (1985) and Guba and Lincoln (1998) proposed various criteria of the trustworthiness of qualitative research: credibility, which relates to validity in quantitative research; dependability correlated with reliability, and transferability related to generalizability in quantitative research. Overall, criteria for evaluating qualitative research focus on how well the researcher analyzed and described the evidence of the study and persuaded the readers to trust the results of the study (Bloomberg & Volpe, 2012, p. 112). Research is valid, when it reflects the described world. I have accurately represented the participants' perspectives (Bloomberg & Volpe, 2012), which provides 84 credibility of the research. This single case study was strengthened by various forms and methods of collected evidence, that is, triangulation of information (Adler & Adler, 1994; Huberman & Miles, 1998; Yin, 1984; 1994), which allowed me to address a broader range of historical, attitudinal, and observational issues" (Yin, 1984, p. 91). According to Miles and Huberman (1994), "triangulation is supposed to support a finding by showing that independent measures of it agree with it or, at least, do not contradict it" (p. 266). I conducted substantial fieldwork, used multiple methods of data collection, collected multiple sources of data, reviewed the transcribed texts and conclusions of the research participants, included colleagues' review of the field notes, and clarified my bias. While triangulation does not eliminate the possibility of misinterpretations or misrepresentations, it is useful because it "reduces the likelihood of misinterpretation, [since] we employ various procedures, including redundancy of data gathering and procedural challenges to explanations" (Stake, 1994, p. 241). Additionally, the use of multiple sources of evidence enhances construct validity and reliability because "essentially [they] provide multiple measures of the same phenomenon" (Yin, 1984, p. 91). All sources of collected data for the proposed research were integrated into a coherent case study of Utah ETE Tuning and were the inclusive basis for the entire study providing a real strength of the case study (Yin, 2014). Yin (2014) emphasized "the development of convergent lines of inquiry" (p. 120) as the most important advantage in using multiple sources of data, which allows a researcher to address a broader range of key issues. Convergence of evidence happens when the findings of the case study are supported by many sources of data. I triangulated the data from individual and focus 85 group interviews, observations, and documentation and intentionally corroborated the comprehensive findings. Figure 3 "Convergence of Evidence" based on Yin (2014) provides a visual representation of convergence of evidence for this study (see p. 93). Since this case study is based on several different sources of information, I believe its findings and conclusions are convincing and accurate. Audit Trail During this study I have kept an audit trail or "chain of evidence" (Yin, 1984), which is a chronological list of all collected data and the analysis process. Accordingly, it accompanied the final research report. My tracking of processes and practices for data collection and interpretation provided dependability of the research. The audit trail provided documentation supporting the use of the intended data collection and analysis methods, and included documents, the time, and place of events, citations of specific conversations, and observations. Consequently, the "chain of evidence" enhanced reliability. The notes included the protocol questions, research procedures, notes regarding preliminary findings, interviews, or observations, relevant citations from documents, and a list of documents reviewed during document analysis. Besides, the field notes were set an opportunity to "compose open ended answers to the questions in the case study protocol" (Yin, 1984, p. 95). Case study field notes and memos kept throughout the study process also increased case reliability because of their ability to document the research process in an audit trail, and to facilitate comparative analysis among emerging categories (Yin, 1984; 1994). In order to present "a holistic and realistic picture" (Bloomberg & Volpe, 2012, p. 86 113), I provided "thick description" (Denzin, 2001) of the research. This refers to the transferability of the research or to "the fit and match between the research context and other contexts as judged by the reader" (Bloomberg & Volpe, 2012, p. 113). Data analysis began with data collected from Tuning interviews, focus groups, observations, and documents. Expanding, I revisited the analysis during the research project through the use of field notes (Mertens, 1997). Field notes were continued throughout the research study to capture "impressions, thoughts, and directions" as well as to provide a "record of analysis, thoughts, interpretations, questions, and directions for further data collection" (Glaser & Strauss, 1998, p. 223). Through data coding, content analysis, discourse analysis, the research results provided holistic and realistic picture of the case under study and let readers understand how Utah ETE Tuning process and practices occurred and whether it was possible to apply them in other settings. The following activities took place in the process of study to manage my subjectivity. First, I participated in a peer research group that supported to examine the biases' impact on the research. This group consisted of doctoral students who also conducted qualitative studies. The colleagues assisted to view my study from different perspectives because they were from outside of my particular area of higher education. Second, I wrote a self-reflective journal, which became a part of my final research. I recorded my ideas, questions, thoughts, personal emotions, and statements in the journal during the research process. This, in turn, arranged for an account of the research process through which I went together with the participants. I constantly verified analyses of the data with the research participants and members of my dissertation committee during the process. 87 Ethical and Political Considerations The purpose of this section is to present in what ways the proposed case study addressed its ethical and political issues. Qualitative researchers are continually supposed to be "ethically sensitive, thoughtful, and knowledgeable" (Lofland & Lofland, 1995, p. 35) because they make decisions vis-à-vis the ethics of their study. Bogdan and Biklen (1998) suggested taking into account the following concerns when conducting research: 1... [T]he subjects' identities should be protected also that the information you collect does not embarrass or in other ways harm them. 2. Treat subjects with respect and seek their cooperation in the research. 3. In negotiating permission to do a study, you should make it clear to those with whom you negotiate what the terms of the agreement are, and you should abide by that contract. 4. Tell the truth when you write up and report your findings. (pp. 44-45) The ethics of qualitative research are unique to the research theme, site, participants, researcher, and methods involved in the study. My study followed the code of ethics presented in the Ethical Standards of the American Educational Research Association (AERA), which states: A main objective of this code is to remind us, as educational researchers, that we should strive to protect [studied] populations, and to maintain the integrity of our research, of our research community, and of all those with whom we have professional relations. (AERA, 1992, p. 1) This research was approved by the Institutional Review Boards (IRB) of the University of Utah, and was begun only after I received the permission to conduct the study. The AERA and IRB guiding principles explicitly address two ethical issues: voluntarily commitment to participate in the research, and the issue of confidentiality. The first ethical principle - autonomy or voluntary commitment - means that, as a researcher, I must, and I did respect each participant as a person capable of making an 88 informed decision about her/his participation in the research study. "It is the principle that seeks to ensure that all human subjects retain autonomy and the ability to judge for themselves what risks are worth taking for the purpose of furthering scientific knowledge" (Bloomberg & Volpe, 2012, p. 111). I ensured that each participant received a full disclosure of the nature of the study, the risks, benefits, and alternatives, with an extended opportunity to ask questions. This principle were explicitly described in the informed consent document, which appeared to be central to the ethics of the research. (See Appendix G for a copy of the consent form utilized at the University of Utah and that was used in this study). Informed consent entailed more than obtaining a signature on a form was given freely, without coercion, and was based on a clear understanding of what participation involves. The consent form explains the study rationale, interview, recording, and transcribing methods, and assurances for voluntary participation and confidentiality (Lincoln & Guba, 1985). From the perspectives of confidentiality, I informed the participants that the information they provided would be kept confidential to the greatest degree possible. The limits that existed to this assurance were clearly explained to the participants prior to the study. The participants were made aware of the right to withdraw from the study at any point in time, without any negative consequences. When the participants asked me to remove some data from the dissertation report, I accepted and satisfied their requests. Ethical and political issues can appear in all phases of the research, including data collection, data analysis, interpretation, and presentation of the study findings (Merriam, 1998). Here may appear a series of problematic questions. For example, would analysis and writing about the collected data denigrate those who produced them? In what ways? 89 Could I be viewed as a "lurker?" More private data should be subjected to even closer ethical reasoning. Overall, I was sensitive to the ethical and political issues that might arise during my study, and I took necessary steps to prevent these issues. During the research I maintained the highest level of respect for the participants of the study. My study aimed to have faculty speak about their Elementary Teacher Education Tuning experiences in a way that would lead to change and could benefit future faculty. My concern was for faculty members, both those directly involved in the Tuning project, and those whose future may be impacted by the results of the research. I maintained this view throughout the research process and into the dissertation writing, and dissemination of the findings. I believe, this study provides valuable insights into the purposive, creative and knowledgeable work of the faculty and leadership engaged in Tuning. Conducting the research, I did everything possible to minimize any potential harm to the participants of the Utah ETE Tuning case study. During the study I was attentive to the participant-participant relationship, and kept in mind the researcher-participants relationship, which was determined by roles, status, and cultural norms. My overall intention was to maximize benefits for any individual participant of the case study, and minimize risk of harm to the individual. Limitations This section explicitly acknowledges the limitations of the study and how they were addressed. Limitations of the study expose the conditions that may weaken the study (Locke, Spirduso, & Silverman, 2000; Rossman & Rallis, 2011). I explored the role of faculty in developing student learning outcomes and competencies for Elementary 90 Teacher Education in Utah colleges and universities. Framing the specific study of "ETE Tuning in Utah" limits this research, including the number of faculty interviews, and meetings' observations that were conducted, and a number of documentary materials that were critically examined. Additionally, voluntary participation in the study also adds response bias to the research. The research sample was restricted to the Utah ETE Tuning team members, so the results may be critiqued because of the limited possibility to generalize this study to other Tuning experiences, but replication in qualitative study is not critical (Marshall & Rossman, 1999). The issue of transferability has been addressed in the section about trustworthiness of the research. The study of Utah ETE Tuning is situated within a specific context, and the reader can make decisions about its usefulness for other settings. Thick and rich description, and detailed information about the context and background of the case study was considered for their applicability in other contexts. The researcher-as-an instrument approach shaped in my values, experiences, and preconceptions also influenced the direction of the research and its findings and conclusion. Identifying these limitations, I took the following actions. First, I acknowledged the research agenda and stated my assumptions up front. Coding schemes of the interviews, observations, documents were examined by the research adviser and through peer review. Additionally, I removed all participants' names from the transcripts and coded blindly so as not to associate any data with any individual. I made a conscious actions to create an honest environment for an open dialogues and conversation. 91 Conclusion This chapter has presented the overview of the research methods that were utilized in the case under study. It has addressed all necessary topics of the methodology chapter recapping and highlighting its main points. The chapter has also offered how the design of the study has been chosen through a convincing rationale for the qualitative case study. This guided the inquiry to answer the research questions and chosen research paradigm, and appropriate methodological literature. The chapter has presented the researcher's positionality and discussed the concerns connected with a researcher as the instrument issues and the researcher's biases for a qualitative inquiry. The chapter has described data collection methods including the rationale for selecting the research site, the participants, and instruments congruent with the research problem that was investigated. Data analysis and synthesis have been described including relevant figures. The chapter has also discussed ethical and political considerations and issues of trustworthiness of this single-case study, and analytical framework utilized for this study to address the lack on faculty research, and articulated the limitations pertaining the study and the ways I addressed them. 92 Figure 1. Key Elements of Tuning Methodology. Tuning USA - Bigger Context Utah Tuning - Context Case - Utah ETE Tuning Figure 2. Utah ETE Tuning Holistic Single-Unit Case Study. 93 Convergence of Evidence (single case study) Individual and focus open-ended interviews Observation (direct and participant) Documents Findings Field notes Figure 3. Convergence of Evidence. Self-reflective journal Memos CHAPTER 4 FINDINGS Introduction This chapter presents the findings of the qualitative research conducted about Tuning Elementary Teacher Education in Utah System of Higher Education. The case under study was focused on the ETE faculty's work aimed to develop student learning outcomes and demonstrable competencies for the Elementary Teacher Education college discipline at the bachelor's and associate level. The following questions guided this research: 1. How have faculty been engaged in the Utah Tuning project? 2. How has the Tuning project influenced Elementary Teacher Education in Utah? 3. a. Who provides the leadership direction for tuning Elementary Teacher Education in Utah? b. What factors have been used to advance the Utah Tuning project? The major findings that emerged from this study: 1. The Utah ETE Tuning process was a developmental, dynamic, complex, complicated, and iterative process. 2. Collaboration was tenacious and tenuous among many different participants 95 within the process. 3. Tuning required interrogating faculty practice. The ETE Tuning work was learning driven. The faculty were learning how to work across the higher education institutions to tune their discipline. The key findings were obtained from different data sources, including16 in-depth individual interviews as well as group and focus group interviews conducted with the ETE Tuning team members and other ETE faculty who were not part of the ETE Tuning project but worked in ETE departments of the Utah colleges and universities. The other sources of data were the appropriate documents, including meeting notes of the ETE Tuning team, observation notes, and the Utah Tuning interim, evaluation, and final reports (For a full list of the documents including Utah Tuning reports, ETE Tuning meeting notes see Appendix I, J, and K). This chapter consists of three main sections. First, I present data on the ETE Tuning as a developmental, dynamic, complex, complicated, and iterative, process. The ETE Tuning process included nonlinear phases such as learning about Tuning and how to tune, defining the ETE discipline and degree specificity, developing student learning outcomes and competencies, and facing and overcoming challenges along the way. Second, I discuss tenacious and tenuous collaboration among many different participants within the process. Relying on their professional expertise, the ETE Tuning team members were contributory into the process and critical for the process. There were also many other participants involved in the process, including the state Tuning Leadership Team, the USOE, ETE department colleagues, ETE department chairs, deans, and provosts. Finally, I address the findings of how Tuning created mechanism of discipline 96 collaboration and educational quality. The chapter addresses how faculty were engaged in the Utah Tuning project, who provided the leadership direction for Tuning ETE in Utah, how the Tuning project influenced Elementary Teacher Education in Utah, and what factors were used to advance this project through the ETE Tuning team members' perception of what they did and how they were Tuning their discipline. Following is a presentation of the findings with details that support and explain each finding and document a broad range of tuners' experiences. Developmental, Dynamic, Complex, Complicated, Iterative Process Originally, the Tuning project in Utah began in two college disciplines (Physics and History) with a grant support from the Lumina Foundation for Education in April 2009. The results of the project were reported as successful in the Utah Tuning interim and final reports (USHE, 2009, 2011, 2012, 2013). Consequently, after the first Tuning initiative, the decision was made to add two more college disciplines - Elementary Teacher Education and General Education Mathematics - to the state Tuning 2 Project. Therefore, the Utah Elementary Teacher Education Tuning project started in 2011. At that time, the Utah System of Higher Education got an additional grant from the Lumina Foundation and could expand its Tuning work until May 2013 (USHE, 2011). The main goal to include these two disciplines was "to coordinate with the Common Core State Standards and their promise to better prepare high school graduates" (USHE, 2014, p. 1) to enter the higher education system. The Utah Elementary Teacher Education Tuning project assembled representatives who were faculty members from the elementary teacher education 97 departments of all eight public institutions: University of Utah (Salt Lake City), Utah State University (Logan), Weber State University (Ogden), Southern Utah University (Cedar City), Snow College (Ephraim), USU - Eastern (Price), Utah Valley University (Orem), and Salt Lake Community College (Salt Lake City). Additionally, all three private institutions - Brigham Young University (Provo), Westminster College (Salt Lake City), and Western Governors University (Salt Lake City) - joined the Tuning work for the Elementary Teacher Education discipline. In addition, the representatives of the Utah State Office of Education (USOE) participated in the ETE Tuning teamwork. In total, the ETE Tuning team consisted of 15 members including 12 faculty, two representatives of the USOE, and one student (http://utahtuning.weebly.com/elementaryeducation-team-members.html). The state Tuning leadership team was mainly represented by Dr. Phyllis "Teddi" Safman, Assistant Commissioner for Academic Affairs, Utah Board of Regents, Dr. William Evenson, consultant, Dr. Norman Jones, consultant, and Dr. Daniel McInerney, consultant (http://utahtuning.weebly.com/contactus.html). Additionally, two external Tuning evaluators, Dr. Randy Davies and Dr. David Williams, were employed throughout the process. Their functions were to provide formative assistance and developmental evaluation consultations to the Utah Tuning leadership team, and advise the discipline Tuning team members on changes and improvements to their Tuning process (Davis & Williams, 2012). The Tuning process for the ETE Tuning team began with orientation in the fall of 2011 and concluded its most intensive phase in the spring of 2013. The ETE Tuning team intentionally worked at developing student learning outcomes and demonstrable competencies for the ETE college discipline during two years and completed its work 98 with the Utah Preservice Teacher Learning Outcomes (UPTLO) document. As a result of the cross-state intentional work to define learning outcomes for postsecondary degrees (http://utahtuning.weebly.com/utah-tuning-reports.html) and create "reference points" for students moving ahead through the college levels, Tuning was defined as "a messy, nonlinear, and iterative process that must be practiced, rethought, muddled through, and adapted to its various contexts; its practitioners must be open to new ways of thinking about teaching and learning in order to sustain Tuning" (USHE, 2014, p. 1). The process was developmental, dynamic, complex, complicated, and iterative. Conceptually, Tuning requires that each student knows, understands and is able to demonstrate required skills in order to get a college degree. Therefore, Tuning represents a paradigm shift in higher education from the status quo of teaching to student learning. The shift is from evaluating students based on content covered to evaluating students based on learning and mastering skills, and it is simultaneously profound, difficult to grasp, and difficult to articulate. As such, the ETE Tuning initiative required each of the tuners and the team as a whole to realign their thought processes to accommodate the change. Because Tuning was a new concept to the ETE Tuning team, from the very beginning it was a learning-driven process for the participants. Figure 4 (p. 182) visualizes the Tuning process as a developmental, dynamic, complex, complicated, and iterative process. The first developmental step for the ETE team was to learn about Tuning and how to tune. This process consumed some five months from September 2011 until April 2012 when the team had a breakthrough moment in comprehending Tuning. This was not an end to the developmental process, but it did represent a launching point 99 from which the team started to tackle the actual work of Tuning. However, each phase of the process was developmental in itself, as the faculty had to decide which skills were critical for preservice teachers to master in a manner that was discipline and degree specific. This required interrogating faculty practice to find new ways to bring improvements to the student learning and evaluation processes. The faculty had to develop and articulate student learning outcomes and demonstrable competencies, and they had to decide how to describe the entire package in a way that could be transparent and transferrable. The ETE Tuning process was also very dynamic as the ETE faculty brought their individual viewpoints to the table for discussion. Although the process was collegial and displayed a high degree of collaboration, it was never smooth. The deliberations ranged from substantive content issues of what students need to know, understand, and be able to demonstrate to discussions about how the Tuning results should be presented in order to be generally and easily understood. Hence, Tuning was full of changes connected with the faculty's evolving thought processes and conclusions. The ETE Tuning process was also complex and complicated due to the many interested parties such as ETE faculty, their department colleagues, students, department chairs, and university and college administrators. In addition, the team faced influences from outside higher education. Since elementary teacher education is closely related to elementary education, it is also subject to interest from parents, state governments, the federal government, school districts, school principals and the Utah State Office of Education. These groups are highly interested in public education and, therefore, have an indirect interest in teacher preparation programs. The deliberations of the Tuning team 100 were affected by the internal debates and outside inputs as the diverse interests could not all be accommodated. As an example, concern for the Utah State Office of Education role in accepting, or not, the final product for use in evaluating preservice teachers caused the Tuning team to adopt the Utah Effective Teaching Standards as a starting point for articulating student learning outcomes and competencies for preservice teachers. Faculty collaboration and collegiality was crucial in helping the Tuning team work through both the discipline issues and the need to accommodate outside influences. Overall, the ETE Tuning process was very nonlinear. The ETE Tuning purposeful work at what student learning outcomes and competencies had to be developed and articulated and how they could be made transparent, measurable, and assessable resulted in many iterations at all steps in the process. The Tuning team had to learn how to tune their college discipline. The work progressed slowly as conclusions were proposed and then examined again and challenged, and often swept aside in favor of new decisions as the tuners exchanged professional experience and considered outside requirements. This was a pattern that would repeat itself throughout the Tuning process. Figure 5 (p. 183) displays the key events and activities throughout the ETE Tuning process. The process consisted of three related activity tracks: the ETE Tuning work, the internal evaluation reporting process, and the grantee reporting process. The ETE Tuning work was the most complex of the three tracks and included activities by the ETE Tuning team, and the state Tuning leadership team to further the Tuning process. The period from September 2011 to April 2012 was a period of intense learning for the Tuning team that resulted in a very important "ah-ha" moment in April 2012 when the team got a 101 grasp on the essence of Tuning and how it differed from existing student assessment processes. This enabled the Tuning team to proceed over the coming months to work at articulating ETE student learning outcomes and competencies until the end of the ETE Tuning project in May 2013. Throughout the Tuning work process there was a high degree of deliberations and collaboration, largely among the ETE Tuning team, but also including the state Tuning leadership team. As the Tuning project was coming to an end, the state Tuning leadership team met with Utah universities and college administrations and institutional Tuning teams in the period of May 2013 till May 2014 to plan for continuing Tuning college disciplines after the Lumina grant funds were exhausted. As shown on Figure 5, the ETE Tuning work was evaluated and presented in the two May 2012 and May 2013 interim evaluation reports, and June 2014 final evaluation report (Davis & Williams, 2012, 2013, 2014). The Utah Tuning work was also reported in the two grantee reports and a final narrative report (USHE, 2012, 2013, 2014). The following three subsections present data on how the ETE faculty were learning to tune, determining the ETE discipline and degree specificity, developing student learning outcomes and demonstrable competencies, and overcoming challenges along their way. Learning about Tuning and How to Tune Tuning, being a new concept for most of the participants, with attendant new vocabulary and unfamiliar expectations along with an absence of skilled practitioners to guide the team past roadblocks, meant that the team and individual participants were obligated to develop an understanding of the process without benefit of structured guidance. The practical aspects of Tuning were difficult for the faculty tuners to 102 internalize. The ETE Tuning project began with an orientation. Faculty members who were the representatives of the ETE programs from all Utah colleges and universities were brought together for their training prior to their regular joint meetings. The goal of the orientation was to introduce faculty to the essence of Tuning and how to go through the process based on Tuning Educational Structures in European countries and on the Tuning USA projects started earlier in other states (e.g., Indiana, Kentucky, Minnesota). Since each Tuning case had its own peculiarities because of a state context and specific state education policies, the ETE team members needed some time to really get into the process. During the orientation, the state Tuning leadership team - Dr. Norman Jones, Dr. Daniel McInerney, Dr. William Evenson, and Dr. Phyllis Safman - actively worked to launch the ETE Tuning project and to introduce, and engage faculty in Tuning. This was being done through creating the foundations of faculty's awareness in Tuning. The Lumina Foundation provided the information materials about the essence of this process. The ETE team viewed the Lumina video on Tuning (http://www.luminafoundation.org/newsroom/topics/tuning:adventures-in-learning.html) to obtain a general understanding of the Tuning process and work. The faculty were introduced to the Bologna Process, which gave impetus for higher education in Europe to demonstrate its value to students, faculty, parents, employers, state educational agencies, and the broader educational community, and in the long run went worldwide through Tuning higher education. As one of the participants recalled, faculty "got the overview and … understood that historically it [Tuning] came from the Bologna process." (Faculty_individual interview_4) As noted in the meeting of the state Tuning leadership 103 team with the University of Utah institutional team, the Tuning project in Europe was developing: Transparency across different countries and ways to transfer across Europe. It was based on mobility of people, so Europe could be seen as one higher education zone. In Utah, we should be able to move seamlessly across schools or change majors without having to take extra courses. (USHE, 2013, p. 126) This quote reveals that Tuning, focusing on student learning with specific outcomes and expectations for students, educators, and employers, develops transparency across higher education institutions, which, in turn, benefits college students. The Tuning initiative as part of the Bologna process aims to identify the core knowledge, skills, and competencies, associated with college degree-level, that is, to clearly identify what exactly a college degree means in relation to the needs and demands of the workforce and society. The understanding of the Tuning historical roots served as an important foundation for faculty to comprehend the overall goals of the Tuning project. Next, the Utah Tuning leadership team provided a review of Tuning to bring the participants to familiarize with the structure, goals, and status of the project. It was explained to the faculty that Tuning did not require standardization for every college to have the same curriculum, pedagogy, and assessment. Individual institutions are free to develop their own curriculum and pedagogy as well as their own assessments, which provides freedom to work within the context of each institution and provides for development of alternate pathways to the same outcomes. Students can then fully understand what they must achieve prior to being granted a degree or transferring to another school. Licensing agencies, accreditors, employers, and others could be informed on what students have learned and the assessment process for determining their proficiency. The report (USHE, 2012) described Tuning as not a destination but a 104 process that required continual updating and improvement for college discipline. The ETE team members were active listeners trying to get into the process and learn about Tuning. Overall, the ETE team was aimed to actively work across the discipline - to tune it - "to make degree outcomes consistent across the state" [for the employers to] "be assured of what a graduate with a degree in the major knows, understands, and is able to do" (USHE, 2011, October 21, p. 2). One of the state Tuning leaders recalled that the ETE faculty "understood that they were going to identify learning outcomes and competencies" (State leader_individual interview_3) for a bachelor's and associate degree level. After the orientation, the state Tuning leaders were bringing faculty "together once a month, with email exchanges in the interim to work on" (State leader_individual interview_3) developing student learning outcomes for elementary teacher education college discipline. The first ETE Tuning working meeting was held in October 2011. The meeting notes (USHE, 2011, October 21) showed the faculty members came to work in the team with certain expectations about their personal roles and expectations for their departments from the Tuning project. The ETE faculty expected "to be able to contribute to the conversation, facilitate the conversation within their individual departments, learn about other institutions and how they all compare," and they were "seeking alignment, not conformity" (USHE, 2011, October 21, p. 2). As a team, the ETE tuners anticipated understanding what the school districts wanted, and to tune the discipline that the fouryear college programs use to prepare students to enter the workforce. For two-year colleges, the ETE tuners anticipated to tune the discipline that so that attending a twoyear college would prepare the students to transfer to the four-year institutions (USHE, 105 2011, October 21). The faculty also expected to work collaboratively with colleagues throughout the state. Additionally, two external Tuning evaluators, Dr. Davies and Dr. Williams, joined the ETE team (USHE, 2011, October 21). Their goal was to evaluate the whole Tuning process and to help the faculty accomplish their Tuning goals. Accordingly, during the project time the evaluators constantly shared the information with the ETE team about the Tuning work from other state teams who were working to tune the other three college disciplines (History, Physics, and General Education Mathematics). The ETE Tuning work began with very general discussions about the goals of the ETE programs, faculty expectations, requirements for the programs and many other discipline specific aspects. The faculty defined the overall goal of the ETE programs as "to graduate someone whom we would want to have as a teacher for our children or grandchildren" (USHE, 2011, October 21, p. 2). One state Tuning leader designated the beginning of the Tuning process as faculty "started out by talking about their own programs, what they were doing, and what was important to them, and trying to find commonalities between themselves." (State leader_individual interview_5) Another faculty member described the beginning of their ETE Tuning work as: We began early on, first, as representatives, talking about what we felt should be the competencies of our preservice teachers, when they left our program … At first, it was just kind of finding the common ground among all of the faculty. (Faculty_individual interview_11) At the first Tuning meeting the faculty also discussed what the bases for outcomes should be. The meeting notes demonstrated the deliberations were about: "What are the needs of the students? What are the needs of the employers? What are the needs of the children in the schools?" (USHE, 2011, October 21, p. 2). Figure 6 (p. 184) illustrates the bases for the ETE student learning outcomes as articulated by the Tuning team in their first 106 meeting. Since the purpose of the ETE Tuning process was to develop ETE student learning outcomes, the Tuning team discussed the foundations that they believed should ground the student learning outcomes. The team sorted out three foundational elements that were of paramount importance in determining student learning outcomes. For the bases of the ETE student learning outcomes, the faculty prioritized the needs of the ETE students themselves, the needs of children in elementary students, and the needs of employers. The dashed lines used in this scheme show that these three bases are interconnected and interrelated with each other. These bases, arrived at during the initial deliberations, provided the touchstone for all decisions made throughout the process. The team members were looking for essential outcomes for preservice teachers. It was hard complex work of developing a common understanding of the Tuning process within the team and articulating student learning outcomes and competencies. There was some confusion at the beginning. One faculty member (a tuner) described her muddled understanding of Tuning at the beginning of the team work: I [faculty member] remember coming back to my office saying ‘I am not really exactly sure what we were doing,' and, I think, not everybody was sure what we were doing. And mostly it was, you know, you need to look at the Degree Qualification Profile, and you need to use action language, and I am like, we were not actually sure what our action is. (Faculty_individual interview_1) Another recall of confusion also emphasized that getting into the process took some time, when "I [ETE Tuning team member] first went into the process, I didn't have any idea what it was all about. It took me a little bit of time to get up to speed" (Faculty_individual interview_5). Faculty were "floundering at that point" (Faculty_individual interview_1). They needed time to figure out what exactly they 107 should be doing. They needed "to go through things and get all the information, to get enough data to know which direction we [tuners] should be going" (Faculty_group interview_1). Early in the process, tuners also expressed confusion over understanding the major differences between competencies and learning outcomes. The confusion included a lack of agreement about the meaning of terms, especially "competencies," and how student learning outcomes and competencies could relate to each other. The tuners agreed that competencies and student learning outcomes functioned together as genusspecies relations or relations of inclusiveness. The ETE profile provided contextualization for student learning. Competencies were defined as statements that draw from the ETE core concepts and describe the levels of learning within ETE as a college degree. They should represent categories of knowledge and learning within ETE and ratcheted up consistent with bachelor's and associate degree. Learning outcomes were defined as statements that draw from the ETE core concepts that describe the student response to learning within the ETE as a college degree. They should represent the requirements of the ETE programs and denote assessable demonstration of learning which should show the proficiency of ETE students' mastered sets of knowledge and processes that compose a given competency. A set of ETE competency statements defined a level of learning at bachelor's and associate degree level, and supported the development of student learning outcomes statements, which were articulated under the umbrella categories of competencies. The ongoing need to agree on the meaning of terms challenged the ETE Tuning work and illustrated the nonlinear nature of the process. During several meetings tuners seriously "discussed what it meant to be a good teacher, and what you need to train 108 yourself to be an elementary teacher" (Faculty_individual interview_2). They were deliberating the essence of the ETE, and their intensive and extensive discussions supported the development of student learning outcomes and ensured a common understanding among tuners. While deliberating the essence of the ETE, they were building a common ground for at least 2-year Tuning project. Practically, the faculty were cognizing the Tuning process and how to tune elementary teacher education college discipline. As experts, faculty tuners had a lot of shared and agreed upon elements. At some point of their Tuning work, they began producing the documents that quantified student learning outcomes, that people could rate their programs and students against them. One Tuning team member recalled the importance of having a common understanding of all discipline notions: When we first began Tuning, we started brainstorming. What is it? What were the competencies that we thought future candidates needed to know to be objective beginning preservice teachers? We brainstormed and we developed a list of what we were looking for essential outcomes for preservice teachers. It sounds easy, but it was hard and we worked at this to make sure that we knew exactly what it was we were talking [about]. If we said a word, everybody agreed that that's what it meant. (Faculty_individual interview_10) The tuners were immersed to a point where they were able to have a substantive discussion of content, and simultaneously resolve the sometimes subtle and not-so-subtle differences in understanding of the language being used. They eventually found a common understanding among them. Learning about Tuning and how to tune was essential to completing the task of developing student learning outcomes with a measure of success. The following section illustrates how tuners were determining the specificity of the ETE discipline for bachelor's and associate degrees. 109 Defining Discipline and Degree Specificity This section presents the findings that the Utah ETE Tuning process was both discipline (Elementary Teacher Education) and degree (bachelor's and associate) specific by its definition. The October 2011 meeting stated the main foci of the of the Utah ETE Tuning work. Figure 7 (p. 185) "Tuning is Discipline and Degree Specific" visually presents the main foci of the process. They were determined to clearly articulate student leaning outcomes and competencies within the Elementary Teacher Education discipline, align all ETE programs across the state, and improve the alignment of students' mastery and its relevance to Elementary Education as a professional field (USHE, 2011, October 21). The figure visualizes that all eight Utah public colleges and universities: U of U (University of Utah), USU (Utah State University), WSU (Weber State University), SUU (Southern Utah University), Snow College, UVU (Utah Valley University), DSU (Dixie State University), and SLCC (Salt Lake Community College) and three private ones: BYU (Brigham Young University), Westminster College, and WGU (Western Governors University) contributed to this process. In order to articulate student learning outcomes and competencies of the ETE discipline for bachelor's and associate's levels, the ETE tuners had to take into account all the requirements and peculiarities of the ETE discipline as a college degree in different colleges and universities in Utah, and distinctiveness of an elementary education as a professional field directly connected with a college teaching discipline. The ETE Tuning team had to face an issue of elementary education being a subject to many standards: national standards, statewide standards, regional standards. The ETE tuners "had to work their way through" (State leader_individual interview_3) 110 many standards. Elementary education "is tied to all kinds of standards," so tuners had to take into account "what was already out there" (Faculty_individual interview_11). One state Tuning leader emphasized that "they [elementary education programs] are just battered with standards" (State leader_individual interview_3). Another faculty added, "every discipline within us [elementary education programs] has a set of standards, there are national science teaching standards, there are social studies standards" (Faculty_individual interview_4). In order to define the ETE specificity at the bachelor's and associate degree level, the faculty tuners collected relevant standards, including InTASC (Interstate Teacher Assessment and Support Consortium), Utah Effective Teaching Standards, and the NCATE (National Council for Accreditation of Teacher Education) report on Clinical Preparation. They brought these guiding documents to the table, which was very helpful to start their discussions from the commonalities. The team members composed standards and department assessments that they sent electronically and studied them as "homework" before the following meeting; they also collected requirement sheets for elementary education from each institution. The faculty had homework after every meeting; they were learning to tune their college discipline. Defining the discipline and degree specificity, the ETE team members considered many questions. For example, what does ETE focuses on? What issues does ETE address? What does "doing ETE involve?" What approaches does ETE employ? What parts of the ETE are delivered in bachelor's degree? What parts of the ETE are delivered in associate degree? What does learning during the ETE college program enable a student to do? What does teaching in the ETE programs enable a student to do? Asking these questions, faculty tuners were also considering how ETE changed in 10-15 years 111 because of the discipline external requirements and internal development in response to research findings. Through their Tuning participation, they accumulated professional knowledge shared by all tuners as the representatives from different ETE college programs. Beginning with the third meeting (February, 2012), every Tuning meeting started with reviewing of the previous work in order to brush up and sharpen emerging drafts of student learning outcomes and competencies. It took almost five months for faculty to really understand what Tuning was about. One state leader emphasized the necessity of time for leaning about Tuning and how to tune a college discipline. She said: "It was April [2012, ETE Tuning started in September 2011], when there was an ‘aha' moment for everybody… Oh, that's what they're doing, that's what this means. And that was extremely important" (State leader_individual interview_3). After going through the national standards and accreditation requirements, tuners came to work at the Utah Effective Teaching Standards, an official state document. However, "the Utah Effective Teaching Standards were for practicing teachers, and our job [tuners'] was to look at preservice teachers and decide what they [student graduates - preservice teachers] could do and not have it to be very overwhelming" (Faculty_individual interview_1). The Grantee final narrative report to Lumina Foundation (USHE, 2014) also stated that "the UETS standards are too broad and were developed for practicing teachers" (p. 3). Accordingly and to get reconciled with the mission of the Tuning process, the ETE Tuning team was working: To make it [UETS] more palatable for preservice teachers, make it more accessible for them because it seems very overwhelming, they [standards] are 49 indicators, and that's an awful lot when you've been in the classroom for roughly six to eight weeks. (Faculty_individual interview_1) This quote exposes that the team intentionally worked to develop and articulate the 112 students learning outcomes and competencies for preservice-teachers that could be assessed based on the UETS. For example, at the February 2012 meeting after reviewing the previous Tuning meeting minutes, the team carefully went through the Utah Effective Teaching Standards, and made sure that the essential learning outcomes that they had drafted so far were aligned with the UETS. The meeting notes showed when the tuners found gaps in their essential learning outcomes that did not reflect what they found in the UETS, they addressed the gap by either revising an existing outcome or drafting a new one. Finally, they decided to follow the UETS format and organize their essential learning outcomes into three superordinate categories: The Learner and Learning, Instructional Practice, and Professional Responsibility (USHE, 2012, February 10). At the October Tuning meeting, a member of the ETE team personalized the goal for ETE college programs as needing to prepare teachers who would meet the expectations she would have for her own children's and grandchildren's teachers. Faculty stated that graduates should be ready to teach when they leave the program. ETE graduates or preservice teachers going to teach in elementary school must have many skills and competencies to be able to teach elementary school children, work with parents, with other teachers, school principals, and broader educational community. One of the ETE team members emphasized the importance for ETE graduates to have a good solid knowledge base: In elementary ed. you are a jack-of-all-trades, you need to know something about every subject. It's not like secondary where you've known something about every subject but your real base of knowledge is in math or history, or science. So, in elementary we are prepping all of the students. Our teacher candidates are prepping their elementary students, so that they can go on and function in secondary education, and they can't do that unless they have a real good solid knowledge base. (Faculty_individual interview_1) 113 Explicit in this reflection is how hard and important an elementary teaching job is. The ETE faculty realized their responsibly to prepare all students for this difficult and highly important job. Implicit in this quote that ETE college programs are directly connected with the quality of teaching in elementary schools. A range of skills that tuners prioritized for the ETE graduates overall embraced classroom management, ability to feel comfortable and confident in front of a class, ability to use technology, approach to assessment, professionalism, behaviors, how to carry yourself, instructional design, and their ability to engage students. The following series of quotes illustrate the range of skills that Tuners considered important for preservice teachers just coming out of college. Some tuners believed that preservice teachers coming out of the universities had to be good at instructional strategies and content knowledge. For example, Instructional strategies and content knowledge are the big two because those two - you can't really teach unless you know some techniques for teaching, and you can't really teach unless you know the subject matter. Because elementary ed. is not like secondary ed. (Faculty_individual interview_1) Other tuners held that ETE graduates must have a good idea of teaching methodologies and different ways of teaching, student engagement and ways of differentiating with their students, and time on task. They do have to have ways, and they have to develop philosophies of how they're going to manage a classroom. Even if those change 10 minutes into their teaching, that they realize that they can't just be everybody's buddy. They have to take on that role of the facilitator/coach/mom/dad, whatever it is they have to take on. (Faculty_individual interview_5) Still other Tuners talked about making certain that preservice teachers should be able to work with teachers who could try to beat their university experience out of them. One faculty member stressed: 114 I really think that the most important thing that they [ETE graduates] have to have is, a good grounding in instructional philosophy. Not just philosophy, but a good practical grounding in what actually works in classrooms. What is research based? What do we know actually works? I think that's really important, and we talked about it in the Tuning process. When they [ETE graduates] leave the university and go out for their student teaching experience, they're going to have teachers who are going to try to explain to them, that what they've learned in school, is not what actually happens in classes. If they've had enough experience and had enough grounding and enough background in instructional methodologies, that are actually evidence based, then they [ETE graduates] can ride out that storm. (Faculty_individual interview_5) The meeting notes also stated that a prepared elementary education graduate "should be able to examine assessment data on a student and decide if he/she received appropriate instruction and adjust instruction accordingly" (USHE, 2011, October 21, p. 3). The unit of analysis for Tuning is at the discipline or program and degree level not the course level. The ETE tuners were specifically focusing on what ETE students would be able to do as a result of their discipline and degree learning. One of the state Tuning leaders emphasized: Tuning is a degree specific. Somebody who has done it can say: ‘oh, yeah, I know. I know our students understand what this is, they can articulate it, they know how they were assessed, and how to demonstrate it. They now understand and are able to do.' (State leader_individual interview_3) Defining the discipline and degree specificity through articulation of learning outcomes potentially benefit students because students could know in advance what actions and how they would be expected and required to demonstrate. Having accumulated the necessary understanding and knowledge, tuners "settled on partway through the process" (Faculty_individual interview_11) what they could do, and how to articulate the competencies for preservice teachers, and began working on the outcome document. The following section describes how the ETE Tuning team was working at articulating the student learning outcomes and competencies. 115 Developing Student Learning Outcomes and Competencies This subsection presents the findings on how the ETE faculty, members of the Utah ETE Tuning team, were literally writing student learning outcomes and competencies. The following findings emphasize what exactly were the faculty engaged in, and what exactly the faculty were doing in the Tuning project. This subsection aims to expose step-by-step how the ETE Tuning team was going through the writing process, what was inside the practical Tuning work, what important aspects of Tuning the ETE faculty had to take into account and articulate through their work. Tuners met monthly and discussed their discipline, and as one tuner said "talking was really helpful" (Faculty_group interview_4) because it guided the team through the initial learning about Tuning work and created foundations for the ETE Tuning. The team members were brainstorming what competencies college graduates needed to have and demonstrate to become objective beginning preservice teachers. For example, at their December 2012 meeting, the ETE Tuning team was engaged in a specific exercise aimed to generate the 5 top student learning outcomes. Additionally, the tuners generated some possible modes of assessment for some learning outcomes. They discussed if certain learning outcomes could be assessable and how to assess these outcomes. After deliberations they agreed that constructed responses to a text-based or video-based prompt, teacher work samples with collected information on students' ability to critically think and reflect, and videotaping themselves and reflecting on the experience could be among others valid forms of assessment. For instance, Figure 8 (p. 185) displays an example of the tuners' whiteboard exercise with their brainstormed ideas from the December 2012 Tuning meeting minutes. The initial list of the generated learning 116 outcomes with some possible modes of assessment, presented in the December 2011 meeting minutes, included: 1) Critical observer and thinker, with the ability to express it and make changes as a consequence a) Self-reflection using a prompt b) Observation of students' behavior 2) Ability to assess student learning and provide appropriate instruction in all subjects; employ interventions to differentiate for all learners (advanced, learners with IEP, struggling) a) Lesson plan with student work and the assessment that was used to determine what the students learned b) Teacher work sample 3) Integrate across content using the Common Core and critically determine appropriate content to teach in terms of accuracy and the given curriculum a) Teacher work sample b) Unit plans c) Portfolios with artifacts 4) Know who the students are teaching in terms of their class, culture, language, cognition and care about them as learners and human beings a) Contextual factors in teacher work sample, accommodations b) Lesson plans with cultural references from students' cultures c) Observations by cooperating teachers and supervisors 5) Establish classroom environment based on caring, responsibility, respect; effective management strategies to maximize learning time and student engagement. (USHE, 2011, December, p. 2) From this list we can see how the ETE Tuning work, or the faculty's intentional work at developing student learning outcomes and competencies started to shape. The initial list was essentially a draft of objectives. This list showed that the initially brainstormed outcomes were mostly articulated in nouns "observer," "thinker," "ability," and so forth, and only some verbs "know," establish." However, Tuning requires articulating student learning outcomes and competencies in measurable and assessable verb terms. The tuners were weighing and examining how to articulate learning outcomes taking into consideration discipline and degree specificity. Student learning outcomes must be action-oriented, and serve as grounds for 117 student actions and performance as the evidence, which would demonstrate that learning occurred. One tuner reflected upon how they were generating a list of student learning outcomes and competencies, and how the topic areas were being shaped: It was a brainstorming session. There was one person taking notes and writing ideas and connecting like a semantic map where ideas were connected with lines and other circles drawn. It was really interesting how we were all able to share and contribute to that process and then slowly shape it into the main topic areas. That was really productive. (Faculty_individual interview_7) The use of brainstorming, developing graphical representations, writing different schemes, visualizing ideas, rephrasing, fine-tuning, and other techniques is illustrative of Tuning teamwork. The Tuning work was functioning with a common understanding of direction and goals and was moving the process forward. This was the kind of intensive professional deliberation that was expected to produce a successful outcome. At the end of a December 2011 meeting discussion, the ETE team expressed their concern about how they did not want their emerging learning outcomes "to be seen as just another set of standards" (USHE, 2011, December, p. 3). The team deliberated how the emerging learning outcomes fit in the Common Core standards and other standards. The meeting notes exposed when they found out the gaps, they continued their deliberations. They voiced their concern that educators felt inundated with standards. The team also asked the representative of the USOE to share the Tuning process with the USOE to make them aware of the Tuning work and how students learning outcomes and competencies for preservice teachers might be integrated with the newly adopted Utah Effective Teaching Standards (USHE, 2011, December, p. 2). To develop student learning outcomes and demonstrable competencies, the ETE Tuning team debated about teaching preparation perspectives in different Utah colleges 118 and universities. The private institutions saw things differently. For example, the Western Governor's University, which is an online, competency-based program, was a very interesting perspective to receive. Sometimes tuners had opposing viewpoints. One of the team members suggested that they should determine what could be assessed, and this, in turn, could determine what the competencies could be. However, one of the Tuning state leaders stood strongly against this idea: We are not doing that. We don't determine what can be assessed then determine the competencies. We determine the competencies and then have the strategies to be able to assess if the students have learned those competencies and can demonstrate them. (State leader_individual interview_3) Here we see many layers, for example, how tuners were trying to find the way through their Tuning work building common professional reference points and leadership role in Tuning. Since the ETE college programs have to work under many requirements and standards and the Utah Effective Teaching Standards are among them, the ETE Tuning team continued their work on student learning outcomes and competence development very closely to the UETS. One team member recalled: "We had an idea, it doesn't matter what we think essential preservice teacher outcomes. The state ultimately decides what the state believes that beginning teachers' outcomes need to be" (Faculty_individual interview_10). The ETE Tuning team leader described the team decision: We started out thinking: "Do we want to do the Utah Effective Teaching Standards or do we want to reinvent the wheel?" My thought as the chair of the committee was: "Well, we can reinvent the wheel, all we want, but we are held to the Utah Effective Teaching Standards. We can spin our wheels quite a bit here, but we are going to end up doing this because no one is going to use it if it's not approved by the state, by the office of ed." (Faculty_individual interview_1) These quotes demonstrate that the team decision was primarily based on powerful 119 external drivers. First, the team took into account the state requirements for elementary education. Second, the USOE licenses teachers, and, as a result, it is a very important and powerful stakeholder in the ETE field. Ultimately, the tuners agreed not to reinvent the wheel, they decided "to take something what was already approved by the legislature and the USOE department, and use that as the template" (Faculty_individual interview_1). The Tuning team met nearly every month for two years and continued its Tuning work during the second year of the project to connect their student learning outcomes with the Utah Effective Teaching Standards. This was a logical move since each college and university education department prepares students for the rigors of teaching in contemporary classrooms. Members of the team articulated the student learning outcomes to bring them in line with USOE elementary education program coursework. Having settled on how to incorporate the UETS into the process, the tuners drove to make the UETS more palatable for preservice teachers, make them more accessible for them because the UETS could be overwhelming, especially for new teachers. One of the Tuning team members pointed out that the large number of standards could overwhelm a pre service teacher with limited classroom time. The ETE team decided: To take the Effective Teaching Standards and to determine what would a beginning teacher look like. What would we [educators] expect a student just coming out of college? Where would we [educators] expect them to be on those Effective Teaching Standards? Which ones were appropriate for a beginning teacher, and which ones were probably something that would take some time to develop some expertise? (Faculty_individual interview_5) Meshing the expectations for beginning teachers with the UETS required that the tuners systematically, comprehensively, and thoroughly discuss the Utah Effective Teaching Standards and scrupulously analyze standard after standard. One of the tuners described 120 the strategy the team employed as a pretty straight translation process: The strategy was to just trying to take what we had already been handed and translate it into something that was appropriate for teacher preparation programs. So, it was this pretty straight translation process, and sifting and sorting, and saying "no, that one doesn't belong, that one doesn't belong, that one doesn't belong; yes, this is Ok." (Faculty_individual interview_4) More than translating, the team talked through each element, discussed, and intentionally analyzed the UETS. The core of the Tuning faculty work was about communicating professional ideas how to develop student learning outcomes and competencies: Lots of talking. Talking, talking, talking. We went through every single one of the Utah Effective Teaching Standards and then the little indicators underneath each one. We talked about every single one of them [UETS] and what they meant, and we revised them. We wrote them out loud. Someone was transcribing, usually me, and making sure that we all agreed on what it meant. That was the most valuable part. There was intense talk about each one and what it really meant. (Faculty_individual interview_4) The faculty tuners wanted student learning outcomes to be consistent across all ETE programs. Because the team included representatives from each of the ETE colleges, this process not only resulted in Tuning the UETS to the reasonable expectations for beginning teachers, it also helped to align all elements of the ETE discipline among all the college and university programs "to be in sync as best we can" (Faculty_group interview_11). One of the faculty described this practical work: "It's like a curriculum map. We know where we're going, we're just not all taking the same way to get there" (Faculty_group interview_11). The tuners intentionally and persistently worked to accomplish a greater agreement and transparency of the ETE discipline's core competencies and learning outcomes. One of the tuners' leaders highlighted that: It was useful for us [faculty] all from all these institutions to sit in the room and to really talk through all those Effective Teaching Standards, and make sure that we [faculty] agreed on what they meant… it was useful to try to put them [UETS] in more plain language. I think, the students have felt when they are doing the 121 portfolio, like they understand more what we were looking for, what does this really mean. (Faculty_individual interview_4) The faculty tuners realized just how important it would be for the standards to be written in plain language. It could facilitate common understanding among institutions, minimizing the potential for individual institutions to form their own understanding and stray from the common direction. It could also help students evaluate their won capabilities and assess their own progress. ETE tuners worked to improve student performance and preparedness for transfer from 2-year colleges to 4-year programs. They developed a "specific observation form" that was linked to the Utah Effective Teaching Standards, and it let the ETE faculty be very specific about what competencies a preservice teacher must demonstrate in order to be an excellent teacher, and go out, and be successful the first year. One faculty member emphasized: Instead of just teaching them [students] and hoping for the best, now we [faculty] are out there. Rather than just telling what we see, we [faculty] observe them [students] and specifically say... For an instructional strategy you did an excellent job using this instructional strategy, here are some other ones, maybe, you can think about to mix it up. Or your content knowledge base showed very well here, you understand the subject really well, and perhaps another way to present it, maybe, would be this. Or sometimes, we say your content knowledge base wasn't… When you were explaining this, I wasn't sure what you were talking about, and I don't think the kids are, the students understand either. So, maybe, perhaps the next time your next lesson needs to activate their background knowledge, and also make sure you understand it yourself when you are teaching it, the students will understand it. (Faculty_individual interview_1) This illustrates the breadth of value that could flow from Tuning. Not only does it aid consistency between programs, it also provides benchmarks for faculty use in evaluating individual students and providing feedback to students. Additional to the concern of assuring that the emerging document addressed 122 student learning outcomes, the Tuners intentionally considered the perspectives about all specific elements of teaching. For example, faculty members prepare ETE students to teach reading, math, or science. In this case, the Utah Effective Teaching Standards are relevant but not very specific. For example, there is only one on content knowledge. Tuners wanted to make sure that their students would graduate from college with accurate content knowledge in reading, math, and science. One Tuning team member recalled their discussion about mathematical content knowledge: What would we expect prospective teachers to have in terms of mathematical content knowledge? We would look at the standard in terms of content knowledge, and then we would have a discussion. Somebody from one University, from Utah State, for example, might say, "Well, in our Math preparation program, we have these courses, and we do these particular things." Snow College might say "Well, we do basically the same, but we have some extra courses." (Faculty_individual interview_5) This shows that the tuners recognized that the UETS needed to be supplemented in the area of content knowledge in order to address the needs of new teachers as well as to produce statewide consistent outcomes. Therefore, the Tuning process supplemented the UETS in this area. Specific teaching practices, and questioning practices, in particular, also came under scrutiny by the tuners. One tuner reflected on the team discussion: Our Utah Effective Teaching Standards are divided into 4 different categories. They're like a minimally proficient and all the way up to a distinguished teacher. The question was, in terms of teaching practices and in terms of things that we [educators] expect teachers to do in the classroom. We [educators] certainly wouldn't expect them [graduates] to be distinguished, for example, in questioning techniques. We [educators] really wouldn't even expect them [graduates] to be in the average range for practicing teachers, but we [educators] wouldn't want them [graduates] to be minimal either. Where would we [educators] expect them [graduates] to be? We [educators] spent a lot of time talking about what kind of questioning practices would they [graduates] really be able to do? What could they [graduates] develop in a preservice program when for an elementary teacher, there are so many subjects and so many competencies they [graduates] have to 123 gain. Where would we [educators] actually expect them [graduates] to be when they [graduates] leave the program? (Faculty_individual interview_5) This resulted in a lot of time being devoted to the assessing the skill levels of ETE students. Accurate assessment of student skills is needed to demonstrate consistent student outcomes across programs. The tuners went through course by course and identified adequate types of assessments the faculty could do that would be applied to different outcomes. They discussed what elements faculty could reasonably observe and evaluate in a preservice study. They analyzed very tough topics of the UETS and attempted to make them applicable to the preservice teacher settings. Then they shared it with their faculty colleagues. Tuners discussed the assessment tools that very much developed and evolved for the last 25 years and currently tied to the UET standards, and could be used for student teaching or for ETE field practicum. They looked at knowledge, skills, and dispositions, and they were working to adjust their evaluation tools for ETE field practicum and student teaching along the standards. To develop and articulate the student learning outcomes and demonstrable competencies, the ETE team also worked at the signature assignments for freshmen, sophomore, junior, and senior students, not to duplicate them but to build on them. The team was: curricular mapping to see that by the end of the time depending on the assignments they [students] had had in previous classes, by the time they [students] finally were in the senior capstone, had we [faculty] taught them [students] how to fulfill all of the 49 standards, and in some way, shape or form had we discussed them [standards], had they been in the assignments that articulated them, was there something that it touched on all of them, so that when we send the students out into the classroom, we are fairly confident that they [students] are not only familiar with the standards but competent and fulfilling them. (Faculty_individual interview_1) Here we see that very important questions including where in courses students were 124 taught about these particular elements, where faculty were going to evaluate whether or not students could do these particular outcomes were a subject of discussion. In order to articulate the demonstrable outcomes and competencies, tuners also discussed what they could collect, what would be the actual artifacts or documentation that they could collect to be able to show that ETE students understood the particular problem, materials, and issues. The ETE Tuning meetings' minutes showed tuners' debates about classroom management plans, student teaching observations, case study analysis that students would do, the portfolio, and teacher work samples. Tuners were trying to make sure that ETE students would be coming up with things for which they would have data to substantiate if they were to make a particular claim about that. Faculty tuners intentionally worked to be able to assess outcomes at different points across the program. They wanted to make sure that student learning outcomes would be assessable and measurable even though many things would be hard to measure and assess, and this part of Tuning was particularly difficult because of developmental character of teaching and learning. As one of the tuners said "it is not possible to frontload student learning outcomes and be able to say ‘oh, done' … we [faculty] don't want them [students] feel like they can just check something off and never think about it again" (Faculty_individual interview_4). The ETE meetings also revealed the faculty's deliberations that many student learning outcomes could come at the end of the program or during student teaching and learning (USHE, 2012, November 30). Since teaching and learning are two developmental processes, students can always improve their performance of a particular outcome, and faculty want their students to be lifelong learners. 125 The faculty tuners were encouraged to seek formal and informal feedback about the ETE discipline core and faculty-wished student learning outcomes and competencies from their department colleagues and all those who were interested in the discipline Tuning (USHE, 2011, October 21). Going through the intensive discussions about what the outcomes would be for preservice teachers, the tuners became more comfortable with talking about their Tuning work as it was continuing. Ultimately, they started to seek feedback. The team gathered informal and formal feedback throughout the project from many educators: faculty colleagues, ETE students and alumni, and potential employers - school districts and school principals who responded with their expectations about ETE graduates' abilities and skills. The faculty tuners were seeking responses and feedback about what specifically ETE graduates should know, understand, and be able to do in terms of the ETE discipline. The rationale for seeking feedback from the faculty colleagues was to get a broader professional community input, obtain substantive feedback, yield greater depth, and opportunities for expanded responses. From recent ETE students, faculty tuners were trying to get responses about what specifically in terms of ETE as their college program they found the most important and useful in their teaching job. From school principals and school districts as target employers, tuners were seeking responses to what specifically, school principals would expect preservice teachers to know, understand, and be able to do. The overall key question was: what are the core concepts, competency statements, and measurable student learning outcomes? The ETE Tuning team agreed that the crucial goal in obtaining feedback was to hear from various groups of their faculty colleagues from all colleges and universities on a variety of the ETE Tuning 126 issues. They also agreed to consider all feedback, regardless of the methods used or the number of respondents who expressed a particular viewpoint. The ETE Tuning team members formally shared their Tuning experience in defining and articulating student learning outcomes and competencies with their faculty colleagues. Faculty participants in the Tuning teamwork also talked with their colleagues in hallways or updated at the department meetings. Colleagues in home colleges and universities helped to identify and sharpen specific student learning outcomes and competencies essential to the ETE discipline. Departments and campuses discussions and feedback provided prospects to identify and articulate student learning outcomes and competencies language more precise and transparent about what specific outcomes should be attained. Additionally, colleagues consulted in efforts to scrutinize the discipline core once it was drafted. Tuners got information from their faculty colleagues and brought it back to the Tuning meetings. They got their colleagues' input, yielded greater depth, substantive feedback, and expanded responses. Faculty colleagues generated valuable input by which tuners could revise all components of their Tuning work - discipline core, reference point outline with students learning outcomes and competencies statements. They consulted students by asking questions before and after classes. One faculty tuner described how they asked students to tune for themselves: We used to have the students interpret the standards. In some ways we asked the students to tune for themselves. What does that mean? What does student environment mean? Here what our indicators are. What do you think it means? And from that process, we had them write belief statements, for example, ‘I believe that an effective learning environment is blah-blah-blah,' and then they would come up with artifacts that matched their belief. (Faculty_individual interview_04) 127 This demonstrates that students provided valuable insights into how the discipline was perceived, what motivated the decision to major in the ETE discipline, expected careers, and disciplines associated with the ETE major through minors and certificate programs. Tuning report (USHE, 2014) also said that alumni, having made the transition from college education to teaching in elementary education, provided insight that both reflected on their educational experiences and considered their professional experience. Essentially, feedback from target employers, school principals of Utah, was obtained. School principals responded with their expectations for ETE graduates entering their professional world and starting teaching. Elementary school principals reported their expectations for graduates with a bachelor's degree in elementary education. The elementary school principals as target employers for preservice teachers prioritized similar skills and abilities for ETE graduates. Table 4 (p. 188) shows school principals' expectations about ETE graduates' abilities and skills (http://utahtuning.weebly.com/uploads/1/4/6/9/14699846/principals_expctns.pdf). They embraced classroom management, which included graduates ability to engage students and use a variety of positive strategies to facilitate student engagement. The second highly ranked was assessment which included ability to interpret data and use them to guide elementary students' learning and instruction. The third highly ranked were instructional planning and instructional strategies which included ability to be intentional and skills to set clear objectives for student learning and to make connections. The following were ability and skills to be collaborative, and a team player, understand student differences in learning styles, and have high expectations for student achievement. Additionally, the school principals listed content knowledge, flexibility 128 and adaptability, lifelong learning, effective communication skills, vision, and strong whole group Tier 1 instruction. The received feedback from their faculty colleagues, students, alumni, target employers was a catalyst for professional consideration by the whole ETE Tuning team. The final decision about the content of their Tuning product remained with the team as they prepared the document to be shared by departments offering bachelors' and associate degrees in the ETE discipline. Knowing when, where, how, and from whom feedback was practically obtained was important contextual information for the ETE tuners to consider. The information was helpful for the ETE team to consider it while they were tuning the discipline. Feedback offered a tremendous range of possibilities, provided unique insights into the ETE discipline, its perception by different constituencies, and the competencies necessary for future success for the ETE Tuning team to continue refining, enhancing and sharpening student learning outcomes and competencies. With a draft in hand, the ETE team compared, contrasted, and ultimately synthesized the general ETE discipline profile, core concepts, and student learning outcomes and competencies with the professional and accreditation standards. All outside resources also prompted reconsideration of the ETE discipline students learning outcomes and competencies draft. Department colleagues were key to attaining a critical mass of instructional staff who endorsed and adopted the conclusions and recommendations of the core ETE Tuning work. In fact, the frequency of these informal consultations, especially with colleagues in home departments, was a predictor of the effectiveness of ETE Tuning in individual colleges and universities. 129 However, the "Tuning project wasn't met with an open attitude" (Faculty_individual interview_7) everywhere. There were fears, some indifference, and resistance. One faculty member reflected upon her experience about seeking feedback from her faculty colleagues: When I came to the faculty, it was very interesting because there was a quite a bit of resistance from faculty. I think it was because this was a Liberal Arts College and so a lot of the faculty were also very socially aware about … At that time, there was another organization coming through evaluating higher ed programs with their teacher, preservice programs and rating them, and so one faculty member was very involved in that, and she became very defensive about the whole Tuning process and thinking that, "Oh, well, who's really funding this? Who is behind this? What's driving this?" not really understanding that it was actually developed by faculty, fellow colleagues, which was very interesting. My role there was very, I think, it was a very different experience, and so the Tuning project wasn't met with open attitude. (Faculty_individual interview_7) This quote illustrates a suspicion by some faculty about the origin of the Tuning project. Explicitly, the implied concern over possible political origin of the Tuning initiative as an innovation in higher education and what kind of change it would bring overshadowed the question of effectiveness. The Tuning team created a description of student learning outcomes and competencies specific for ETE college discipline at bachelor's and associate level. That faculty member continued to reflect that: What's ironic now is that I received a correspondence from the department chair. She actually wanted to have a copy of the things that we developed because they needed more specific information on how to evaluate preservice teachers and what outcomes they need to focus on and how to measure. The Tuning project standards gave more information than the current … that the standards they had already been using. I thought that was enlightening because at least they're remembering that, "Oh, yeah, we did have someone come to us and represent some specific standards." (Faculty_individual interview_7) This validates the need for measurable and degree and level specific student learning outcomes and competencies, and the mechanism or the system of the preservice teachers' 130 evaluation. Attaining consensus among faculty who did not participate in the ETE Tuning teamwork was extremely important because seeking and responding to such input also signaled to faculty colleagues that the ETE Tuning process was open, inclusive, developmental, and dynamic. Essentially, what started as an initial ETE Tuning draft statement was becoming a cross-state ETE degree specification. The Tuning team created a good product of student learning outcomes and competencies specific for ETE college discipline at bachelor's and associate level. As work progressed in ETE Tuning, the team achieved an initial closure to both formatting and content issues for their product named the Utah Preservice Teacher Learning Outcomes (UPTLOs) to move forward. The team members compared their agreed-upon student learning outcomes and competencies in articulated UPTLOs with what was articulated in the UETS. As one tuner described: It was both gratifying and validating to find out that they [standards] met the same level of competence and qualifications. I had seen instructors and professionals, and colleagues had determined the essential outcomes for teacher candidates, and compared with what was articulated in the Utah Effective Teaching Standards. Although the two groups of outcomes were coming from different groups of people, they were very close in what was determined to be essential skills. That lets me know that there are some accepting practices that we know through research, through practice, through trial and error, through whatever, and those keep rising to the top. This is what we need in order to prepare teachers. (Faculty_individual interview_10) Confirmation is important to validate new work. It adds confidence to the team and in the outcome. It is especially powerful when different groups reach similar conclusions through different processes or means. The ETE Tuning process of developing student learning outcomes and competencies was a developmental, dynamic, and iterative process though the description sometimes looks like linear steps. Each step of the process was an informational 131 supplement for the process as a whole. As each step was completed, the ETE team could realign all the information and the draft documents that they developed at that time. The ETE tuners were continuingly refining their developmental product as additional data were collected and brought to the team table. At every meeting the team reviewed and brushed up the student learning outcomes while developing and articulating the indicators of student learning outcomes to make them measurable and assessable. Tuning the ETE college discipline, that is, learning how to tune, defining the ETE specificity for bachelor's and associate college degree, and developing student learning outcomes and competencies, the Tuning team was facing many challenges. The following section reveals these challenges. Facing Challenges The analysis showed that the challenges were both at the development stage of the Tuning process and at the implementation stage of the process. One challenge that bridged both stages was that of faculty acceptance. Utah Tuning evaluators (Davies & Williams, 2014) completed faculty surveys wherein they asked questions about the respondents' general understanding of Tuning and respondents' frequency of discussion of Tuning and student learning outcomes in their departments. The question on general understanding of Tuning produced the results shown in Table 5 (see p. 188). The data show that 70 % of ETE faculty surveyed considered themselves at least partially knowledgeable about Tuning despite the fact that 22 % did not even respond to the question. Responses to the question on frequency of discussions are shown on Table 6 (see p. 188). Of those who responded, only 17 % had faculty discussions of Tuning at 132 least monthly, whereas 74 % of ETE faculty discussed learning outcomes at least monthly. This means that ETE faculty are comfortable with learning outcomes but not so with Tuning, despite the two being very closely related. The evaluation report (Davies & Williams, 2014) goes on to discuss resistance to Tuning and some of the possible reasons behind it. The evaluators found that some faculty had "concerns regarding standardization" (p. 9) but ETE faculty concerns distinguished between "a common (or standardized) set of learning outcomes" (p. 10) which they found more acceptable while faculty "also believe their program is not compelled to prepare and assess their students in a standardized way" (Ibid). The researchers also found that some faculty questioned whether "Tuning is a grassroots initiative. Often when introduced to the concept of Tuning, faculty asked who was asking them to do this and seemed to be questioning the motives behind the initiative" (p. 12). Another barrier is found in the reality that "[t]eacher education programs have many masters" (Ibid). While developing student learning outcomes and competencies for their college programs, the ETE team faced the challenge to work with many standards and regulations. Elementary education as a target field for graduates of the ETE college programs is very much regulated. Consequently, the ETE programs have external national and state pressures. The ETE programs have to follow very many specific external evaluation requirements and standards including the Common Core, CAEP guidelines, the Utah Effective Teaching Standards, TEAC and InTASC. The ETE programs are in control of what they do, and the ETE departments cannot just decide about some necessary outcomes. State agencies and national accrediting bodies 133 determine what faculty have to teach in their classes. One of the team members expressed it in a half-joking manner: "in Elementary Ed we are very lucky to have a large number of people telling us how we are supposed to be doing our job" (Faculty_individual interview_01). Accordingly, the challenge for the ETE Tuning team was first to understand and take into account all teaching standards that they were held to. All teacher education programs in Utah are beholden to meet standards for statewide licensures and beholden to accreditation, agencies like CAEP. Consequently, one of the main obstacles of adapting the Tuning process was a perception of some department faculty as another set of standards. One of the ETE Tuning team members described this faculty's reaction: "We have TEAC already. We have CAEP. We have the Utah Standards. So why Tuning? This is another thing we have already done, so that was also another sense of dissatisfaction of the faculty" (Faculty_individual interview_07). Another faculty believed that, before the Tuning project the ETE programs looked "more tuned than un-tuned" (Faculty_individual interview_11) because all institutions were under the same requirements, which created more similarities than differences. Connected with this, some tuners thought that the process of trying to establish student learning outcomes was irrelevant because the outcomes had already been determined. Another team members said: We probably do not need to tune because we have already been tuned. We have already had these groups [experts] that have decided what people should know, be able to do as beginning teachers, and I do not think they are wrong. I think sometimes it is hard to measure. I do not think that the things they have decided important are wrong. I agree with them. Sometimes they put more emphasis on one thing than another thing that I would necessarily do, like the TEAC process. (Faculty_individual interview_04) The emphasis is on the fact that there are very many specific external requirements, like 134 external licensing requirements. Programs are in control because matters have already been decided by others. The feeling is that expert groups have developed existing standards and the programs are already "tuned." This displays a misunderstanding of what Tuning is trying to do - establish a working, replicable tool to measure learning outcomes, in other words, to define and measure student success in easily understood ways, rather than determine what is to be taught or even to describe expectations in nonspecific, nonmeasurable terms. Connected with the state requirements and standards, one of the challenges for the ETE team was to make certain that student learning outcomes should be universal enough for college graduates who would go and teach in other states or internationally. The team was concerned if graduates would be competent to teach not only statewide in Utah. One of the faculty tuner expressed a very serious concern that UETS standards seemed to be a barrier for preservice teachers going to teach outside of Utah. It was a challenging subject of discussions for the ETE Tuning team. She said: That actually was one of the challenges in using the Utah Effective Teaching Standards. All of the universities, but, in particular, a college like BYU [Brigham Young University], has more people from out of state than a lot of the other universities do. Even though they all do, but BYU has this really diverse population. It became a conversation of, "Are these Utah Effective Teaching Standards what they [graduates] are going to encounter if they are teaching in Georgia or New York or wherever it is that they end up?" And she continued: One of the obstacles was, making certain that it [learning outcomes] was universal enough, so that it wasn't just tied into the Utah way of doing things. UETS were based on the National InTASC standards, which came from the Council of Chief State School Officers, and that is the National Standard. The Utah standards were just adapted from that. But how to make sure that it [a learning outcome] was universal enough? There were a lot of discussions about that. "Are we just preparing teachers for Utah or are we preparing teachers for other places as well, even internationally?" (Faculty_individual interview_05) 135 The faculty member is concerned that state requirements to strictly follow UETS might be a barrier for ETE graduates who could come to study in Utah and but would be teaching outside of the state. The faculty member also implied an important feature of the UETS. They are a potential barrier to bringing teachers into Utah who have studied in institutions outside of the state, as those teachers would not necessarily be prepared under similar standards. Thus, the UETS, and any state specific standards for that matter, can serve as a barrier to dispersal of students to other jurisdictions as well as to recruiting teachers trained outside the state. Among other things, this could be a barrier to the spread of new ideas and techniques. Another challenge was the ETE tuners' concern about the heavy language of the UETS and, accordingly, discrepancy of the standards' language and their main audience - elementary teachers. For example, in one of the group interviews faculty members were very critical about the heavy language of the UETS: Speaker 1: I do a little bit of model teaching. We are routinely, we are always in classrooms. The people that make these forms, these decisions, are not in the classroom on a regular basis. We want it [UETS] to be usable for teachers. We do not want to dummy things down. But we don't want to use so much jargon and so much educational-ease that it is totally lost on the teacher. She continued: Years ago I was over at Head Start. Most of my Head Start teachers barely had a high school diploma and they were using these big terms ... There is a couple of terms that I didn't even know what they were, and you expect a preschool teacher with a GED to interpret that and try to the child. I think we do that in the regular education system routinely. Sometimes we jargon things up educationally so much that we are not going to see results on the front line because they looked at that standard and go, "I'm not exactly sure what it is," so they move on. Speaker 2: Then you just do not even try, yeah. Speaker 1: They move on to something they can understand. Speaker 2: Yes. This is very true. Speaker 1: I thought, "We just need to be careful." I think elementary, of all places, because we have people working with little kids on a regular basis, not 136 heavy, heavy content like you would get in a secondary English class. You have got elementary teachers who speak that language and talk that language, and you give them these big heavy things, and the interpretation is not always there. (Faculty_group interview_1) The central idea of the faculty critical comments is about practicality and applicability of all standards and requirements in general, and UETS, in particular. Besides the number of the standards and requirements, the language of the UETS is not always transparent and clear. There exists a real need to modify the language of UETS and make it more workable for preservice elementary school teachers. For example, Speaker 1: I'd like the form to be shorter. (laughs). They're 49 indicators, which is a lot of things for a preservice teacher. Speaker 2: There're a few things like, technology appears in about five or six different standards. It would be nice if we just had a standard that said ‘they effectively use technology in all their teaching responsibilities.' Rather than have to mark, "Do they use it in this? Do they use it in this?" (Faculty_group interview_1). A faculty member gave an example how UETS could be not applicable. An anecdotal situation happened in one of the schools. Statement 2 of the UETS: "The teacher candidate collaborates with families, colleagues and other professionals to promote student growth and development." (UETS, 2013). A site teacher said that he had to give an A grade for a student even though he did not observe how a student teacher could demonstrate this skill: Speaker 1: He said, he put an A at midterm because he said, "I haven't seen her [a student teacher] do that." They haven't had their teacher conference yet. He doesn't think that any student teacher would have the opportunity to actually show that. He said, "Maybe you could change the wording on that." (Faculty_group interview_1) The main idea in these reflections is that UETS are not always appropriate for preservice teachers. The standards are very wordy and need more work to be articulate and applicable. It is not always possible to observe how a student can demonstrate some 137 skills because of different reasons. Faculty cannot always observe testing or assessment portion. Some schools do not even allow student teachers to be in a classroom during tests. The main challenge was how to make UETS practical and applicable. All interviewed ETE department faculty expressed their preference to have a modified version of UETS for preservice teachers. It was one of the goals of the ETE Tuning work to break down the Utah Effective Teaching Standards into measurable terms and make them transparent for preservice teachers. It was a challenge. All interviewed faculty expressed their concern and worries about students' burden on the number of standards for graduates to remember and follow. It is very important for all teacher candidates to understand all of the UETS in order to get a teacher's license, and what they are going to face in their teaching profession. A teacher candidate must understand all of the 49 standards presented in UETS and has to focus on them and follow them. The UETS are very much confused for preservice teachers. Many of the UETS standards overlap. Therefore, the interviewed faculty were concerned and doubt how viable the UETS standards could be for ETE students' learning growth. However, all interviewed faculty expressed that they were not in a position to change these institutional rules: strictly follow the UETS. Facing this challenge and responsibility to properly prepare students for their teaching profession, faculty could only "keep massaging our practices and better pedagogy in class" (Faculty_group interview_1). Faculty expressed their disbelief that the situation could be changed. "It'll never fly because we've already tried that. We've gone up that hill and been slaughtered, so it's not going to happen" (Faculty_group interview_1). One of the biggest challenges was also coming to a consensus. A lot of it was 138 based on individuals' experiences and expertise in different areas and different research bases that ETE Tuning team members came from. They were able to talk through those things and come to the commonality. For example, What do we mean by this? Is this really that important? I think those were the challenges. Challenges of getting what, ten people in a room, with varying backgrounds? It was very collegial. So when you are saying "challenging," not challenging in the sense that there were some personality clashes or anything … just trying to contribute the very best everybody could, and then come to a common understanding. (Faculty_individual interview_11) Here we see how different perspectives needed to be reconciled into a common understanding. In terms of teaching methodology or teaching science that is taught in a college of education, the obstacles that ETE Tuning team came up against were differences in philosophy of, "Are we inquiry based or do we believe in direct instruction?" (Faculty_individual interview_05). That was a subject of deliberations and coming to consensus. Eventually, the faculty as experts came to conclusion that preservice teachers should be able to demonstrate both. Some of the challenges that the ETE team faced were the faculty members' different approaches in what and how they were looking at signature assignments. Even though they were all from elementary education programs, their approaches were very different. This was an obstacle to achieving professional consensus, but ultimately, the consensus produced a robust result in the form of Utah Preservice Teacher Learning Outcomes. Another challenge was an organizational one connected with logistics. It was difficult for faculty from the universities and colleges far from Salt Lake City to attend every ETE Tuning meeting. One team members reflected upon this challenge: It was not easy to get people from around the state, a large geographical area, who have many roles and responsibilities at their institution to find time to come together and meet. With technology it's easier than it's ever been. But 139 sometimes it's not the same as when you are in person. Having physical distance when you're in focused on a singular purpose can be a barrier. Because of logistics, not everyone was at every meeting every time, which you could never have but it would have been nice if there would've been some way to include participants to get together. (Faculty_individual interview_10) Explicitly, face-to-face participation was very important and needed. Physical presence eliminated additional barriers and supported intensive deliberations and active Tuning work. Those who could not come could miss valuable information from the Tuning participants. The people who were on the team were all very much involved and busy and had responsible roles in their work. As professionals, faculty were all pulled in many directions. Logistically, there were many things important at that time. The team member also implied that part of the challenge was that the university had to commit to professional development, to faculty development. Faculty needed to be given time. The faculty needed to talk to each other, but they were not given time to talk. Another set of challenges during the Tuning work was connected with articulating student learning outcomes in measurable and assessable terms. The ETE Tuning team's biggest concern was the disciplinary perspective. One of the tuners remembered the team apprehension and uneasiness: "How do we articulate so that students understand what we are trying to help them develop? Then how do we measure that? I think it continued to befuddle us a bit. That increasingly became the conversation" (Faculty_individual interview_08). Another team member recalled how the process of articulating student learning outcomes and demonstrable competencies in measurable terms was very challenging: What I recall and, I think, this is the most difficult thing for anyone. How do you measure? It is really like the word choice, and being precise and explicit enough so that a behavior can be observed and measured. I think that was the most challenging thing. It was a lot of teasing out words and throwing on ideas, and 140 then rethinking, reflecting during that process, and what would that look like. (Faculty_individual interview_07) The team was teasing out, throwing on ideas, rethinking, reflecting, drafting, refining, and choosing words to be precise, explicit, and transparent that students could demonstrate their skills, and faculty and school teachers could observe and assess students' skills. The most challenging thing connected with being explicit and precise in developing student learning outcomes and competencies in measurable terms was to express student learning outcomes in measurable terms for some very specific abilities. For example, being able to articulate how to measure how much a teacher candidate cares about his or her elementary school kids was very difficult for tuners. Tuners had to have documentation that faculty would be able to measure how graduates care about elementary school students which was really important. The challenge was also to articulate measurement for students' ability to use classroom technology accurately and reliably. The UETS standards require preservice teachers be capable of using classroom technology. However, in reality, it is very hard to measure it because of the variability across teachers and classrooms, and school districts, all the classrooms are different and have different kind of technology. Besides, the cooperating teachers whom the students work with are not always technologically savvy. They might have equipment but they do not know how to use it so they cannot show students-graduates how to use it. One faculty member emphasized: I think that was the most challenging thing because when you think of outcomes, you think, we'll have to measure progress or where the students rank, but then what will that assessment piece look like? In Special Ed, we really focus on that observable behavior, the measuring, and the explicitness and so it's challenging. Another thing we do is teach our students how to write these individual learning 141 goals. Students who qualify for Special Education have an individual learning plan and so students (preservice). We have to teach our students to basically write the outcome measure for the individual child that they're working with. (Faculty_individual interview_07) Because of the unique nature of Special Education, it is possible that these programs are accidentally further advanced down the path of Tuning. The real world need to articulate outcomes for elementary students in these programs has resulted in preparation programs that evaluate college students on their ability to articulate measurable outcomes. The main and important questions were how to be more effective and what questions to consider. This faculty member continued: What was more challenging for me if my assignment going to meet that standard and then how would I know or what validation or verification. Where's the validity? And what am I doing - the assignment or product? How does that fit in? Is it really representative? It's an interesting question because that whole concept of assessment and measuring students' outcome can be very subjective and so how can one be more objective with that? That was a challenging thing. (Faculty_individual interview_07) This displays the difficulty a teacher would face when considering how to design an assignment to produce a desired learning outcome. Under a Tuning guided environment, it is not adequate to simply cover material and assume that once a student has mastered the material, they have achieved the desired outcome. Instead, Tuning demands that a teacher design a lesson and convey the lesson to students in a manner that they will achieve the desired outcome, and know, understand and be able to demonstrate their learning outcomes. At the implementation stage, the biggest challenge directly connected with many standards and regulations was that of institutional in higher education. There were worries among the team members if they could make necessary changes in the institutions. What would it take to make those changes in the institutions? For example, 142 some of the courses that elementary majors take for Mathematics are taught in their math departments. They are not taught in their elementary education department. The question then was, ‘Can we get the Math department to go along with these changes that we are talking about or the Science Department or anything of that nature?' (Faculty_individual interview_05). Ultimately, this is a problem of determining what the boundaries of the ETE Tuning program should be. Are the boundaries set in the discipline of education, or do they include subject matter content traditionally assigned to other academic disciplines such as mathematics, humanities, or social sciences? The biggest challenge of all, and one that has not yet been overcome, is to obtain acceptance of the Tuning results. This is a problem at both the state regulatory level (USOE) and the individual institutional level. Describing the challenge of translating the Tuning work down to a college or an ETE department, one of the tuners emphasized: It [Tuning] didn't translate down into, at least, my college, my school of education. I guess a specific example is just in talking to the department chair about what was going on with Tuning, and there was never really any response to that. It is as if it did not really matter. I'm making the interpretation that it just didn't matter at that time because there was so much change going on within our school. Everybody was trying to figure out where they were going. Then it was looming with the accreditation and all of that stuff which we did well in accreditation, but that's where all the energy was focused. It wasn't the right time for this level. Tuning seems like it occurred as a separate entity, almost in a vacuum itself. Outside of what the normal operations of the college were. That seems to be the biggest barrier. (Faculty_individual interview_11) This faculty tuner was facing a situation where the school was undergoing an accreditation review that consumed the department chair's attention, which is very understandable. The department chair must assure continued accreditation in order to have a viable program. Viewed in terms of Maslow's Hierarchy of Needs, accreditation is somewhere near the food and shelter level and Tuning is more akin to self- 143 actualization. This, of course, is only one possible impediment to acceptance at the department level. However, it points up the possibility that implementing Tuning at the institutional level may need to be viewed as part of a continuous improvement protocol rather than as a project. Directly connected with institutional barriers there was the issue of acceptance by Utah governmental institutions. As one of the tuners said: The challenge was knowing whether or not the State would support them [UPTLOs]. And I think they [USOE] thought that they would in the beginning. They [USOE] had a representative on the committee. But apparently it was not communicated well to the Office of Education what was going on. And it [UPTLOs] wasn't accepted. (Faculty_individual interview_06) The USOE participated in the Tuning project, but turnover in the USOE representative to the project reflected not buying-in by the USOE of the ETE process and outcomes. Ultimately, the State Office of Education had already determined the UETS standards and viewed the Tuning results as interfering with those standards, rather than supporting them. The ETE team basically went through the entire process of Tuning, and then they were told that they could not use the developed learning outcomes for preservice teachers. As a result, faculty experienced much frustration. As one of the tuners recalled: We did experience a lot of frustration. I was 100% on board. I thought we had a great product. I thought it [UPTLOs] would make a difference. And we were just told to shut down. And I think this could've been known up front. And the effort would've been better expended in another area, not education, just because of the nature. (Faculty_individual interview_06) The frustration experienced by this faculty member reflects her place in the project structure. The Tuning team was only one part of the project structure, and was responsible for the work product. The faculty members were not responsible for implementing or assuring acceptance of the product by the USOE. Tuning meetings were 144 creative and fruitful, the developmental stage of the project was productive even though there were some specific challenges that the team managed to overcome. However, implementation at the state level was beyond the faculty's authority. The USOE did not adopt the student learning outcomes and competencies for preservice teachers into the existing structure of ETE governance. The Tuning project did not exist in a vacuum. There were many powerful forces influencing education administration at the state level. The ETE faculty tuners did not have a power base but was more of a technical effort to effect change. The director of the Tuning project explained that the Lumina Foundation funded the initial phase of Tuning, but the product needed to be intrinsically motivated in colleges and universities. Tuning needed "to be used as a tool in the institutions. It is intended to be helpful, not to be imposed (USHE, 2013, January 25, p. 2). The following section presents data on ETE Tuning as a social and relational process. Collaboration: Tenacious and Tenuous This section presents the ETE Tuning project as a social process with many participants including faculty, state Tuning leadership team, Utah State Office of Education, department colleagues, and chairs, deans, provosts, students, alumni, school principals. Faculty were critical in this process, they drove the process of student learning outcomes and competencies development. All other participants played their roles in this process. 145 Faculty Collaboration and Collegiality This subsection described faculty collaboration and collegiality within the Tuning process. The data showed that the members of the Tuning team considered their collaboration to be a positive, social experience. Faculty understood that their work together involved true professional collaboration as each person could contribute ideas and then be willing to have those ideas discussed, challenged, modified, or dropped as the discussion evolved. The Tuning work was an amiable process. As an example, one of the team members commented, "there was not a lot of heel dragging, foot dragging, or even a lot of resistance" (Faculty_individual interview_05). The collaboration in the team meant that all input was valued as an addition to the debate and would be adopted or not based on group consensus of its merits. All tuners saw value in the process. Every time when they came together in an interdisciplinary way and collaborative way, they benefited from that. It was difficult because it depended on everyone bringing their own ideas, expertise, and personality, and being willing to cooperate and participate. In Tuning, faculty were learning to be very intentional, articulate about what they were doing to identify what could work and what could not work. There was neither aggression nor negative attitude in the team. One state Tuning leader emphasized that the Tuning atmosphere was very collegial and reciprocal. She reflected upon the ETE Tuning collaborative and collegial work: The group was gathered and someone threw out a competency, and the other said "no, no, action verb," and then they would start breaking it down. They [ETE tuners] like the other Tuning groups developed such an esprit de corps. They really enjoyed working with one another. They really enjoyed, and you could feel that in the meeting. They understood what needed to be done, and they worked with one another. At that point already, we were including private institutions too, and what it meant for their campuses, what was going on in their departments, they brought all of that to bear. It was just delightful to hear they bat 146 this around and take it apart, determine, question one another. They did what a good academic would do, that is, to raise the questions and not assume that everything is fine. That was just delightful. I just loved it. (State leader_individual interview_3) ETE Tuning was the developing process. Ideas from all team members and their department colleagues were very helpful. Faculty tuners wanted to do what was right for their students and for the elementary students in Utah, or even nationwide, for whom they would be preparing teachers. The process was very collegial. There were no personality clashes. Everybody was trying to contribute the very best everybody could, and then come to a common understanding to overcome challenges of getting commonality with varying backgrounds. One new faculty member recalled that being part of the ETE Tuning team she could meet and work with the representatives from all Utah universities. It was very useful for her professional development: The group that I worked with was always very willing to share their things and something new, and send them to each other. There was not the sense of ‘well, I designed this, and it is only for me.' It made it [Tuning work] easier. I think the collaboration and collegiality is great in elementary education. We have been very fortunate to work with people who are so easy to get a work with. (Faculty_individual interview_1) There was much collaboration in the ETE Tuning process, and faculty could collaborate back and forth and asked for things. ETE Tuning was a model of tenacious collaboration, both collaboration among faculty, and also how to create more collaborative relationships between faculty and students. The reports (USHE, 2012, 2013, 2014) and all meeting notes revealed that the process was very collaborative and very productive. Faculty all focused on certain things that they felt were important for student learning and faculty teaching. The working atmosphere was very respectful and very professional. The team leaders facilitated the 147 team Tuning work. Faculty felt they were able to contribute. They felt very comfortable to be able to contribute. Everyone could invest in the outcome. Everyone understood the difficulty of the task and also the importance of identifying what skills preservice teachers needed to be effective teachers, which was the aim that all tuners shared. Essentially, tuners were liaisons between the state Tuning committee and their department faculty, and also reported back. They committed into the process. They came to the Tuning meetings and contributed, participated, worked at the development of the final document. Comprehending the essence of Tuning, in general, and deliberating the student learning outcomes and competencies, in particular, the faculty were actually actively developing the ETE Tuning process. Inside the active Tuning work the ETE team members determined their agenda, they determined how they would manage their meetings. In fact, two-year and four-year faculty worked together to identify and articulate student learning outcomes and competencies for bachelor's and associate degree. One of the state Tuning leaders said "It was really their process" (State leader_individual interview_3). The faculty determined what and how of the process because "they were experts in their own fields what the learning outcomes ought to be, and so they worked through those. They worked through the competencies" (State leader_individual interview_3). This leader emphasized that was "extremely important … the concept of Tuning really took hold and people talked about it. They saw that it was faculty-driven" (State leader_individual interview_3). The ETE Tuning work provided for tenacious faculty collaboration. From professional academic experience faculty know that collaboration is not 148 easy. It is actually very hard especially when faculty do not have a powerful position. One faculty member emphasized: Collaboration is a real challenge. I am not always as successful as I would like to be because I am not in a power position, I am not Chair, and I am not the Dean. To get my colleagues to actually sit down and talk about these issues is not as easy. I keep trying to hone my skills as a collaborator and trying to figure out where I would find allies that would make this conversation legitimate. (Faculty_individual interview_8) This tuner recognizes that collaboration requires committed participants and it is very difficult for individual teachers to muster support for collaborative efforts in the absence of support from a higher authority. Administrative indifference would remove a powerful motivation for individuals to participate, and administrative opposition would absolutely doom any such effort. In this connection, ETE Tuning was also an effort for the state Tuning leaders. The Tuning process could not work unless the faculty came together. The effort was how to get the faculty together, how to create working atmosphere, how to help faculty see that Tuning work was worth their time and worth their energy. It is very often a challenge to get faculty together to discuss issues of higher education quality. An important realization here is that the Tuning concept came from the top, but it would not work unless the faculty actually got together and did the Tuning work. That element was very challenging. It is part of a larger issue for how people get along and how they work together professionally. There is no simple answer or solution for this issue. It is important at a department and college or university, at the state level to have expectations that the faculty can get together and can do their work. That was an important element of ETE Tuning. Each party played their role very well. The following subsection reveals findings about the state Tuning leadership role in the ETE Tuning process. 149 The State Tuning Leadership Team Role This subsection presents data about the leadership role of the state Tuning team. The Utah Tuning effort was organized through the Utah System of Higher Education, the state body that governs all public institutions of higher education in the state. The state Tuning leadership team was responsible for organizing the Lumina grant, hiring experienced consultants, organizing the elementary teacher education Tuning team, and outreach to the individual universities and colleges. The Utah ETE Tuning project was launched top down. This is how one of the state educational leaders described it: The State Office of Education, the Board of Regents, and all of the deans of our ten education teacher preparations programs, all meet together in counsel called the Utah Counsel of Education Deans. We meet together monthly. I personally no longer meet with that group since my role has changed, but I had that opportunity to meet with them for seven years. "Teddi" Safman, who at the time was the representative of the Board of Regents, brought to us this idea of getting a grant through the Lumina foundation and starting a Tuning project. (Gr3InterviewTuningETE) This suggestion perfectly matched other Utah universities and colleges' intentions to improve the quality of higher education. The same state educational leader continued: At the time we talked about doing three things. One, was to ensure that all of the ten institutions were more aligned and how they were preparing teachers. Two, was to look at all the content specific areas, so whether it was elementary education or math education or science education, that they were ensuring they had the right outcomes for their students to help prepare them to teach successfully. The third thing was also to look at best practices and research and that's really aligned with the second point. But, just trying to ensure that as they looked at the outcomes that they had, that they really were researched-based. So, all the deans agreed to participate in the Tuning project. (Gr3InterviewTuningETE) Explicit in this reflection is that the Tuning project was a good match for the state efforts to continually improve the quality of education in all ten-teacher preparations programs. It was initiated by the representative of the Board of Regents and agreed upon by the 150 deans and the State Office of Education. However, the Tuning work, as one of the faculty tuners emphasized, "couldn't have happened without a previous pedagogical wok" (Faculty_individual interview_8). Prior to the Tuning project, faculty already had a history of coming together and deliberating their college disciplines. In personal communication, Dr. Phyllis Safman (November 23, 2015) described a timeline of activities that led to and included the faculty discipline majors' meetings: In 1992, faculty from the University of Utah and Utah State University came together to discuss problems with transfer. Namely, that students from community college (we had five then) had to repeat courses they had already taken once they could transfer from the community college to the university. In 1993, the group expanded to include faculty teaching general education from all nine institutions. In 1995, they constituted themselves as the general education task force with representation from all nine institutions. In 1997, they started the Faculty Discipline Majors' meeting with about 22 academic majors that contributed to general education programs. In 2000 or 2001, the general education task force requested from the Regents that they become the "Regents General Education Task Force." The Regents approved. She continued depicting how these policies incorporated together: In 2000-2001, the five policies that referred to general education were subsumed under one policy, R470. Thus, policy covered all aspects of transfer and articulation throughout the Utah System of Higher Education. Also included in the R470 were other elements of general education, including the Faculty Discipline Majors' Meetings, the Regents General Education Task Force, and the "What is an Educated Person?" conference. Ultimately, the majors' meetings included 33 academic disciplines that contributed to general education across the now eight institutions. (Personal communication, November 23, 2015) Explicit in her description is that Utah faculty knew how to work across the institutions, and the Utah System of Higher Education already developed a good infrastructure for Tuning such as the Faculty Discipline Majors' Meetings, and the annual "What is an Educated Person?" conference. The Utah System of Higher Education established the Regents' General 151 Education Task Force and General Education Area Work Groups in order to implement the General Education policy. The Regents' General Education Task Force aims to "establish overarching learning goals in the core and other General Education areas, based on recommendations made by the General Education faculty and Majors' Meeting committees" (http ://higheredutah.org/wp-content/uploads/2014/05/R470-04_16.pdf). General Education Area Work Groups provide: Recommendations on competencies underlying each General Education area and suggestions on methods used to assess student learning outcomes in relation to the Essential Learning Outcomes and institutional learning expectations, … meet annually during the ‘What is an Educated Person?' conference, as needed, review the General education competencies and learning goals in each area, and discuss and compare programs. (http://higheredutah.org/wpcontent/uploads/2014/05/R470-04_16.pdf) The infrastructure appears to be performed to encourage an effort such as this Tuning project, and there would appear to be a ready-made slot where Tuning would fit into annual efforts to "review the General Education competencies and learning goals in each area, and discuss and compare programs" (http://higheredutah.org/wpcontent/uploads/2014/05/R470-04_16.pdf). Utah educational infrastructure seems to clearly accept the Tuning or other similar activity. Accordingly, the Faculty Discipline ETE Majors' meetings created an organization and foundation for the Tuning process. One of the state Tuning leaders described that faculty "knew how to come together, and they had experience in identifying what the competencies were" (State leader_individual interview_3). The ETE faculty had met together for years. Faculty had already had a framework for working together across their discipline. Faculty "were accustomed to coming together once a year" and "they had experience doing this kind of identification" (State leader_individual 152 interview_3). One of the ETE Tuning team members emphasized that majors' meetings "help articulate … there's been a great effort to make sure that courses at the different institutions are equivalent, have equivalent elements to them, so that kids find it easier to be able to transfer among the institutions" (Faculty_individual interview_11). The overall USHE efforts to focus on learning goals and student learning outcomes for all state colleges and universities were among the factors that advanced Tuning in Utah. Nevertheless, the ETE Tuning team needed more support for Tuning their college discipline because their work was much more laborious and difficult than the annual Faculty Discipline ETE Majors' meetings. It was intensive work aimed to create studentcentered, measurable, degree and level specific student learning outcomes and demonstrable competencies, and the system of the preservice teachers' assessment. The state Tuning leaders recognized the faculty's expertise and their competency to tune the discipline. ETE faculty got stipends for their Tuning work, and were valued team members. The Utah Tuning leadership team emphasized that ETE Tuning work was a faculty driven process, not a top down process. There is an important distinction to be made between the top down nature of initiating the Utah ETE Tuning project and the work of the ETE Tuning team. The state Tuning leadership team was responsible for initiating the project, launching the project, and doing the entrepreneurial work for the Tuning project. However, the expert work of Tuning ETE programs was left entirely to the ETE faculty. The leadership team was also engaged with the ETE Tuning team, attending meetings with the team and providing guidance as needed, but not directing the Tuning itself. Instead, the Utah Tuning leadership team concentrated its efforts on 153 factors that would advance Tuning in Utah such as building support for Tuning and developing structures to assure that Tuning would be sustainable in Utah. In order to be successful, Tuning must be acceptable to a broad community of interested persons and institutions. During the course of the ETE Tuning effort, the state Tuning leadership team was very active in communicating the work of Tuning to the public, to the education community and to accrediting agencies and organizations. The purpose of the communication effort was to disseminate information on Tuning in general, and about the ETE Tuning work in particular. Tuning is an emerging concept and building support for the project requires educating the public, students, teachers, policy makers, administrators, regulators, and others. ETE Tuning benefits from both general communications about Tuning and communications specific to ETE Tuning. One of the avenues for information dissemination was to get information placed into print media. In September of 2011, The Deseret News published an article on Tuning "Giving higher education a tune-up: Utah takes steps to ensure students have right skills for workforce" (Fattah, G., 2011, September 12) announcing the award of the Lumina Foundation grant and describing Tuning: The ultimate goal of Tuning, Gygi [director of Utah Tuning] said, is to make it clear to students, parents and policymakers what a student must know and be able to do for each major and degree level. For lawmakers, the approach gives them information so they can allocate resources based on an understanding of what a given degree means for students, society and industry. (Fattah, 2011, September 12, p. 3) The article went on to describe the Utah Tuning effort and how it would gather information from employers and others to align student skills with employer needs. This article, in a mass- media publication, was aimed at helping the public understand Tuning, and what it could bring to improve higher education. 154 The Tuning effort was also reported in Higher Ed Matters, the weekly newsletter, published by the Utah System of Higher Education. The newsletter has a circulation of about 800 consisting of "Utah higher education insiders, legislators, and members of the community." (Melanie Heath, communication director of USHE, personal communication, January 26, 2016). Publishing in this forum was directed at Utah policy makers and higher education professionals. The Chronicle of Higher Education also published an article on the Utah Tuning effort, which promoted the value of Tuning to students and changes that could possibly result from Tuning: The process builds in accountability, Evenson told me. Once you've defined the outcomes, you can ask, ‘Are the programs really doing that?' If a student finishes and can't do what's advertised, they'll say, ‘I've been shortchanged.' Transparency makes it easier for students, parents, and policy makers to make the right choices. (Carey, 2010, http://chronicle.com/article/The-Chimera-ofCollege-Brands/65764/) This shows that students could take charge of their education in ways not previously possible. They could also objectively judge their own skills against the Tuning standards and determine whether they think that they have received adequate education from their institution. It will also let students and prospective students make judgments about the cost benefit ratio of an education at various schools: The openness inherent to Tuning and other, similar processes will make plain that college courses do not vary in quality in anything like the way that archaic, prestige- and money-driven brands imply. Once you've defined the goals, you can prove what everyone knows but few want to admit: From an educational standpoint, institutional brands are largely an illusion for which students routinely overpay. The best teaching might be at Salt Lake Community College, or Weber State, or somewhere else entirely. It might even be from a place that's not an institution at all, but rather a provider of individual, à la carte courses. Openness will let us know. (Ibid) This communication, to a nationwide audience, was setting the foundation for acceptance of Tuning in the larger education community and in the political and public spheres as 155 well. In addition, it explained that Tuning would put more power in the hands of individuals as they evaluate education choice and value. Intentional communication by the state Tuning leadership team also included numerous professional papers, presentations to groups within the education community, and participation on various boards and committees (USHE, 2011, 2012, 2013). As with the print communication efforts, these outreach efforts were designed to disseminate knowledge about Tuning in general and about ETE Tuning specifically. Following are brief descriptions of some of these communications and their importance to the Tuning project. For example, Dr. Phyllis Safman, was a keynote speaker for the November 2012 Western Interstate Commission for Higher Education (WICHE) Commissioners meeting in Salt Lake City where she discussed Tuning, among other topics. WICHE being a regional body covering 15 western states and the Mariana Islands and composed of policy makers, educators, and business and community leaders, is a very influential body and is the locus of many policy discussions. Having a policy level body such as WICHE knowledgeable and accepting of Tuning was very important to acceptance in the higher education community in the Western United States, and therefore, would be helpful to the long term success of the ETE Tuning project. Other examples included a presentation to the Kentucky Council on Postsecondary Education, Statewide Tuning on March 2, 2012, wherein Dr. Daniel McInerney addressed some of the practical lessons learned from the Utah Tuning effort in a talk entitled From Theory to Practice: Institutionalizing the Tuning Process (USHE, 2012). Dr. Norman Jones was very active in presenting Tuning to the professional community with six papers and presentations in the 2011 to 2012 period. These 156 presentations and publications represented an important step in the evolution of the Utah Tuning project as it moved from learning and doing to disseminating practical knowledge and advice on how to implement the practice of Tuning. The state Tuning leadership team also worked with the accrediting agencies to simplify and streamline the reporting process. For instance, in January 2012, two members of the state Tuning leadership team, Dr. Bill Evenson and Dr. Janet Gygi, presented information on Utah's ongoing work in Tuning elementary teacher education to leaders of both NCATE and TEAC, agencies that play an important role in accrediting teacher education. On another occasion there was an informal dinner meeting with representatives of these organizations and a teleconference meeting with the executive vice president of TEAC. The state Tuning leadership team realized how important it was for the accrediting agencies to learn about and understand ETE Tuning and how it could improve elementary teacher education. In this part of the process, the state Tuning leadership team was advocating for Tuning. Their role was to support and promote Tuning implementation and institutionalization through collaboration with outside groups and agencies. As the Tuning project was drawing to a close, the state Tuning leadership team conducted outreach to all of the universities and colleges in Utah. The purpose of the outreach was to review the progress to date and to set the stage for continued progress once the Lumina grant expired. The meetings were held between the dates of March 22, 2013 and May 8, 2014 with the purpose of preparing for the end to grant funding for Tuning (USHE, 2013; 2014). These meetings were attended by members of the state Tuning leadership team at each college or university and were plenary to all four Tuning 157 disciplines. Participants varied by institutions, but generally included representatives from the Provost's, or Vice President of Academic Affairs offices as well as department chairs and institutional Tuning team leaders and participants. For example, the Utah State University team included department heads from the School of Teacher Education, the Physics department, and the Mathematics and Statistics department. It also involved the Vice Provost and the director of the Provost Office in addition to five representatives from the discipline Tuning teams. (USHE, 2013) At the University of Utah, the team included The Senior Vice President of Academic Affairs, the Associate Vice President, Academic Affairs, the Deputy Chief Global Officer, Chair of the History department, and a member of the physics Tuning ream. A similar range of representatives was found in each of the other college and university teams. This demonstrated a commitment from the highest levels of the institutions to understand and respond to Tuning. It also provided interaction among discipline Tuning team members, college or university leaders, and the state Tuning leadership team. The meetings were all organized around the same agenda that included reviewing the Tuning project, discussing the in-progress status of Tuning in Utah, a discussion of Tuning after expiration of the Lumina grant, and general discussion and questions. The meetings were well-attended and well-received, and a number of participants expressed a desire to continue to progress in Tuning. Not all attendees at each of the institutional meetings were intimately involved with the Tuning project, so the state Tuning leadership team provided a review of Tuning familiarize the participants with the structure, goals, and status of the project before discussing expected future activities. The Tuning leaders 158 explained that the ongoing focus on Tuning moved Utah ahead of other states in student learning outcomes and competencies development (USHE, 2013, 2014). However, with the Lumina Tuning grant coming to an end, the state Tuning leadership team intended to prepare the Utah colleges and universities for a new Tuning environment. Without the outside funds there would have to be some changes to the program but the State Tuning leadership team expected that Tuning would continue now that a statewide foundation and cross-institutional coordination had been established. Continuing to advance the statewide Tuning process in Utah would require ongoing meeting and coordination among institutions. Tuning at the state level would continue to be coordinated by Dr. Phyllis Safman. Future statewide Tuning would be more limited and the focus of work would shift to individual institutions, which would proceed with Tuning their curricula. Semi-annual meetings of the institutional Tuning representatives would be held, the first at the Faculty Discipline Majors' Meetings in the fall, and the second would be held later in the school year. Beyond these meetings, the Tuning representatives from each institution would communicate through informal channels. The presentations emphasized that it is important for the current momentum to be maintained if progress is to be made. Institutional meetings discussed that the programs could be different, but they needed common language. For example, Institutions do not have to have the same language for defining learning outcomes, but at majors meetings, they need to talk about what a specific course means at different institutions, and what is essential to have the experiences transfer from institution to institution. Students think they are transferring things that they are not. (USHE, 2013, p. 114) Another area of difficulty for colleges and universities has been transfer between two-year to four-year programs resulting from the "disparity in learning outcomes for 159 essentially the same courses" (USHE, 2013, p. 114). Aligning individual programs with the Tuning results should help make more consistent outcomes to enhance transfer across institutions. For example, Weber State University had proposed a preeducation major, and the initiative appeared to be well received by education deans across the four-year institutions (USHE, 2013, p. 120). The Tuning project's development of competencies and outcomes will be an asset for this program and may assist in bringing additional recognition to work completed at two-year institutions. Weber State University's proposed preeducation major could begin by using the existing Tuning results, but it will require ongoing cross-institutional coordination with the four-year institutions to assure smooth transfer with no loss of credits for transferring students. Each ETE program in Utah was expected to begin the process of institutionalizing Tuning. This would mean that each participating department would be responsible for creating expectations and outcomes and communicating these to students. Ongoing communication would be the responsibility of the Tuning team chairs at each institution. There is an expectation that cross disciplinary Tuning meeting would be held on each campus. Mutual support across the disciplines would be important as the ETE Tuning project moves into the institutional implementation phase; it will provide broader perspectives to the ETE tuners. Faculty questions were similar across institutions, showing concern that the momentum of the Tuning project would be lost with the expiration of the Lumina grant; that without this resource there would be no ability to move forward either state-wide or within individual institutions. For example, "at the University of Utah, they are wondering how to establish an office of accreditation that is focused on student learning 160 rather than on the process" (USHE, 2013). There was also concern about overcoming institutional resistance to Tuning. One of the largest concerns for the education faculty was that the USOE had not accepted the UPTLOs and that as a result, ETE Tuning would not be able to advance. Tuning is a multiyear, iterative process that must go forward within the context of life at each university. The everyday problems of running a university can and do interrupt or delay the transition. For instance, UVU was undergoing a search for a new dean, and the associate dean left. The upheaval caused by these important institutional changes diverted attention away from Tuning. Overcoming faculty resistance requires ongoing work. At one university the faculty shared a departmental concern about assessment and standardization. Progress has been slow in an effort to address the concerns. One thing that has helped overcome this resistance is that the university has begun to require that outcomes be written into course syllabi. Having clearly articulated outcomes from Tuning helps the faculty see the correspondence between the Tuning results and the academic requirements. Elementary teacher education is different from the other disciplines that were part of the Tuning project. Elementary teacher education programs have pressure from a number of interested outside organizations. Principal among these organizations are the Utah State Office of Education (USOE), which licenses teachers in Utah and is a separate agency from the Utah System of Higher Education. The USOE has its own statutory obligations, governing board, approved regulations, and political pressures. The Tuning project is "not a stand-alone department but a small piece of a large education program" (USHE, 2014, p. 102). 161 The state Tuning leadership team initiated coordination with the USOE from the outset of the Tuning project. Two USOE representatives were initially placed on the team, with only one actively participating. Recognizing the importance of the USOE standards, the Tuning team worked from the USOE standards and made adjustments to accommodate the lack of classroom experience of preservice teachers. Despite these efforts, it is acknowledged that the USOE did not fully understand the Tuning process and its expectations, and initially rebuffed incorporating the Tuning results into the teacher licensing process. Ongoing work with the USOE has resulted in some progress in clearing up the misunderstanding and work will continue. The following subsection demonstrates the finding about the key role of the Utah State Office of Education. The USOE Key Role This subsection presents data about the key role of the Utah State Office of Education in the ETE Tuning process. The USOE was one of the stakeholders in the ETE Tuning process. Education in general and elementary education in particular, is currently under scrutiny by politicians at all levels as illustrated by the fact that there are both federal and state offices concerned with teacher preparation and licensure, whether directly or indirectly. In terms of elementary education, having the support of the State Office of Education in the Tuning project was very important: Because we [USOE] verify what they [preparation programs] send us. We [USOE] license them [graduates] and verify that they [graduates] are ready to go out and teach. That, I think, makes a difference, as well. If people know that the USOE has been involved and agrees with the processes that have happened, it makes a big difference. (Faculty_individual interview_05) As a public education agency, the USOE does not have direct control over the ETE 162 programs. However, the USOE does examine the ETE programs and any program that does not meet with USOE approval could find that its graduates may have greater difficulty achieving licensure. Overall, K-12 public education faces a challenge to meet the evolving requirements from the US Department of Education. For that reason, the State Board of Education has been ratcheting up the requirements for the ETE preparation programs to align with the state standards and endorsement programs. One state leader explained the growing role of the state educational agency: The state has been somewhat hands off of that [preparation programs] in the past but there is a lot of movement toward taking a larger and larger role. Particularly, new Title Two Regulations require going forward a lot more data from preparation programs, so that just as teachers are measuring their effectiveness with students, preparation programs are, going forward, going to have to measure their effectiveness in producing teachers who can be effective with students. (State leader_individual interview_12) This illustrates a chain of control starting with the US Department of Education and flowing through the State Board of Education and the USOE to the teacher preparation programs. Defacto, the USOE has a measure of control over the ETE preparation programs. In 2010, the Utah State Board of Education started the Utah Effectiveness Project for High Quality Education. Utah school teachers, administrators, and university representatives worked in Utah Effective Teaching Standards Work Group and RubricWriting Subgroups. The first year was conceptualization, and then the educators worked in different committees that developed the two sets of standards: the Utah Effective Teaching Standards and the Utah Educational Leadership Standards. These standards provide: 163 A basis for coherent system for all state and local educators as they develop a vision of an effective statewide system. The system is aligned with the goal of educator preparation programs and is designed to support a consistent and mutually reinforcing continuum of preparation, licensure, recruitment, induction, evaluation, and professional development of teachers and educational leaders. (UETS, 2013, p. xiii) The UETS represent a description of highly effective teaching that is currently authorized by State Board Rule (R277-530) (http://schools.utah.gov/CURR/educatoreffectiveness/ Standards/R277530.aspx). The standards are required for all educators in Utah public education. The UETS are based on the most recent Interstate Teacher Assessment and Support Consortium (InTASC) standards approved by the Council of Chief State School Officers (CCSSO) (UETS, 2013, p. xi). In view of that, the USOE is a very powerful actor in the Utah state education system and the agency plays a very important role in determining the directions for all state teacher preparation programs. Implementation procedures were needed to bring the UETS into the education main stream in Utah schools. Implementation started with "a lot of professional development in [the school] districts" (Faculty_individual interview_14). In addition to professional development, the USOE began to work on a model program that is: Based on the standards and is comprised of three major pieces, one is professional, demonstration of professionalism which is really an observation tool and an observation process. The second one is a measurement of student growth that could be attributed to the teacher and the last one is stakeholder input. (Faculty_individual interview_14) Most school districts are using the state model tool but some have chosen to developed individual models as allowed in State Board of Education rules. Reflected in the interview, full implementation of the UETS began with the 2015-2016 school year so every Utah teacher will be evaluated on state approved standards. The model provides an accommodation for teachers in their first three years of service, but the standards are still 164 very high. The interview participant explained that even experienced teachers find areas where their skills and knowledge need to be updated to align with current research. She also recognized that although the model makes an accommodation for teachers in their first three years, preservice teachers are a little bit different because they are still developing. Accordingly, changes and updates are needed for the ETE preparation programs. In 2011, when the ETE Tuning project started, representatives of the USOE were invited to participate. During the Tuning process, there were few USOE representatives in the ETE Tuning team, and the early representative from the USOE to the Tuning team resigned during the embedding process leaving the USOE with only one representative to fully understand how the many different viewpoints were represented in developing and executing the Tuning teamwork. ETE Tuning team members expressed concern that the limited USOE involvement would lead to problems with accepting the Tuning teamwork product: What we needed, the stronger role from the Utah State Office of Education. What is the purpose of our work? There is a large section of rules that governs teacher preparation. And in some ways we [tuners] did not consider that had to be addressed. They [USOE] needed to know exactly what it was that Tuning was trying to do, and they had to be on board with what we were trying to do with Tuning. If they are not on board with us, we can't do it because they are the ones who in the end say whether or not they are going to approve our program. (Faculty_individual interview_06) The ETE Tuners were aware of the importance of USOE in accepting the Tuning team work product, and also aware of the consequences if the USOE did not approve. However, the Tuning team was not in a position to directly influence the Office of Education. The following statements show frustration by ETE tuners with the limited USOE participation and the effect that had on perceptions of the work product by the 165 Board of education: I felt like the State Board of Education missed their role. Therefore, I think, they undermined a lot of what Tuning board members were doing. They were invited to attend all of the meetings, but they did not, and, I think, that hindered. I did not want to imply that the Board of Education purposefully devalued our work. I am not intimate with the role that the Utah State Board of Education played. They did not see, or want to acknowledge the work we were doing. They were a nonrole, a nonplayer. Then I personally felt that they complained that they were not included. It was their choice not to be included. (Faculty_individual interview_10) The choice by the USOE to limit their participation in the Tuning project was apparent to the tuners who were disappointed with how the USOE received their work product: The sad truth is also that we, the Tuning team, had forwarded the document to the Utah State Office of Education. I don't know who in particular. But as far as I have understood, that document sort of just died or disappeared. I do not see any evidence of it. (Faculty_individual interview_11) The outcome was deflating for the ETE faculty. They dedicated time, effort, and expertise to a difficult and sometimes contentious process in a sincere hope of producing a work product that would improve ETE preparation and hoped to get confirmation, or at least consideration of their final product. All ETE Tuning team members were excited to implement UPTLOs in their preparation programs. They knew how they were going to implement the outcomes because there was a lot of Tuning work done, and a lot of faculty efforts and expertise were contributed into this Tuning final product. They were going to train the department faculty how to tune the discipline, and they were going to get UPTLOs pervasive throughout the program. For example, at Weber State University it was a two-day training with all of the faculty where tuners introduced the tuned outcomes, and the faculty talked about every course where different elements would be taught. Faculty members recalled: 166 We started to make changes, and then we dropped everything. We were told specifically in an email: "You must have your outcomes according to Utah Effective Teaching Standards. Period, no question. Signed." … We stopped talking about the tuned outcomes and started talking about UETS. What the State law tells us is what we have to do. …We did lots of back-and-forth debates about it but their [USOE] final word was: "You will be UETS." (Faculty_individual interview_06) The Tuning team began to realize that Tuning would not be adopted by the USOE: …we were told what to do by the agency that licenses our students, and they [USOE] hold it over our head. They made it very clear to us that they thought that it [Tuning] was a great thing to talk these things through and make sure that we all understood, but they still don't want us to use the tuned outcomes except for the portfolio. So, if you don't include the State Office of Education when it comes to elementary education, it doesn't matter what you think. It is the Utah State Office of Education who holds the power. And they hold the power. (Faculty_individual interview_04) The USOE affirmed the UETS and not the UPTLOs would be used as the standard for evaluating per-service teachers and ETE program design: We ended up having to go back to the topic and teaching standards because of the fact that the Utah Office of Education told us, point blank: ‘You have to use the original Utah certificate teaching method. You cannot use these tuned learning outcomes.' Because they are the ones that determine whether our program is approved, then that's what we had to do. So, in some ways, this was a lot of wasted effort. (Faculty_individual interview_06) Consequently, the ETE tuners felt undervalued and frustrated. The USOE said that the Utah Preservice Teacher Learning Outcomes could only be in the portfolio and not in the syllabi. The State Office made it clear to ETE faculty that they had to use UETS full version. After that the ETE Tuning team developed a pared down version of UPTLOs for the portfolio, but the USOE still insisted that ETE programs were supposed to use the full version of the Utah Effective Teaching Standards. Faculty members from the universities' ETE programs felt frustrated too. One faculty member remembered the time: 167 When I moved up to Weber State, the Tuning process was nearing completion. At that time Weber State was looking for cites to building their foundation, and the ideas were so articulated and well written. Weber State fully acknowledged those [UPTLOs] and brought that into plans for accreditation to move forward. We probably would have used the turning learning outcome, except, at that time, the State Board said: "No, we are not going to use the preservice learning outcomes because the Utah Effective Teaching outcomes." It was not that much of a difference, but it was just enough of a difference that they [USOE] would not approve the preservice teaching and learning outcomes and said that we must articulate them with the Utah Effective Teaching Standards. (Faculty_individual interview_10) Because of the external evaluation requirements, the Weber State University ETE faculty could not directly implement the Tuning results into the curriculum. The department instead had to meet the UETS guidelines to the letter. ETE faculty felt a lot of pressure from the USOE: We have a State Board Rule that says: "You have to teach these classes and these classes. You have to evaluate your preservice teachers to make sure they give these topics for teaching." So, to spend the time and effort on Tuning does not help because people have to meet these other external criteria. (Faculty_individual interview_06) The external pressure demoralized this tuner: Essentially, Tuning in Elementary Education needs to be dropped. It really does. Because it's not viable. We have too much external pressure. It's something we can't do. So, we have to just do what is required by the State. (Faculty_individual interview_06) The frustration and hopelessness in these comments is understandable. Because the UPTLOs were not to be used in the program syllabi as the faculty tuners envisioned, the value of incorporating them into the portfolios was not fully recognized. However, the comments also reflect a lack of understanding of the political environment that elementary education exists in. The Tuning project did not take the steps needed to build the necessary support inside the public education hierarchy. The following section presents data how ETE Tuning required faculty to question their teaching practices. 168 Tuning Requires Interrogating Faculty Practice This section demonstrates that the Tuning process required interrogating faculty practice. The Utah ETE Tuning project started as a project, and turned out to be an iterative process. The Tuning work and thought process were nonlinear and involved many iterations and circular discussions. When viewed in retrospect, the faculty learning process was a complex and complicated one. Tuners had an epiphany regarding Tuning itself. It demanded more critical examination of teaching, and analytical thinking of the tuners themselves. The ETE Tuning work gave time for faculty tuners and their department colleagues to go deeply into their discipline. This is how one of the faculty tuners described it: It [Tuning] really gives you an idea, it gives you the time to think deeply about your discipline, and we don't always get that. As professors, we don't always have the time to think deeply about our discipline, and, I think, this is very helpful. It just gave us time to do that. (Faculty_individual interview_1) This statement shows that it is not only possible, but also common for teachers to fall into something of a routine, one that is not conducive to critical examination of practices. Tuning forced this participant to first realize that she did not always think deeply about her discipline and second, forced her to do so. Learning-Driven Process The Utah ETE Tuning process was learning-driven, and the ETE tuners were learning-driven. Their work was solely focused on developing a deep understanding of the Tuning process and, accordingly, transparently articulating measurable student learning outcomes and competencies for the ETE discipline. Tuners worked at articulating "what makes a teacher a teacher" and how to teach students to be competent 169 teachers. This, in turn, contributed to how to teach a deep understanding of the subject matter in the minds of their students. Figure 9 (p. 187) shows Tuning as a learning driven process. Tuning is learning centric, with student learning at the core and faculty learning how to teach students surrounding it. The dashed line between the two processes demonstrate that they are interrelated. Faculty must learn what makes a good teacher and then they must learn the ways to teach these things to students. The ETE tuners' purposeful team work was "to look at the scholarship of how students learn" (Faculty_individual interview_8). During the active Tuning work the intentional discussions were about "what's the most important for students to learn" and they were based on disciplinary expectations or the nature of the universities whether they were research universities or community-based institutions. The Tuning meetings were student learning-centered. Tuners discussed many questions aimed to identify student learning outcomes: What do we want from students at the end of the time when they graduate? What do they know, understand and are able to do? How can we quantify that? Can they demonstrate content knowledge? Can they demonstrate instructional strategies? What do they look like? (Faculty_individual interview_1) There were many deliberations in terms of teaching practices and in terms of outcomes and assessment tools that faculty tuners expected preservice teachers to do in the classrooms. The tuners were working on defining the essential pieces that students "must have upon graduating so that they're then able to handle their own classrooms and to meet the standards of the state?" (Faculty_individual interview_7). Purposefully and consciously, the learning-driven tuners were concerned with an array of factors that could influence student learning. For example, "the approach to assessment, the professionalism, the behaviors, how to carry yourself, the instructional design, being able 170 to engage students" (Faculty_individual interview_7). Besides, the tuners spent a lot of time "talking about what kind of questioning practices they [students] would really be able to do. Should we [faculty] expect them [students] to be able to do, and what could they develop in a preservice program?" (Faculty_individual interview_5). The tuners also raised many evaluation questions about college teaching and learning. They were asking "What are we doing as an institution to evaluate our students and how do we know that they're learning what they need to be learning in their preservice programs?" (Faculty_individual interview_7). Another question that came up was "how each of the institutions was evaluating their students in their preservice program and this was specifically, the Elementary Ed Preservice program" (Faculty_individual interview_7). One ETE team member emphasized that: Teacher education is so specific that we [faculty] have to really make sure that our students are graduating - actually come away with specific skills because it's a professional degree. So, there was an added component of how we should define this. (Faculty_individual interview_7) Professional degrees are, by their nature, designed to produce working practitioners. A qualified working practitioner needs a mixture of theory and technique in order to be successful. There was a recognition that evaluating these disparate areas of knowledge would require specific evaluation. Accordingly, the tuners "were really trying to make sure that we [faculty] were coming up with things for which we would have data to substantiate if we were to make a particular claim about that" (Faculty_individual interview_4). Focusing on their student learning, learning-driven tuners took intentional actions to see what the desired kind of learning took place. During their Tuning meetings (USHE, 2011; USHE 2012; USHE 2013) faculty deliberated "what would be the actual artifact or documentation that we 171 [faculty] might collect to be able to show that the students understand that particular thing" (Faculty_individual interview_4). They made notes about classroom management plans, student teaching observations, and case study analysis for students to do, the portfolio, and teacher work samples (USHE, 2013; 2014). The overall attitude was, "How can I [faculty] really help them [students] learn?" (Faculty_individual interview_8). The ETE Tuning work was a learning experience for tuners. In order to tune the ETE programs, that is, to develop student learning outcomes and competencies across the discipline, not for one course or a program, and articulate what students must know and be able to do, the tuners had to learn how to tune their programs. One state Tuning leader described how faculty - the ETE Tuning team members - were learning To be intentional, very intentional, articulate, and intentional about what they are doing. And identify what works or what doesn't work. There was no punishment there. You try and … we ought to question if you are not trying new things, and you think. How do you know that you are doing all these right things? (State leader_individual interview_3) They were learning through their intensive Tuning discussions. The intentional Tuning deliberations "were needed to look at what was going on in our classrooms and what our teachers, coming in, needed to think about, so they were prepared to get out to our public institutions" (Faculty_group interview_2). ETE Tuning was a multifaceted educational collaborative process about learning and teaching concerning the key people in colleges and universities - students. Namely, it was a process about students learning how to be competent and be ready to go to elementary education to teach elementary school students to be competent too. Second, it was a process about faculty learning how to teach students to be able to demonstrate their 172 competencies, and about faculty teaching of students to be highly competent elementary education teachers. These many facets were all interconnected and interrelated. The following subsection presents how ETE Tuning created discipline collaboration mechanism. Tuning Creates Mechanism of Discipline Collaboration The ETE Tuning work was not yet another thing that faculty got try. One of the tuners, state team leaders, characterized Tuning as: Building on. Tuning asked for a shift, asked for a thinking about the relationship between teaching and learning, and the necessity to help faculty, not only be thoughtful about that but to actually figure out how to do this and share ideas and come up with new strategies, new tools. (Faculty_individual interview_8) This tuner captured the idea that Tuning is more than simply writing new standards. It requires greater depth of analysis into connecting teaching to student outcomes. Not only that, but the tuners would need to develop new tools for teachers to use to determine whether students had actually mastered the material or merely repeated a verbatim response presupplied by the teacher. Recognizing that Tuning required critical examination of the relationship between teaching and learning, the Tuning team began to do just that. ETE Tuning made tuners and many of their department colleagues to develop deliberate teaching practices: It [Tuning]'s really made us much more aware that we have to have very deliberate teaching practices that we can't just assume that students know it, that we have to have really explicit instruction on what we are teaching, why we are teaching it, what the purposes and how they can transmit that knowledge, in turn, to their students. And so, I think it's just made us more aware that explicit instruction is very important when you want someone to make a connection. You have to give them all the tools to make the connection because very often they are not going to make it on their own unless you explicitly explain - you are going to be using these five things to get to this point. (Faculty_individual interview_1) 173 This reflects the beginning of understanding that effective teaching requires that students know that each instruction has a direct connection to an expected outcome. Moreover, the expected outcome is connected to a specific understanding that the student must develop in order to be successful. Another faculty, a member of the Utah ETE Tuning team, described how the Tuning work taught her to be concrete and helped her in her new position at the university: [W]hat has helped me in my new position here is really understanding those core pieces, those essential elements or how you want to define them in distinguishing what a veteran teacher or a more experienced professional would have versus our students coming out…. It's always coming back to these essential things. Are the students engaged? Are they using research-based practices? Are they using data to make informed decisions? Are they communicating and presenting things in a professional way? So, all of these things have been bury in my mind. What's been helpful for me is that, I think, not simplicity, but the level of being concrete has helped me. Being there [in the Tuning team] and explicit. (Faculty_individual interview _7) This faculty member is moving from the concept of conveying information to students to the concrete indicators that are needed to evaluate success in the classroom. Being able to make these observations and recognize when the students are actually showing these indicators would reflect success not only for the student, but also for the instructor. Deliberate teaching practices were a subject of faculty discussion in the ETE Tuning process since teaching: Really need to be more focused on what we [faculty] are doing, we can't just have an assignment because it is fun. There is a place for that. We have to have it [assignment] be something that shows competencies right away. And we have to list essential learning outcomes in our syllabi and then talk about them and list them on the board and tell the students why we are doing it. It's not just enough to have it in one place that you need to have it on canvas, and in your paper syllabus and then you need to talk about it in class. And so you are connecting it all the way through, it's not just listed in one place. (Faculty_individual interview_1) This faculty tuner has grasped that Tuning can bring the purpose and need for every 174 lesson to prominence. Courses that are loaded with filler material would be a detriment to achieving the goals and objectives set forth in the syllabi, and Tuning process would help identify these deficiencies and provide a guide to redesign these courses. In a group interview faculty emphasized that the process of improving teaching practices is ongoing. Fundamentally, Tuning is about student learning and improving teaching practices. One cannot ever say: We're done and check off the box. There's always something new or different coming down the pipeline. It's always evolving and changing. Speaker 2: As we [faculty] do more research, this is why our curriculum changes because we've learned more about brain-based learning or whatever and that impacts how we teach, what we teach, and that's why it's always changing. That's why it will always change. (Faculty_group interview_2) Explicitly, the tuners recognized the power of Tuning, but at the same time, they recognized that it would be important to retune in the future. What is unsaid here is whether Tuning should be a more or less continuous process or a series of events separated by a period of time to allow new practices to develop. Also unsaid is whether the Tuning process would serve for an indefinite period, or whether it, too, would need to be replaced at some future date, although such considerations are outside the scope of this Tuning exercise. The tuners created their meetings' atmosphere that supported their learning. They collected and brought to their meetings additional information and shared it with each other. They became trustworthy sources of knowledge for each other that helped them to shape their teaching practices and looked at them from a different lens than they looked at them before. They shared their thinking and understanding, and were convincing each other of the value of developing a deeper, more accurate understanding how to tune their discipline. One of the tuners highlighted the appreciation of her professional 175 development during the Tuning process: What was impressed upon me personally as an instructor was the importance in making sure that I am developing to prepare teachers in the different arenas. I need to start making connections and start back mapping to make sure students are prepared when they get to that point. (Faculty_individual interview_10) The tuners were forced to think in ways they had not previously practiced. This was an unintended outcome, but one with substantial benefit to the individual participants. At the same time, the ETE Tuning faculty brought the Tuning ideas to their departments and initiated the conversations about the importance of the Tuning work for the quality of a college degree. The department discussions, in turn, impacted faculty on how important to be explicit: Listening to conversations in our faculty meetings has helped me [faculty member of the ETE department] be clear about how everything I'm doing needs to relate to learning objectives. Explicitness, that's the right term. I actually realize now the importance of going back to the objectives to talk about them, so they [students] don't just think we made things up out of the air. Doing all that has helped me. (Faculty_group interview_2) This faculty member realized that her students' learning experience would be improved by making information on program objectives available to the students. It would help the students understand the value and importance of the curriculum to their future success. The faculty tuners challenged their teaching through creation of conditions in which student learning would occur. For example, One of the Utah Effective Teaching Standards is that teachers are supposed to be able to ensure an inclusive learning environment that allow each student to learn to reach their learning goals, and so we said "what's that mean inclusive?" And so we talked, we gave specific examples to each other, we brainstormed labeling things in the classroom in different languages, using like popsicle sticks with all the students names on it, that we were calling on students in a fair way, different traffic patterns so that if you need it to accommodate a wheel chair. That's our assignment: "don't assume that all students come from the same background." (Faculty_individual interview_4) 176 The tuners were deliberating among themselves and learning, as a group, how to think about a very complex task, that is, how to impart a thought process to their students, and subsequently deciding how to evaluate the students for their ability to employ the process. Consequently, ETE Tuning encouraged some faculty to move from the Tuning rich conversation with colleagues to have some more individual study, a real selfreflection: We [faculty] should be very explicit. We should try to figure out ways, experiences that help them [students] achieve their professional goals. We can't just suppose if they're going this figure out on their own. Again, because they [students] don't necessarily come from backgrounds that encourage that. (Faculty_individual interview_8) Self-reflection has caused this faculty member to reexamine her own approach to helping students achieve their professional goals. While Tuning may not cause all participants to react similarly, this shows how one participant was being personally influenced by her work. One of the tuners was very articulate about "a number of competencies" that faculty who are actively involved in elementary teacher training programs, who teach different courses of elementary teacher education, need to have: For faculty members or in preparing of preservices … There needs to be content and there also needs to be pedagogy, there also has to be the ability to model. There also has to be awareness so that faculty are kept up to date with current practices. There also needs to be an understanding of the climate currently, an economic and political climate that education finds itself in, an understanding the context where our teachers are going to be placed and what realities they face, that they will face in their students' environments. (Faculty_individual interview_10) Making competent elementary teachers means making teachers who have a broad range of skills. These skills go beyond classroom skills and include a need to understand macro-economic and political concepts that influence education policy and acceptance of education by the public. Through deliberations the tuners created learning experiences 177 that promoted both long-term learning and appropriate recall and application beyond their Tuning meetings. The in-depth discussions of ETE Tuning supported faculty to be thoughtful, deliberate, and considered about their teaching practices. For instance, When I [faculty member - tuner] was looking at the tuned outcomes and deciding which ones I want to require them [students] to have an artifact for and which ones could be optional, I would use this similar process in my head and say, "Ok, which one is more important, which one are they most able to provide documentation for, not necessarily which one is easier to show, but which one are they likely to be able to provide a quality artifact for." That's where we were slicing and dicing again. (Faculty_individual interview_4) The faculty recognized that, although the outcomes are all important, physical demonstration of mastery is not possible for each skill that must be demonstrated. Thus, judging whether a student has mastered an expected outcome requires multifaceted evaluation and use of individual judgment by the faculty. Another example is a faculty member who had to rethink some of the things she did in her class to help students to better understand the vital need for having social studies in elementary school and wanting them to be able to articulate to a colleague, administrator, or a parent why they incorporated or integrated social studies into their classroom. She developed an elevator statement: As the semester progresses, they [students] keep adding to their elevator statement about why social studies is important. By the end of the semester, they've written a paragraph that's really succinct. They can say it in an elevator in 30 seconds to somebody to prove to them why social studies is important. It helped me [a faculty member] to help them [students] better articulate why that particular subject and curriculum [social studies in elementary school] is important in an elementary school. (Faculty_group interview_2) The prior quote helped establish that it is important for ETE students to understand the value of the curricula to their success; this one takes the next step by encouraging ETE students to think about, and succinctly articulate the importance of the subject matter that 178 they are imparting to their own students. The ETE tuners were working at developing learning-driven activities, evidences, perspectives that contributed to student learning and faculty learning and teaching, which in turn, contributed to quality of a college degree. Conclusion In presenting the findings, the chapter has answered the questions that guided this research: 1. How have faculty been engaged in the Utah Tuning Project? 2. How has the Tuning Project influenced Elementary Teacher Education in Utah? 3. a. Who provided the leadership direction for Tuning Elementary Education in Utah? b. What factors have been used to advance the Utah Tuning project? The Utah Tuning initiative was a large-scale effort that involved faculty representatives from all Utah ETE programs including public, and private four year and two year colleges and universities. The faculty tuners worked for almost two years to develop student learning outcomes and demonstrable competencies as a tool for evaluating preservice teachers, and by extension, provide guidance to ETE programs that would inform their curricula to achieve the desired outcomes. A high degree of collaboration among the faculty was a characteristic of the process. The faculty engaged the substantive work of Tuning by throwing out ideas and having some accepted and others challenged and ultimately rejected as they were all working for the betterment of the education system and the students. Faculty engagement in the Tuning process was limited by logistics, geography, 179 and faculty work load. The faculty tuners were from all of the colleges and universities in Utah and were therefore scattered throughout the state. The tuners met once per month with home assignments between meetings. This schedule was dictated by the fact that the tuners were all faculty members with full time positions in their respective colleges. Learning to tune took nearly five months, a significant portion of the time spent on the process. While the faculty were engaged in the process from day one, the fact that the tuners had never been exposed to the Tuning process limited early progress as it initially appeared that the outcome would look like any number of other sets of standards that could be used to evaluate teachers. The differences between Tuning work product and existing standards such as the UETS was subtle, but powerful and important. It took the tuners time to grasp the difference. The process was iterative. It involved deconstructing the existing standards and constructing new standards that were measurable and replicable. Often, the tuners found that the reconstructed product needed additional deconstruction to achieve the desired outcome. Each time a topic was revisited, it resulted in greater understanding and consensus among the Tuning team. In the end, the tuners produced a document, the UPTLOs, with a very high degree of consensus. Elementary Teacher Education in Utah is evolving to incorporate the results of the Tuning work. Some colleges and universities are incorporating the Tuning results into their pedagogy. Others are incorporating them into their curriculum. Weber State University has proposed the idea of an associate degree in education. The Tuning work would be used to help design the curriculum to align the training in the associate program with needs of the four-year institutions so that students can transfer between programs 180 without losing credits. The state Tuning leadership team was working with the USOE to incorporate the UPTLOs into the USOE teacher certification. This is an ongoing process. The state Tuning leadership team worked for the duration of the process as advocates for Tuning with a number of significant interested parties including the public, state government, the Utah higher education community, accrediting agencies, interstate policy boards, and other states wishing to implement Tuning. This work was necessary to gain broad acceptance for Tuning as a concept in higher education. The Utah Tuning project was advanced by a number of factors beginning with the existing infrastructure of higher education in Utah. The Regents' General Education Task Force and General Education Area Work Group together with the annual "What is an Educated Person?" conference and the annual Faculty Discipline Majors' Meetings had established the basis for collaboration among the colleges and universities. The state Tuning leadership team and the ETE Tuning team were also factors that advanced the project. Supporting the Tuning teams was the Lumina Foundation grant that provided funding for the Tuning effort. The colleges and universities cooperated in the project and individual faculty contributed their time and expertise to advance the project. Tuning is a cultural shift in higher education and thus will require many years to fully implement. The UPTLOs were completed at approximately the same time as the Lumina grant was fully expended; this was also the beginning of institutionalization of Tuning at the colleges and universities. The state Tuning leadership team prepared the colleges and universities for continuing Tuning after the grant. 181 Figure 4. Developmental, Dynamic, Complex, Complicated, and Iterative Process. orientation ETE Tuning work September 2011 ETE Tuning project started May 2013 ETE Tuning project ended April 2012 "a-ha" moment in ETE Tuning Developing ETE student learning outcomes and competences March 2013 Learning about Tuning and how to tune The state Tuning Leadership meetings with institutional teams d e l i b e r a t i o n and c o l l a b o r a t i o n May 2012 Interim Evaluation report Internal evaluation reporting process Grantee reporting process June 2011 May 2012 June 2012 Grantee Interim Narrative report May 2013 Interim Evaluation report June 2014 Final Evaluation report May 2013 June 2014 Grantee Interim Narrative report Grantee Final report 182 Figure 5. The Utah ETE Tuning Project: Key Points and Timeline. May 2014 183 Needs of students Student Needs of children in learning elementary outcomes schools Needs of employers Figure 6. The Bases for the ETE Student Learning Outcomes. 184 Discipline and Degree Specific Elementary Teacher Education discipline Bachelor's and Associate degree Main foci of the Utah ETE Tuning work was to: - Clearly articulate student learning outcomes and competencies within ETE for bachelor's and associate levels - Align all ETE programs across the state - Improve the students' mastery and its relevance to Elementary Education as a professional field U of U USU WSU UVU Westminster College SUU DSU BYU Figure 7. Tuning is Discipline and Degree Specific. SLCC WGU Snow College 185 Figure 8. Tuner's Whiteboard Exercise. Source: (USHE, 2011, December 2, p. 3) 186 Faculty learning what makes a teacher a teacher and Student learning to be competent teachers how to teach students to be competent teachers Figure 9. Tuning is Learning-Driven Process. 187 Table 4. School Principals' Expectations About ETE Graduates' Abilities and Skills. Student outcomes / skills reported by school principals How many principals listed this Classroom management 8 Assessment 6 Instructional planning and instructional 6 strategies Being collaborative and a team player 5 Differentiated curriculum 3 Content knowledge 3 Flexibility and adaptability 3 Lifelong learning 3 Effective communication skills 2 Vision 1 Strong whole group Tier 1 instruction 1 188 Table 5. ETE Faculty Understanding of Tuning. 2 Level of Knowledge Knowledgeable Partial Don't Know No Answer Total Responses 15 13 3 9 40 Percentage 38 32 8 22 Table 6. ETE Faculty Frequency of Discussions. 3 Never 1-2 per year 3-4 per year Monthly 2-3 per month Weekly No response Total Tuning Responses Percentage 2 5 3 8 14 35 3 8 1 3 0 0 17 43 40 Learning Outcomes Responses Percentage 0 0 3 8 5 13 10 25 4 10 1 3 17 43 40 Note. Adapted from "Utah Tuning Project Final Evaluation Report," by R. Davies and D. Williams, June 2014, Brigham Young University. Provo, UT, p. 6. 3 Note. Adapted from "Utah Tuning Project Final Evaluation Report," by R. Davies and D. Williams, June 2014, Brigham Young University. Provo, UT, p. 7. 2 CHAPTER 5 DISCUSSION Introduction The previous chapter provided the findings of a single case study of the Utah Elementary Teacher Education Tuning process. As stated earlier, the purpose of my single case study was to explore how the ETE faculty intentionally worked at developing and articulating student learning outcomes and competencies for a baccalaureate and associate college degree in Utah that took place during the ETE Tuning project. I analyzed the role of faculty and the role of the Utah Tuning leadership team and examined the principles and applications of Tuning methodology for Elementary Teacher Education. As illustrated, the Tuning case for baccalaureate and associate degree in Elementary Teacher Education in Utah was complex, dynamic, developmental, challenging, multifaceted, and complicated. These attributes represent a mosaic of Tuning as a phenomenon that aims to develop transparent pathways through articulated learning outcomes of a college degree in order to improve student learning and faculty teaching, and meet the new realities higher education is facing. The new realities include accountability issues, decreasing financial support for higher education, quality assessment, outcome-based, competency-based education, and improvement movement in general. 190 The purpose of the discussion chapter is to answer the "so what" question and discuss the lessons and issues that arise from the ETE Tuning process that will need to be addressed as we deal with the changes and challenges in higher education. The lessons fall in two categories: the state Tuning leadership and Tuning faculty expert work illustrating the complexities of the ETE Tuning process following implications and pointing the way to further research. This chapter presents the insights of this single case study through the institutional work analysis, attempts to create holistic and consistent integration of the findings of the research with literature, research, and practice. The chapter is organized in four sections: introduction with an overview of the findings and main foci of institutional work as an analytical lens of the study; second, leadership of ETE Tuning as agents of change; third, the Utah Tuning faculty expert work, and forth, implications of the research. Overview of the Findings In this research, I studied the Utah Elementary Teacher Education Tuning process aimed at developing and articulating student learning outcomes and demonstrable competencies for the ETE college discipline. The ETE Tuning involved all Utah colleges and universities and other actors, all of which were expected to accept an educational change process in order to achieve full implementation of the ETE Tuning work. Tuning as a methodology of articulating student learning outcomes and competencies "with its clear and consistent focus on student learning and development provides faculty, administrators and others with a guide for improving teaching and learning in academic programs" (Haworth & Conrad, 1997, p. 174). 191 Based on the findings, the Utah ETE Tuning process was a complex, multidimensional, and complicated process of creating new sets of rules and norms of articulating student learning outcomes and competencies and new rules to determine what ETE students must know, understand, and be able to demonstrate, which resulted in the Utah Preservice Teacher Learning Outcomes. This complexity reflected the Harsh's (2010) definition of complex problems based on "relationships and interrelated internal and external factors that require evolutionary change for successful resolution or organizational growth to occur" (p. 1). Outside the ETE Tuning process there were many stakeholders who determined the rules, policies, standards, and requirements for elementary education and, accordingly, elementary teacher preparation programs. Figure 10 (p. 265) visually presents ETE Tuning as a complex and complicated process. Based on the findings, inside the Tuning process, there was the faculty's work as experts aimed at developing student learning outcomes and competencies. The internal factors that required evolutionary change embraced the faculty's expert work with a focus on a college discipline and degree specificity. The faculty's expert work was a learning driven process including student learning-centered factor, faculty learning factor, faculty teaching factor, and faculty collaboration and collegiality factor. The faculty worked together to find common understanding and definitions for desired student learning outcomes and to find common, unambiguous language to convert that understanding to well-articulated statements of outcomes and competencies. It required that the ETE faculty engage in self-learning and required collaboration and collegiality. Within the collegiality there were many discussions, debates, and multiple reinterpretations needed before consensus was reached. As shown in Figure 10, the boundaries between the 192 internal and external factors were porous, with influence moving mostly from the external factors toward the internal factors. This was largely due to the state Tuning leadership team position of facilitation and support in creating the faculty Tuning team of ETE professionals, thereby limiting the influence of external factors and assuring that the act of Tuning would be a faculty driven process. The arrows in the figure illustrate the influence of external factors to the internal ETE Tuning work. The position of the state Tuning leadership teamwork in the figure is shown as between external and internal factors because the work of state Tuning leadership was aimed at providing organization and leadership of Tuning in the state. Evidenced in the research, the external factors were the USOE key role, school principals, accrediting agencies, and elementary education governing requirements and standards. The players external to the Utah ETE Tuning process were not necessarily in agreement with the goal of the Tuning process. The reasons for this were as varied as the players involved and included inflexible regulations, bureaucratic inertia, and skepticism, but underlying the reasons in most cases was a lack of understanding by the outside players of what Tuning was. Throughout the Tuning process the state Tuning leadership team was responsible for communicating with the outside players. The faculty tuners were not directly exposed to the external players, but as practicing professionals were indirectly exposed to, and very much aware of, their influence. The Tuning team made decisions, based in part on their knowledge of these external factors, such as the decision to use the UETS as the basis for articulating the Utah Preservice Teacher Learning Outcomes. This was a direct attempt to match the work of the Tuning team with the state requirements and standards governing elementary education. But, however much the 193 faculty Tuning team worked to accommodate the external factors, they displayed fidelity to their mission of improving ETE education by articulating learning outcomes and competencies. The ETE Tuning faculty work was a learning-driven process of articulating the specificity of Elementary Teacher Education programs using outcomes-based and competency-based descriptions, that is, specifically unfolding what was inside the ETE degree in Utah colleges and universities. Faculty learning as professional development in the Tuning process played a crucial role. These factors themselves represent complex and dynamic knowledge domains with a "rich" content. Tuning needed time for all involved in the process to buy into the process, understand its benefits, and reach full comprehension in order to accept it. The ETE Tuning experience as a whole was nonlinear process that demonstrated special complexity and complicatedness of practical ETE Tuning work, first, and, second, the implementation of its results at all levels: departmental, college, university, state. The ETE Tuning work as an expert process was set within a political structure for implementation that has yet to realize the full potential of ETE Tuning, and will not be able to do so until the USOE accepts the results. Generally, ETE faculty tuners' work as a core of this cross-state Tuning initiative was aimed to enhance quality of ETE college degrees. This overall goal to improve ETE college degree quality was being shaped through articulating the discipline and college degree specificity focused on studentcentered learning, outcomes-and competency-based approach. In order to explore the Elementary Teacher Education Tuning process in Utah, I utilized the concept of institutional work as an analytical framework. The following section presents the ETE 194 Tuning through an institutional work lens. Institutional Work Analysis This subsection overviews the main emphases of institutional work which, as noted in the literature review, is an emerging field of institutional theory. Institutional theorists (Greenwood, Oliver, Sahlin-Andersson & Suddaby, 2008; Lawrence, Suddaby & Leca, 2006) have defined institutional work as individuals and organizations' purposive actions designed to create new institutions, maintain, or disrupt existing institutions. Thus, the concept of institutional work focuses on the actors' intentional actions within existing institutions (structures, formal rules, norms) (Battilana & D'Aunno, 2009; Hargrave & Van de Ven, 2009; Kraatz, 2009; Zilber, 2009). Accordingly, institutional work is closely connected with the notion of institutions, which is central to institutional theory. Institutional theory, being a form of an organizational analysis and a predecessor to the concept of institutional work, defines institutions as organized and established structures and procedures comprised of "normative rules," "regulatory processes," (Scott, 1995, p. 34) and shared meanings that define relationships among actors (Scott, 1995; 2003). Jepperson (1991) described institutions as "taken-for-granted," "culturally embedded" understandings which require and explain formal and informal behaviors and actions. According to Jepperson (1991), institutions are "‘enabling structures,' or social ‘programs,' or performance scripts'" (p. 145). He described them as metaphors that "connotes stable designs for chronically repeated activity sequence" (Jepperson 1991, p. 145). For instance, the academic discipline was among his examples of institutions 195 (Jepperson, 1991). Education is another example of an institution in Meyer's (1977) The Effects of Education as an Institution, where education was seen as "a system of institutionalized rites transforming social roles through powerful initiation ceremonies and as an agent transforming society by creating new classes of personnel with new types of authoritative knowledge" (p. 56). Furthermore, Meyer (1977) argued that education denoted "a set of institutional rules which legitimately classify and authoritatively allocated individuals to positions in society and institutional impact of education on societal structure itself - on behavior of people throughout" (p. 59). In my research, I applied the definition of institutions as structures, formal rules and norms, which are not equivalent to organizations. In institutional theory, it is institutions (structures, formal rules, norms) that guide interactions, social positions and interpretation of the actions of others within a social system. This contrasts with the institutional work concept which focuses on actors' purposive roles in creating, maintaining, and disrupting institutions. Researchers (Greenwood, et. al, 2008; Lawrence & Suddaby, 2006; Zilber 2013) have articulated three main foci in the institutional work research, namely, the who, what, and how of institutional work. First, the "who" question studies the main actors that accomplish the construction of new structures, rules, and norms, that is, work on institutions. Suddaby and Viale (2011), Empson, Cleaver, and Allen (2013), Singh and Jayanti (2013) have investigated the interplay between institutional work and professions, and established that main actors could be a wide range of actors with resources and skills, including leaders, professionals, and nonprofessionals. Second, what constitutes institutional work? What is the role of actors in 196 accomplishing the construction of rules, structures, and norms? Are they passive, receptive, or are they active agents of change with capacity to act through intentional actions? Scholars (Battilana & D'Aunno, 2009; Zundel, Holt & Cornelissen, 2012) have examined the question of what makes up institutional work. Their studies debated "structure-agency" relations and focused on agency, or the ability of those engaged in institutional work to act as agents of change. In the studies of Giddens (1984) and Sewell (1992), structures were presented as both the medium and outcomes of social practices; structure and agency always coexist in continuous dynamic relationships. Third, how does institutional work occur? How do the actors accomplish their roles? The answer is in three micropractices: creating, maintaining, and disrupting institutions (Greenwood, et. al, 2008; Lawrence & Suddaby, 2006) each of which is shaped through various forms of institutional work. These three keystones of institutional work are closely interrelated and are in continuous evolution (Hardy & Maguire, 2008; Roberts, 2008; Sahlin & Wedlin, 2008; Schneiberg & Lounsbury; 2008). Overall, the concept of institutional work examines the role and agency of institutional actors and how they accomplish their work. Essentially, work on institutions recognizes the importance of change. I analyzed the ETE Tuning through the institutional work concept which permitted me to explore the faculty's intentional work aimed at developing and articulating student learning outcomes and competencies and the role of the state Tuning leadership team who acted as entrepreneurs in the Tuning process. For the Utah ETE Tuning process, both the entrepreneurial work of the state Tuning leadership team and the expert work of the Tuning faculty was consciously designed to create new institutions of 197 Utah ETE education by determining what students must know, understand and be able to demonstrate to become a preservice elementary teacher and define new rules to assess student learning outcomes. Overall, the two teams were purposely engaged to lead related aspects of institutional work, that is, the work of entrepreneurs, and the work of experts. The work of the two teams led to a natural division of the forms of institutional work employed by each team: with the state Tuning leadership team acting mostly through articulating, advocacy, and constructing normative networks, and the ETE Tuning faculty team acting mostly through participatory learning, defining the discipline core, and overall driving the ETE Tuning work. The Who, What, and How of Utah ETE Tuning at the state level is illustrated in Figure 11 (p. 266). As shown on the figure, there were two main teams, the state Tuning leadership team and the ETE Tuning team, symbolized by two circles. The teams worked alongside one another in the Tuning process, with certain interaction, and provided complementary work for each other. The state Tuning leadership team provided agency (Bandura, 2001; Musolf, 2001) in leadership and organization of the Tuning initiative. The faculty Tuning team filled in the process with their experts' work using their functional and operational expertise to develop student learning outcomes for preservice teachers that resulted in the Utah Preservice Teacher Learning Outcomes. Finally, the teams accomplished their work utilizing various "practices" (Dacin & Dacin, 2008; Davis & Anderson, 2008; Fiss, 2008; Lawrence & Suddaby, 2006; Rao & Kenney, 2008) of institutional work. The state Tuning leadership team was engaged in defining the composition of, and facilitating the work of the faculty Tuning team, providing for the education of the faculty tuners and constructing normative networks (Greenwood, et. al, 198 2008; Lawrence & Suddaby, 2006) through advocacy. The ETE faculty Tuning team was engaged in "participatory learning," (Lane & Wenger, 1991) defining the discipline core, fitting Tuning into the state requirements, and overall, driving the Tuning work. The dashed lines symbolize that all parts are interrelated with each other. The arrows directed to the institutional work of Tuning display external factors for the process. The arrows that go outside illustrate the connection with the ETE departments in all Utah colleges and universities. Figure 11 (p. 187) displays a series of concentric circles radiating outward from the ETE Tuning institutional work. As one faculty member put it: "[M]y metaphor was a pebble in the pool when you toss the pebble, and then the ripples go out, and out, and out" (Faculty_focus group interview). The shading in the figure symbolically depicts a ripple. Developing and implementing ETE Tuning is akin to tossing a pebble into a pond. This metaphor applies to both the institutional work of the faculty tuners and the institutional work of the state Tuning leadership team. Utah ETE Tuning mostly contained two main processes as a continuous chain of events: the ETE Tuning faculty work and the state Tuning leadership team work. As noted earlier, both were aimed at improving elementary teacher education in Utah through Tuning. The internal life of these processes was filled with faculty tuners' intelligent work and that of the state Tuning leaders. The faculty tuners' work and the work of the state Tuning leadership team were the intellectual forces of the whole Tuning process. Based on the overview of institutional work analysis, namely, purposive actions of individuals and organizations, I will now move to the "so what" of the ETE Tuning purposive process. Specifically, the following sections present the lessons learned from 199 the research through institutional work as an analytical lens of this research. They show that the state Tuning leadership team was mostly engaged in a series of events and activities designed to facilitate and support the work of the ETE faculty tuners as an expert team and to promote the implementation of the Tuning reform throughout the ETE discipline in Utah. This institutional work evolved as the leadership team moved from securing grant to consultation with employers of ETE students to forming the ETE Tuning team, to communicating with and educating external agencies, to overall organization of the Tuning process in the state. This was a continuous chain of events designed to result in an outcome. Leadership and ETE Tuning: Agents of Change Based on the findings, the Utah ETE Tuning effort was supported by twin pillars of leadership and experts' work. Both pillars were absolutely necessary to make Tuning work, but neither pillar alone was enough to support the project. As expressed by one of the faculty tuners: [E]ach party played their role very well. USHE and the Board of Regents, I think, their role was of organizer, cheerleader. They kept the process moving. Members and those in higher education's role were more supporting it [Tuning] and implementing it [Tuning]. (Faculty_individual interview_10) This reflection explains that Tuning leadership and faculty work accompanied and contributed to the Tuning process. This is how another faculty tuner described it: "an important realization that Tuning may have come from the top but it won't work unless the faculty actually get together and do the work" (Faculty_interview_8). This demonstrates that without the professional work of the ETE Tuning faculty team, the state Tuning leadership team would have no results to present, and without the state 200 Tuning leadership team, the faculty Tuning team would likely have never been able to move the process forward. Another faculty tuner stressed upon, "If a bunch of university professors had gotten together and decided to do this on their own, it wouldn't have carried the weight" (Faculty_individual interview_5). This illustrates that the ETE faculty tuners recognized and valued the role of the state Tuning leadership team who were agents of change in the process, focusing on student learning. In higher education: When faculty and administrators are committed to student learning and development, they take two tasks seriously. First, they know that learning … is fundamental to what they do and who they are as educators. This overriding clarity of purpose provides faculty and administrators with a linking pin that incorporates the learner and the learning process fully into program planning and evaluation efforts. Second, a clear and consistent commitment to students' learning keeps faculty and administrators attuned to the needs and expectations of those whom they directly serve: students and employers. (Haworth & Conrad, 1997, p. 168). The gist of the problem raised by Haworth and Conrad (1997) requires consistent work of leaders and professionals. Institutional work scholars (Empson, Cleaver, & Allen, 2013; Singh & Jayanti, 2013; Suddaby & Viale, 2011) demonstrated that leaders provided entrepreneurial skills to initiate, guide, support and facilitate the change process. At the same time, leaders doing their entrepreneurial institutional work require the support of professionals, who are also doing institutional work. Similarly, the Utah ETE Tuning project required two main groups of actors: professionals, and leaders. According to Scott and Christensen (1995), "institutions do not ‘just grow.' They must be constructed and maintained as well as adapted and changed" (p. 303). This type of leadership was present and mattered in the ETE Tuning process. In fact, the state leadership made the ETE Tuning process possible. However, the state Tuning leadership team could not have been successful as the only actor. Importantly, the ETE faculty tuners made the process 201 work. The Tuning leadership and ETE faculty tuners demonstrated "collective intelligence and commitment" (Haworth & Conrad, 1997, p. 169) into the process of Tuning a college discipline. Both teams were deeply engaged in the process doing their key leadership and critical faculty work respectfully. In institutional work the focal role of actors is central (Battilana & D'Aunno, 2006). The studies of program quality in higher education (Bowden & Marton, 1998; Harvey & Knight, 1996; Haworth & Conrad, 1997; Tierney, 1998) emphasize the idea of engagement and commitment of all players involved in this field. For example, the engagement theory is organized around "student, faculty, and administrative engagement in teaching and learning" (Haworth & Conrad, 1997, p. xii) as the central idea, and highlights "the pivotal role" of the faculty, students and administrators "in fostering mutually supportive teaching and learning in programs of high quality" (p. xiv). Like the engagement principle, the state Tuning leadership team employed a variety of practices to advance the ETE Tuning project. For example, they defined the ETE Tuning team composition and the process in which it would work. They facilitated the work of the faculty Tuning team, and they were constructed normative networks for Tuning by advocating for Tuning with outside actors as well as with the colleges and universities to provide a receptive audience for the Tuning results. As evidenced in this research, the state Tuning leaders demonstrated understanding of their potential to influence ETE Tuning through agency as "the ability to adapt to their environment, to change the institutions that shape them" (Musolf, 2001, p. 278). DiMaggio (1988) emphasizing the role of actors in institutional changes, wrote: "New institutions arise when organized actors with sufficient resources (institutional 202 entrepreneur) see in them an opportunity to realize interests that they value highly" (p. 14, italicized text in original). The concept of agency invokes such notions as will, intentionality, motivation, capacity, interest, choice, freedom (Battilana & D'Aunno, 2006). Agency implies that actors' actions are conscious, reflective, intelligent, and leaders' characteristics are applicable to the Tuning process to construct the opportunities for college discipline improvement. The state Tuning leaders were conscious of the value of the Tuning methodology for a college discipline, and believed that it was their responsibility to improve the quality of higher education in the state by intentionally working to transform it. One of the state Tuning leaders described her role: My role was to make sure that people understood Tuning, the value of Tuning. How fortunate we were to tune, how successful we were in terms of the process of Tuning. When you talk about the changing the culture of higher education, you are talking about changing how universities and colleges function, how their colleges and departments within the big structure of an institution, and that takes more work, and that's over time. (State leader_individual interview_3) This displays that state Tuning leaders work as agents recognizing the specific nature of higher education. This quote exposes that changes in higher education takes a lot of work, efforts, and time. Implicitly, the reflection deals with challenges. Ewell (1998) in Achieving High Performance: the Policy Dimension recommended among many other things to define consistent public agendas, establish incentives for cooperation among universities, and build in "a larger vision of what society as a whole wants its higher education system to accomplish" (p. 158). He accentuated that real transformation required "a simultaneous effort …, changes must be synchronous and mutually reinforcing" (p. 122). This, in turn, constitutes "a major challenge for leadership, both within higher education and among policymakers committed to its support and 203 development" (p. 123). As related to institutional work, institutional entrepreneurs are: [A]gents who deploy the resources at their disposal to create, alter, and empower institutions. Such actors serve as agents of legitimacy who support the creation of new institutions and reform existing institutions in ways that they deem to be appropriate and aligned with their interests. (Greenwood, Oliver, Sahlin, & Suddaby, 2008 p. 633) As this analysis demonstrates, the state Tuning leadership acted with a sense of agency, namely, behaved "strategically, sometimes conforming but often negotiating, protesting, resisting, and hiding from the dictates of regulatory and symbolic systems" (Scott, 1995, p. xxi). The state Tuning leadership displayed their agency through numerous characteristics including their ability to initiate ETE Tuning through obtaining the Lumina grant for Tuning in Utah, support faculty work, their capacity to influence the expectations and guide the process, and their efforts to assure that all colleges and universities are involved in the process. The following subsections will discuss how the state Tuning leadership team and faculty team, specifically, accomplished their institutional work, for instance, in creating the ETE Tuning team. Creating the ETE Tuning Team The research findings revealed that the state Tuning leadership team defined the ETE faculty Tuning team as an ad-hoc group formed for the sole purpose of developing and articulating student learning outcomes and competencies for ETE programs at the baccalaureate and associate level. This reflects Tuning leaders' institutional work as "practical-evaluative" (Emirbayer & Mische, 1998, p. 994). Each college and university in Utah was asked to appoint a representative, or representatives to the Tuning team. One state Tuning leader described the process: 204 In order to get all this [Tuning] off the ground, the chairs of those committees [Tuning committees] took really the most of the activity. We identified faculty who were able to provide that [Tuning] through a list of people, a list of faculty who attended the Faculty Discipline Major's meetings which are now in the 17th year. (State leader_individual interview_3) The state Tuning leadership team made a conscious decision about the composition of the ETE Tuning team based on their judgment of who were needed to best help the community progress. This reflection unfolds that Tuning leaders responded "to the demands and contingencies of the present" (Emirbayer & Mische, 1998, p. 994), which discloses the leaders' capacity to make practical and normative decision. This demonstrates that the state Tuning leadership team defined the composition of the ETE faculty Tuning group as a "community of practice" (Lane & Wenger, 1991). Horn (2005) and Wenger (2010) studied composition issues in communities of practice; who should participate in a community of practice. The composition of a community of practice is intertwined with its purpose. In some instances, the composition of the community partially defines the purpose. In the case of Utah ETE Tuning, the purpose of the team was defined in advance and, therefore, was used to define the composition of the team. The Utah Tuning leadership team decided that an ETE Tuning team composed almost exclusively of ETE faculty (only one student representative and a representative from the USOE) would be best qualified to accomplish the Tuning work. The state Tuning leadership team defined the ETE Tuning team membership by composing the team of the ETE professionals as experts, and excluding others such as school districts, the public, politicians, and other groups who routinely participate in efforts at public education reform. However, the selection of representatives was not consistent among colleges and 205 universities with some sending department heads and others sending somewhat junior or clinical faculty. One faculty tuner said: "I came into the Tuning project by accident. I attended a state office meeting where there were other institutes of higher education faculty members from all over Utah came and met, and [redacted] was there (Faculty_individual interview_07) . Another example, one of the universities initially sent a faculty member who did not fully support the Tuning process and actually stopped attending the meetings. So, while the state Tuning leadership work was purposeful, not everyone's was. One faculty tuner recalled: The Board of Regions posted it [Tuning] and they asked for representation from every institution. The faculty member who is no longer part of our faculty was assigned to be our representative. That faculty member was also working with the State Office of Education simultaneously on the new teacher evaluation based on the UETS. The UETS was being drafted, and this faculty member was serving on that committee for the teacher evaluation portion, observational protocol portion of UETS. He'd also been asked to attend the Tuning meetings representing [redacted] but decided personally that Tuning was redundant. That it represented the left hand didn't know what the right hand was doing, that Board of Regions was doing one thing, and the State Office of Education was doing something else. Even though there was a State Office of Education representative also on the Tuning committee, and that person quit attending Tuning. (Faculty_Focus group interview) This resulted in a call from the Board of Regents to the provost of the university to ask in effect: "Why isn't anybody coming from [redacted]? Have you guys dropped out of the process?" (Faculty_Focus group interview). Following that call, a new representative was assigned to the Tuning team who was an active contributor to the process. According to Collins (2001), "it is one thing above all others: the ability to get and keep enough of the right people" (p. 17). From institutional work analysis, through defining the ETE Tuning team membership the state Tuning leaders created a new structure with certain rules and 206 norms. Institutional work theorists (Lawrence and Suddaby, 2006) described defining as "the construction of rule systems that confer status or identity, [or] define boundaries of membership… within a field" (p. 222). In case of ETE Tuning, the rule systems to "define boundaries of membership" were not specifically defined. This reveals that the unstructured appointment process resulted in some disconnects between faculty tuners and their home departments. It proves that "it is people that matter - people make quality happen" (Haworth and Conrad, 1997, p. 171). Once it was determined that the ETE Tuning team would be composed of ETE faculty, selecting the faculty representatives to the ETE Tuning team was a partially unstructured process that ultimately worked to limit the effectiveness of the team in two ways. First, when it came to representing the university on the Tuning team and, second, when it came to carrying the Tuning message back to universities and colleges. It also relates to the process of institutionalizing Tuning at the colleges and universities after the expiration of the Lumina grant for Utah Tuning. Facilitating the ETE Tuning Teamwork The state Tuning leadership team not only defined the ETE Tuning team, they fully engaged in facilitating the work of the Tuning faculty team. "Support for faculty stands out as yet another important attribute for high quality programs" (Haworth & Conrad, 1997, p. 150). Related to institutional work, "the formation of projects is always an interactive, culturally embedded process by which social actors negotiate their paths toward the future" (Emirbayer & Mische, 1998, p. 984). The first act of facilitation was to secure the Lumina Foundation grant, which in turn, made the work of the ETE Tuning 207 faculty team, as well as the work of the Tuning leadership team possible. As this analysis demonstrates, the state Tuning leadership team facilitated the Tuning process through their efforts and involvement in "the educating of actors [faculty tuners] in skills and knowledge necessary to support the new institution" (Lawrence & Suddaby, 2006, p. 227). Allied with institutional work, the ETE Tuning process, as the process of creating new rules for student learning assessment, involved "the development of novel practices as well as connecting those practices to control mechanisms" (Ibid.). Accordingly, Tuning work required the faculty tuners to acquire significant new understanding and skills. In order for the ETE faculty tuners to develop "novel practices" of articulating student learning outcomes in measurable and assessable terms, the state leadership team provided an orientation workshop and brought experts in the field prior to the Tuning work. The state Tuning leaders also provided two paid consultants who informed the faculty tuners about the work and experience of other Tuning subject (history, physics, and general education mathematics) groups and provided evaluations throughout the Tuning process. In addition to these formal education steps, the faculty tuners received indirect education through the many publications and presentations by the state Tuning leadership team and also attended the annual "What is an Educated Person?" conference. Educating of the Tuning participants was cognitive work of great importance for the ETE Tuning experts' work happened. This was essential for institutional work because educating and learning is "a source of institutional change" and progression where "selective and inferential learning processes, both within and across fields, produce institutional change" (Haunschild & Chandler, 2008, p. 643). One faculty tuner 208 described the important facilitating and supporting role of Tuning leadership team in the state: For the State Board to call those meetings and make it possible for people to get together. There were consequences that our [faculty's] conversations mattered. Then with the introduction of the Tuning Process, that again, getting us together, bringing in experts who could talk about it, and also helping to get us to Indiana and various exotic places to have this other conversations. There's no question that the State Office becomes a very important facilitator, a very important supporter. (Faculty_individual interview_8) All this demonstrates the efforts of the state Tuning leadership team in assuring that faculty "on the same page" as regards their expectations to develop student learning outcomes and demonstrable competencies for a college discipline. Explicitly also, the faculty tuners recognized that the support was the work of change agents who made the Tuning process possible in the state. The faculty are the key resources for the new reality of American higher education, and, accordingly, for the ability of the colleges and universities to become responsive. Therefore, educational leaders "must do a great deal to support faculty efforts to learn about learning and teaching" (Chaffee, 1998, p. 32). Supported by the findings from my research, the state Tuning leaders displayed their agency through understanding that faculty members are critical in developing student learning outcomes: Who knows the value to students? Working directly with the faculty to listen to what they were doing, and what they were struggling with, what they understood, what they tried to do was extremely important factor [to advance Tuning in the state]. (State leader_individual interview_3) The leader believes that success of Tuning requires faculty being critical and state Tuning leaders being supportive. The attitude of the state Tuning leaders is specifically noted as positive and appreciative to the ETE Tuning chairs: The people really taking the lead there were chairs … we did change leaders at 209 the beginning … When Dee took it over, Dee was extremely well suited to leading the efforts. Sylvia, when she came back, was an incredible team member, just incredible. She had great ideas, and she got to interpret them, and there was a sense of humor, and all of that. (State leader_individual interview_3) The Tuning leadership team also acted as facilitators for the ETE Tuning team by providing stipend for faculty tuners, and funding their travel. As this analysis reveals, the state Tuning leaders demonstrated their appreciative attitude and position to faculty's work. Accompanying institutional work, state Tuning leaders' agency was "embedded process of social engagement" (Battilana & D'Aunno, 2006, p. 47). One Tuning leader expressed this: We were asking faculty to take on quite a bit. That's on top of their full time teaching jobs, committee and so forth. To recognize faculty was extremely important too which is what we did. It's about faculty, what faculty does. Who knows the value to students? We did certainly recognize the faculty, and we did give them stipends which they very much deserved, and they became the spirit of the core because they understood how they worked with one another; they enjoyed coming together; they worked with one another to work through issues. (State leader_individual interview_3) These exhibit the state Tuning leadership mindset and energies in doing their institutional work. They acknowledged faculty's work as critical in improving quality of higher education through developing transparent learning outcomes for students. The state Tuning leadership also acted as facilitators for the ETE Tuning faculty by reserving venues, and scheduling meetings, seemingly trivial functions. Facilitating also included making sure that the faculty tuners were at least minimally supported by their home college or university in their Tuning activities, and helping to construct the networks needed to advance Tuning in Utah. Without this support and facilitation, the ETE Tuning team would not have been able to accomplish its professional Tuning work. These activities, some large and others seemingly trivial, all required the state Tuning 210 leaders to act as agents of change. Without this institutional work by the state Tuning leadership team, the process would not have moved forward. The institutional work analysis demonstrated that through the Tuning process, the state Tuning leadership maintained the inclusion of all higher education colleges and universities in the state to work towards higher education quality improvement. In institutional work, maintaining "involves supporting, repairing or recreating the social mechanisms that ensure compliance" (Lawrence & Suddaby, 2006, p. 230). Higher education leadership in Utah has its history of gathering faculty state wide once a year at the Faculty Discipline Major's meetings and at the annual "What is an Educated Person?" conference, which provides for faculty members and administrators to discuss the higher education issues across the Utah colleges and universities. Supported by the findings, in the case of ETE Tuning maintaining of the social mechanisms (Major's meetings and the state higher education conference) was done very intensively and with a focus on student learning and learning outcomes. Importantly, my research findings revealed that the state Tuning leadership keeps on performing an institutional maintenance task by setting the agenda and providing avenues for continuing Tuning dialogue after the expiration of the Lumina Foundation grant at the annual Discipline Major's meetings and "What is an Educated Person?" conference. As one tuner observed: I think it's been interesting to see what's happened with the Educated Person conference and the process that's gone through. Tuning has been very much now part of that. I think to the extent that [the ‘What is an Educated Person?' conference] sustains a statewide conversation, but also reiterates how important this [Tuning] is, and emphasizes again that any kind of enrichment in education is both a top down and a bottom up process. (Faculty_individual interview_08) This reflection exposes that members of the Tuning team recognize the need and value 211 for maintaining of the discipline Tuning process, and that the state Tuning leadership team has worked to provide avenues for professional deliberations and supporting Tuning. The following subsection discusses how the state Tuning leadership team was constructing normative network for the Tuning process in Utah. Constructing Normative Networks The institutional work of constructing normative networks was defined by Lawrence and Suddaby (2006) as "the interorganizational connections through which practices become normatively sanctioned and which form the relevant peer group with respect to normative compliance, monitoring and evaluation" (p. 224). As stated in the reports (USHE, 2013, 2014), the state Tuning leadership team worked with the administration of all higher education colleges and universities in Utah to disseminate the Tuning methodology and to lay the groundwork for continued Tuning work after the Lumina grant expired. The state Tuning leadership team shared their understandings of Tuning direction "by communicating it to internal and external audiences" (Haworth & Conrad, 1997, p. 33). They demonstrated their responsibility for constructing the normative networks needed to advance Tuning and to set the stage for translation of the Tuning results into ETE policy and infrastructure. The state Tuning leadership team understood of the importance to work with authoritative bodies. One state Tuning leader underlined: You always have to bring your upper administration on board, at least to try to sell the idea. That, I think, is very important. I think we did that. I think I did it mostly because of I was the one with the standing. So, I think that was very important factor. (State leader_individual interview_3) This reveals that the Tuning leaders also intentionally worked with educational 212 administration at different levels. The reports (USHE 2012, 2013, 2014) present an array of communications such as: meetings, presentations, conferences, publications, and interviews in the state media, with various educational representatives of all levels of administration. These interactions between the Tuning project and the larger education community were part of the entrepreneurial work of the state Tuning leadership team that concentrated on vision, facilitation, and guiding the process. Keith (1998) stressed on the importance of external relationships for enhancing quality of higher education and suggested partnerships with government "to transform institutional performance so that it is aligned with public purposes" (p. 167). Braskamp and Wergin (1998) in Forming New Social Partnerships emphasized that higher education has a unique opportunity to contribute to the society: Academic leaders thus need new skills as reality shapers. They need to be able to sit beside their internal and external colleagues (faculty, students, citizens, politicians) in ways that persuade one and all to be clear about what higher education should and should not deliver. (p. 86) In Utah, the state Tuning leadership team was constructing networks for a process through advocacy which would ultimately contribute to the improvement of ETE in the state and thereby to society in general. In institutional work analysis, advocacy is part of creating institutions (Lawrence & Suddaby, 2006). The research revealed that advocacy by the state Tuning leadership team encompassed activities to meet with and advocate for ETE Tuning with accrediting agencies, political leaders, and even the colleges and universities who sent representatives to the ETE Tuning team. Lawrence and Suddaby (2006) defined advocacy as "the mobilization of political and regulatory support through direct and deliberate techniques of social suasion" (p. 221). The Utah ETE Tuning project was a small establishment, lacking in political power and legitimacy. 213 Consequently, the state Tuning leadership team intentionally worked to "actively shape their institutional environment and… acquire cognitive legitimacy" (Lawrence & Suddaby, 2006, p. 222). Likewise, Wenger (2010) emphasized the interactions with external stakeholders and their role in the process for providing resources to sustain a productive partnership. The state Tuning leaders had many interactions about the state Tuning work to build regulatory and political support. One state Tuning leader said: You need to make sure that accreditation, we are North West Commissioner on Colleges and Universities, is here. We made sure that they got materials. Whether North West understood it or not is another question. I had a discussion with their president, and she didn't know, it was very surprising. But I think it's atypical. She continued: The larger organizations where the state higher education executive officers belong to, and CEO belongs to, were informed. Presentations were made at all those levels. So, you had outside groups. They are not outside of higher ed. but they work at the other things with top administration. We had them informed too. I did a lot of speaking. I think Bill did speaking. And both Dan and Norm did speaking in American Historical Association. I did for SHEEO [State Higher Education Executive Officers Association], for WICHI [Western Interstate Commission for Higher Education], for AAC&U [Association of American Colleges and Universities]. (State leader_individual interview_3) Explicitly, advocacy was a vital part of institutional work for the state Tuning leadership team who were using it to create new normative networks, institutions needed to support ETE Tuning. Tuning would require the reallocation, and, perhaps, the creation of new social and political capital to convince educators, regulators, practitioners, and employers that Tuning represented an improvement in the preparation of elementary teachers. Additionally, the Utah Tuning leadership team was advocating for a process with various powerful entities outside of higher education, some of which have direct power 214 over institutions of higher education and others, which have indirect power. Their work with the Utah State Office of Education was directed at helping the USOE understand the nature of Tuning and why it would improve elementary education. As the organization that licenses teachers in Utah, the USOE has indirect power over the fate of ETE Tuning in Utah. Also, the Tuning leadership team worked with the state legislature that holds budget authority over publicly funded higher education institutions in Utah and has direct interest in the activities of the USOE. Regardless of how these entities exercise their power, they were important constituencies, and the Utah Tuning leadership team were tireless advocates for Tuning with these entities. As one member of the state Tuning leadership team said: I mean we were talking everywhere and anywhere, and so you had these outside groups who came to understand it [Tuning] and respect it as well. I thought that there was important too. That lent creditability to the efforts. It's also a kind of support. To communicate the idea [of Tuning], to use any venue to do it, any excuse to do it, we all did it. People need to hear the idea multiple times and present it in a variety of ways too. To get the message, it must be presented in a number of ways. (State leader_individual interview_3) The core of the problem raised in this reflection relates to the crucial role of advocacy in constructing normative network. The difficulty of advocacy as institutional work for ETE Tuning in Utah is best understood by noting that none of these institutions (USOE, legislature, and accrediting agencies) have the power or authority to guarantee acceptance and success of the ETE Tuning project, but all have a large measure of power to impede its success, and all had to be exposed to the process multiple times, and in different ways. However, advocacy was not limited to the state Tuning leadership team, but also included advocacy by individual faculty tuners in their departments that resulted or did not result in moving the departments forward to embrace the Tuning outcomes. Based on 215 the findings of the ETE Tuning process investigation, the departments with strong advocates were more receptive to implementing Tuning, and moved forward with greater rapidity than the departments that did not have strong advocates. Kolb et al. (2013) and Jones (2012) also emphasized that advocates were needed across the educational community to advance Tuning. For example, the ETE department at Weber State University delegated the head of the department and another faculty to the Tuning team. As a result, Weber State University took almost immediate steps to implement the outcomes for preservice teachers and later began developing a two-year associate degree program. Another example, Salt Lake Community College had a parallel Degree Qualification Profile initiative focused on improving higher education, so they were very receptive to the idea of Tuning as a reform. Salt Lake Community College connected Tuning with their Degree Qualification Profile. One faculty tuner shared: At the conclusion of the Tuning project, SLCC EDU [education unit] faculty were invited to participate in the DQP Project. I connected DQP with Tuning indicators as a way to increase effective EDU course outcomes. This "twist of fate" brought Tuning into a new light in the department. (Faculty_individual interview_09) These examples explain that the state Tuning leadership team and the faculty tuners who demonstrated a sense of agency were both acting as agents of change, complementing each other's work. This, in turn, solidified Tuning as a new normative network in Utah. The state Tuning leadership was constructing Tuning process and faculty tuners were filling in ETE Tuning as a new structure with their purposive work on articulating learning outcomes for students. Implicitly, these examples sit in stark contrasts with the USOE that experienced turnover in their representation on the Tuning team and did not adopt the UPTLOs as a measure of student learning outcomes and competencies for 216 preservice teachers. The state Tuning leadership team were constructing the normative network and environment in which ETE Tuning could take place. Their activities included advocacy as a form of institutional work, and advocates for change are, by definition, agents of change. For example, the ETE faculty Tuning team itself was a new statewide normative network. One state Tuning leader believes in changing the culture of higher education through continuous advocacy. She accentuated: [I]f we do nothing, then we get nothing. If we do these, we continue to talk about it, we continue to practice it [Tuning] no matter in what form, then over time we begin to change the culture of higher education. So, Tuning is number one more student-centered, number two more intentional in terms of teaching, learning, competencies, outcomes and assessment. You know, are these assessments worthwhile? Do they really move students to the next level? But without that, you don't change the culture. If you do nothing, you have nothing. (State leader_individual interview_3) This demonstrates how the state Tuning leaders being highly involved in the Tuning process are strong and focused on improvement of higher education in the state. Their mindset and efforts aim to do things better because they see the value and benefits of implementing Tuning into college education. The importance of constructing normative network is also the main thread of this reflection. Clark (1998) studying organizational pathways of transformation into entrepreneurial universities underscored: Collective entrepreneurial action at these levels is at the heart of transformation phenomenon. Acting from on-high, national and state systems of higher education are blunt instruments of significant change; acting from below, individual faculty members or administrators are limited in what they can do. But groups, large and small - central and departmental - of faculty and administrators (and sometimes students!) can fashion new structures, processes, orientations whereby a university becomes biased toward adaptive change. (p. 4) The institutional work analysis further demonstrated that the state Tuning leadership team engaged in efforts to disrupt the silos or compartmentalization of ETE 217 faculty in academia, in general. The institutional work theorists (Lawrence & Suddaby, 2006) formulated that "disrupting institutions involves attacking or undermining the mechanisms that lead members to comply with institutions" (p. 235). The state Tuning leadership undermined the silos by creating the ETE faculty Tuning ad-hoc group, which involved the representatives of all Utah colleges and universities and defined their interactions as monthly. This disruption, in turn, led to the creation of a new structure, the Utah ETE Tuning faculty team, which had its own goal, objectives, deliberation subject, meeting schedule, and the emerged state Tuning website. When the ETE faculty tuners completed their Tuning project and effectively ended the need for the intensive phase of their Tuning work and monthly meetings across the discipline, there occurred the "reverse" disruption of the ETE Tuning faculty work. That led the faculty to revert to the annual Faculty Discipline Major's meetings as the norm for cross state faculty interaction. One faculty member observed that an annual Major's meeting would not, in her opinion, be adequate to maintain momentum: The Major's meetings meet once a year, we have a tendency to fall off our radar about different things. If we made it [discipline meetings] twice a year, that we'd have more of a chance to follow through on some things because we get a really good discussion going on about different things that we concern about in teacher ed. But then it [discussion] tends to fall off our radar if we don't see them [faculty tuners] for eleven months. I think my preference is always to touch base with people on a more regular basis. I think that's how change happens rather than expecting people to do it on their own. I think people … it's easier for people to change if they have support to do it. (Faculty_individual interview_01) In this reflection, the faculty tuner voices dissatisfaction with rare meeting of faculty members across their discipline. This explains that the faculty tuners valued their professional meetings and collaboration and desired that to continue because regularbasis meetings would let the faculty deliberate and work out the solutions to benefit their 218 college students and college discipline together. This faculty tuner has expressed her desire to have the state Tuning leadership provide for maintenance of the Tuning conversations by arranging for a second annual faculty meeting to involve all ETE programs in Utah in this dialogue. The following section discusses the practices of the ETE faculty tuners as critical in the Tuning process through institutional work lens and discusses how it is connected with a bigger picture of quality models in higher education. The Utah ETE Tuning Faculty Expert Work In this section, I will discuss what constituted the work of the Elementary Teacher Education faculty tuners through the institutional work lens. Specifically, what they accomplished during their ETE Tuning work and how they accomplished their work. During the two year project, the ETE Tuning faculty team worked at developing student learning outcomes and competencies and creating new rules and procedures of evaluating outcomes and competencies for ETE college graduates. Creating and articulating of new rules and norms also involved the development of new practices and the necessity to connect these practices with controlling mechanisms of assessment (Lawrence & Suddaby, 2006). In order to accomplish their ETE Tuning work on new rules, norms and structures, the ETE faculty team worked as professional experts of their discipline in the Tuning process. The following subsections present the key milestones of the Utah ETE Tuning faculty work and demonstrate the lessons learned from the research. 219 Participatory Learning in ETE Tuning Based on the research findings, I assert that understanding the process of Tuning and how to tune the ETE discipline was necessary and important for the ETE Tuning team members. Faculty tuners had to achieve a deep understanding of the process. This included recognizing that ETE Tuning was not another standardization "having standards does not mean standardization" (Braskamp & Wergin, p. 86). This also included that facilitators of the process did not have preconceived outcomes in mind, and, consequently, the faculty tuners were responsible for developing the outcomes. One of the team members exposed: "we went off the rail at the very beginning, we were not exactly sure what we were doing, but we had a pretty great group of people, and so we figured it [Tuning methodology] out" (Faculty_individual interview_01). This displays that Tuning was a "learning-in-working" (Brown & Duguid, 1991) process for the faculty members that "best represents the fluid evolution of learning through practice" (Ibid, p. 41). Implicitly, this also contains the ETE Tuning team needed time to learn about, accumulate knowledge of, and understand the essence of the Tuning process. Likewise, the reports on Tuning from Utah and other states (ICHE, 2010; MNOHE, 2010; USHE, 2009) and Evenson (2012) described that Tuning work required several work sessions for a team to gain "ownership of the process" (p. 20). This comprised understanding of historical roots of Tuning, its goals and objectives, the benefits for students, faculty, department programs, colleges and higher education in general. In order to achieve a well-developed, rich base of understanding of Tuning, to incorporate Tuning within themselves and make their understanding of the process personal, the ETE faculty tuners needed to learn how to tune their college discipline. 220 In the case of ETE Tuning particular work, the faculty members tried what they knew before Tuning but "went off the rail." However, being professionals in their field, they were able to develop insights, and construct new options through practice because: [P]ractice is central to understanding work. Abstractions detached from practice distort or obscure intricacies of that practice. Without a clear understanding of those intricacies and the role they play, the practice itself cannot be well understood, engendered (through training), or enhanced (through innovation). (Brown & Duguid, 1991, p.40) One of the faculty tuners reflected on her Tuning work: "I learned a lot from that work [Tuning]. That means that I got involved in the training. I got involved in the meetings and ultimately got involved in writing that first report [on Tuning in Utah]" (Faculty_individual interview_08). This reflection exposes that the ETE faculty tuners were active participants of the process, and simultaneously were learning together through their participatory learning. Researchers (Barr, 1996; Boreham & Morgan, 2004; Horn, 2005; Lave & Wenger, 1991; Wenger, 2010) who have studied participatory learning put an emphasis on the learning community as active participants in the process, rather than simply recipients of direct instruction. This also relates to creating new institutions, which, in turn, shape the work and understanding within institutions. The participatory learning research identified the "discipline of practice" as the "how" and "what to do" of group interaction, that was necessary for the participants to learn together in a community of practice. Confirming this, the ETE faculty tuners being active learners in their discipline of practice contributed to the process through their participation at the same time. According to Horn (2005), one of the most important ways that communities of practice learn is through "collegial conversations" (p. 225), and collegial deliberations shape the 221 norms, structures, process, and institutional work. The research findings revealed that the ETE faculty tuners were actively participating through debating, deliberating, and discussing in group interactions. All members made contributions to the discussion and therefore to the learning. One faculty tuner called this process cross-pollination and healthy: It [Tuning] certainly creates another layer of collegial relationships with other people thinking about what you are thinking about. And you do bring it if there's another person teaching the same course, you can say, ‘At that meeting when they brought up this, and we ought to think about that. Utah State is doing this, or Dixie has adopted this and such. Let's consider it, we might reject it.' I think there's cross-pollination there, that's healthy. (Faculty_focus group interview) Another faculty member continued: "Academic freedom can coexist and new ideas can be brought in through collaborative work. It's nice because these teams allow continuity of outcomes" (Faculty_focus group interview). This demonstrates that the faculty tuners understood that Tuning was not an infringement on academic freedom, but valued that it was collaborative, creative institutional work. The ETE Tuning case study also affirmed the research (Evenson, 2012; Jones, 2012; McInerney, n.d.; Kolb et al., 2013) that the ETE Tuning work was faculty's genuine professional development. This is how one faculty described her professional development: I think it has been healthy for me as an individual to be able to say, ‘Okay, this is where we are at, and this is the time that I am in now. I am not in the time that was 10 years ago. This is where I am now. I need to understand this technology, I need to do these things, and I need to approach my work from this perspective.' I fine-tune my own career as we go. (Faculty_focus group interview). She continued: What this whole process of renewal and constant renewal has done for me in Tuning along the way has kept me alive. I could easily be sitting, and I have struggled a lot of times with. Here's a new concept, this is what we are doing 222 now, and so let's get on board, and let's have a little learning going on here and get this done. (Faculty_focus group interview). The subject taken up in this reflection is professional development as a form of norming that is modelling a process and part of purposive work of actors. This faculty member openly acknowledges "participation as learning perspective" (Lave and Wenger, 1991), appreciates her own professional development through the Tuning work and feels grateful for the renewal Tuning has brought to her enthusiasm for teaching. This unfolds that ETE Tuning engaged faculty tuners in work on institutions as new rules and norms for their students through participatory learning in the discipline of practice which served as actual professional development for faculty members. Haworth and Conrad (1997) emphasized that "when practiced with the openness and flexibility required of any meaningful learning effort, it promises to reveal program strengths and limitations and to spark informed suggestions targeted at improving the quality of teaching and learning" (p. 171). Bowden and Marton (1998) echoed, "quality in a university context has a lot to do with the quality of learning and the quality of learning has a lot to with qualities of different ways of seeing" (p. 219) and participation is very powerful. The faculty tuners were united with one common goal - to articulate student learning outcomes in measurable and assessable terms. Institutional work analysis states that the idea of engagement was at the center of the ETE Tuning work because the faculty tuners' work contributed significant time and efforts on enriching learning experiences for students by writing student learning outcomes and demonstrable competencies. This reflection by one faculty tuner agrees with that idea: The news, for other people who didn't get to hear all that conversation, they were like ‘what, why did you do that?' It doesn't make any sense for them because they didn't see how the sausage was made. It's buy in. That participation itself is 223 a powerful part of the process. (Faculty_individual interview_04) This faculty member has drawn our attention to the fact that participatory learning of the ETE Tuning team members was necessary and fruitful for creating and articulating student learning outcomes and competencies for elementary teacher education programs in Utah. She also points out how important it is for tuners to buy in, and that one needs to be actively involved in order "to own" it. Her reflection implied that the faculty tuners were agents responsible for changing the institution of elementary teacher education, and this was an important form of institutional work. The following subsection discusses how ETE Tuning faculty team was defining the discipline core and what lessons could be learned from their work. Defining the Discipline Core As discussed earlier, the ETE Tuning team went through very important period of learning and embracing what Tuning was and how to tune the ETE college discipline. Through sharing their professional know-hows, including knowledge, skills, competencies, experiences, talents, and overall proficiencies, the ETE faculty tuners found themselves in their practical purposive Tuning work. Employing the terminology of the report "Tuning American Higher Education: The Process" (IEBC, 2012), the ETE faculty tuners were defining the ETE discipline core. In their journey of discovery, they had to find answers for many questions: What competencies make a student into a teacher? How do we faculty have to prepare students to become professionals? What exactly must we as faculty do, and how can we articulate what must be done in plain language that is easily understood, specific, and unambiguous? In McKiernan and 224 Birtwistle's (2010) terminology, how can faculty make the implicit explicit regarding the knowledge, understanding and skills required for a degree in the ETE discipline. As stressed in Emblems of Quality on Higher Education (Harvey & Knight, 1996), "transparency means being explicit, clear and open about the aims of the programs, the processes of teaching and learning, the modes of and criteria for assessing students, and the intended student attainments" (Harvey & Knight, 1996, p. 40). In the case of Tuning, the ETE tuners were building a comprehensive and transparent picture of the ETE college discipline through their extensive professional deliberations and sharing about how their college departments delivered and should deliver the ETE programs to their students, how different departments taught and should teach their students, how faculty should work with students and help students master their skills. Based on the findings, the ETE faculty tuners' functional and operational work of defining the ETE discipline core was "intelligent, situated institutional action" (Lawrence, Suddaby, & Leca, 2006, p. 219). Their intelligent work included intentionality, coordination, context, time, efforts, and lots of professional discussions about the nature of preparing elementary teachers and the ETE core concepts. In defining the ETE discipline core, the ETE students' learning and outcomes were at the cutting edge of the faculty tuners' deliberations. The team focused on the ETE purpose, and specific essential knowledge and skills as core concepts of the ETE. For example, the meeting notes demonstrate the faculty's discussion: Students need knowledge of child development. They need to have a realistic idea of what an elementary teacher actually does and a critical eye on what can be better. They usually get child development [classes] in lower division courses. This [outcome] is reinforced in lesson plans, including a preassessment lesson plan and their senior project. They will gain the ability to talk and give a rationale of what they are doing. This creates a common lexicon. (USHE, 2013, p. 79) 225 Supported by the meeting notes, the ETE faculty tuners were specifically working at the learner development outcomes. They stated that a teacher candidate: "Collaborates with families, colleagues, and other professionals to promote student growth and development." (USHE, May 21, 2012, p. 1) Then they stated that the candidates' proficiency would be assessed based on "portfolio or teacher work samples with artifacts that demonstrate their understanding and enactment of these principles" (USHE, May 21, 2012, p. 1). Next, the assessment could be conveyed to students through, for example, "attending PLCs, SEPs, IEPs, newsletters in different languages, attending grade level meetings" (USHE, May 21, 2012, p. 1). These extracts expose the tuners' intelligent purposive actions of defining the core of the college discipline which constitute their institutional work. The meeting notes also demonstrate the intellectual work of the faculty tuners as experts on what they prioritized and articulated in the core of ETE college discipline to enhance the students' education and empower them for future teaching. In higher education, "a transformative approach to quality is about enhancing and empowering students, which requires a focus on the total learning experience, all aspects of students' experience that impact upon their learning" (Harvey & Knight, 1996, p. 39). The team also focused on the discipline structure, subfields, traditions, and new tendencies, evolution, in general, and specific application. Through the discussions, faculty tuners constructed the ETE profile. The general picture of the ETE discipline embraced its focus areas, the variety of teaching approaches to the traditional sub-fields (e.g., reading, math, and writing) and emerging sub-fields (e.g., use of new technology). The faculty tuners' deliberations about traditional core concepts of the ETE discipline 226 emphasized the commonality among tuners. In contrast, the identification of different types of student learning outcomes and competencies highlighted layers of specialization and concentration that were not widely shared among the ETE Tuning team members. One of the faculty tuners reflected on the nature of this work: We recognize the creative nature of what we are doing and then there really can be more than one right answer. It doesn't always have to be precise as we've ... We also acknowledge that learners are different, and that they need different things … It's academically healthy to have different perspectives on the same ideas. (Faculty_focus group interview) Changing the institution of elementary teacher education required creativity, efforts by the faculty tuners, and it also required that they sift through the available paths and select one that they collectively believed would provide the most benefit. Conceptually, the ETE faculty tuners were searching inside an elementary teacher education discipline, identifying what was important inside the field, grabbing it, pulling it to the surface, and exposing it for all to see. Through their Tuning work aimed at defining the ETE discipline core, the ETE faculty tuners shaped a real product, and their work became visible and inspectable in Utah Preservice Teacher Learning Objectives. Together, faculty tuners identified and ascertained what content, learning, and skills were expected at bachelor's and associate degree level, and how students could demonstrate their learning and skills. They developed and articulated competency statements and measurable student learning outcomes for preservice teachers in elementary education. It was ETE experts' work on articulation of new rules and norms for their students. Their work product was a new institution for elementary teacher education, and the Tuning faculty worked as agents who created this change. 227 Disrupting Silos Based on the research findings, the ETE Tuning faculty engaged in efforts to disrupt the existing faculty compartmentalization and the process of evaluating the readiness of students to become classroom teachers. In institutional work concept, disrupting is connected with "attacking or undermining" (Lawrence & Suddaby, 2006) the existing rules or norms. Institutional work analysis demonstrated that ETE faculty tuners disrupted silos by their interdepartmental intensive work, communication, and collaboration across all Utah colleges and universities. They also disrupted the existing institution of evaluating college graduates' readiness for entering the classroom as preservice teachers by creating new student learning outcomes and competencies. In higher education "engaged faculty are an essential feature of high-quality programs" (Haworth & Conrad, 1997, p. 46). Related to institutional work, disrupting the compartmentalization of the various ETE departments was something that happened by design of the ETE faculty Tuning team, but it was also supported by the collaborative work of the ETE faculty tuners. As organized, the ETE Tuning team included representatives from each ETE department at Utah colleges and universities. This in itself disrupted the insular compartments that each ETE department was accustomed to, although faculty had annual Elementary Education Major's Meetings in Utah. Prior to the beginning of the Tuning process, each department designed their curriculum as they saw fit, with the external considerations being whether the accrediting agencies would approve their program and whether the USOE would accept their graduates. There was no strong need to consider what other colleges and universities were doing, and there was limited developed structure for collaboration. Although ETE 228 educators met their counterparts from other colleges and universities in social and professional conferences, but there was no well-developed model for collaboration. Tuning changed all of that by bringing the ETE faculty for their frequent professional meetings and intentional work together with a common goal of defining student learning outcomes and competencies. Once the faculty defined these characteristics there was a need to consider how the curriculum would instill those characteristics in its graduates and how the department would evaluate the outcomes. Green (1997) emphasizing forces for change wrote "a remarkable amount of fundamental change in the structure of higher education has resulted from faculty initiatives to change curriculum and pedagogy, and from the imperative that most teachers feel to be on the cutting edge of their discipline" (p. 21). The efforts of each department would have to be done in a way that would withstand evaluation against an objective standard from outside the department that was agreed upon by the Utah ETE community at large. Institutional work analysis exposes that the disruption of compartmentalization was so powerful, and the results so compelling that the ETE faculty developed a recognition of the value of collaborative discussions with representatives from the other colleges and universities across the state. One faculty member expressed her desire for continued meetings with other ETE departments: We haven't met as Tuning group for a while. What we do now is the major's meeting that we bring our information to the major's meetings, and very often the people who were on the Tuning group are now the reps for the major's from all universities. So we only see each other once a year. I had a meeting with [redacted] about a month ago and he asked what I would have liked if there're my preferences. I said we would like to meet twice a year because we are used to meeting on a much more frequent basis to talk about things. That was our first thing that we would like to meet twice a year. (Faculty_individual interview_01) So, while the disruption of compartmentalization was powerful, it was also fragile. This 229 faculty tuner expressed a strong need to meet with representatives of other departments to continue the collaboration. Unspoken but implicit in this request is a realization that if interdepartmental meetings were not continued with adequate frequency, the interdepartmental collaboration as a new model that is needed to sustain Tuning could be lost. The ETE faculty members also expressed the needs to meet with other disciplines. As one faculty member said: We would also like to meet with the math faculty - the math Tuning committee because our students struggle the most in that particular content area. And that if we met with the math instructors, and they gave us their overview, and we gave them our overview of elementary ed., perhaps, our students would be a little bit more successful. And so there were my two requests, and so I am not sure what that higher ed is going to do. (Faculty_individual interview_01). This is an extraordinary request for a couple of reasons. First, it is an admission that the ETE faculty requires collaboration with and assistance from other disciplines to fully prepare preservice teachers. Second, it shows that this faculty tuner, having experienced the power of interdepartmental collaboration among ETE programs in Tuning, is willing to venture out of her discipline and collaborate with other disciplines to improve the preparation program for elementary teachers. The gist of the issue raised in these reflections is that faculty members need more interactions across their discipline field and with other closely connected disciplines to promote student growth and development. Connected with faculty's institutional work to purposively create or disrupt institutions, Tuning is truly a breakdown of compartments in higher education. Supported by the findings of the research, the ETE faculty further engaged in activities by disrupting the current practice of evaluating graduates' competence by tallying courses completed and grades received; in other words - seat time. This will not be done simply or quickly as the existing system is an old one and completely ingrained 230 in the current system of higher education. The state Tuning report stated: "For a century we have relied on the ‘credit hour' as a proxy for learning. Credits represent the Carnegie unit, launched in the early 1900s" (USHE 2014, p. 58). One could even say that the current system of counting credit hours is the very core of the existing higher education system, or at a minimum the core of evaluating students' progress through the system. However, this system has its weaknesses, "the credit hour cannot measure success of degree or the outcomes" (Ibid). Zemsky (2009b), stressed "The nature of the academy sucks the air out of piecemeal reforms. People lose interest; old ways win out; new problems arise" (p. 209). Evidenced in the research, learning-focus Tuning of student outcomes and competencies implies, in the long term, disrupting the seat time model for determining the suitability of a student to progress beyond being a recipient of knowledge and become a classroom teacher. The ETE faculty recognized that their work would contribute into the disruption of the existing system. One faculty tuner stressed: The implications of Tuning, I think, are absolutely tremendous. That makes it not only an important process, but an essential one, and one that can be disruptive to a certain extent, especially on the University's side. Also on the K12's side can be disruptive, but there's nothing wrong with being disruptive, as long as you're moving in a positive direction. (Faculty_individual interview_05) This presents that the faculty member not only recognized that their work would disrupt the existing norms in ETE, it would also eventually disrupt some norms in elementary education, but in a positive way. She continued: "If we change something. Yeah, sometimes we have to disrupt those traditional connection structures" (Faculty_individual interview_05). Even though the faculty knew their work was potentially disruptive, they realized that it represented real and valuable change to their education model. Discussing 231 the effect of Tuning on the syllabi at one department, a faculty tuner said: My department, for example, we've committed to our syllabi. On our syllabi, in some form or another, the learning outcomes are there, that we've agreed to. They may not apply on all classes, but they're there. That's a real change from the syllabi in the past, but there hasn't been as rich a discussion as I think they should be with the connection between the learning outcomes and the experiences that students are being offered to achieve those learning outcomes. I think that's an area that really could be much more discussed, and I look forward to that conversation where the process is. (Faculty_individual interview_08) This faculty member understood that Tuning would be an ongoing process that would require a great deal more work. Connected with institutional work, faculty could change the existing structures and models in higher education through frequent intentional deliberations and by participating in the collaborative writing the degree specifications. The expectation that change would be slow was also echoed by the Tuning report: We are slowly moving away from tallying hours, credits, and grades as "measurements" of higher education and recognizing that we need more sophisticated and nuanced types of evidence to demonstrate how and where students achieve the competencies and learning outcomes we lay out. The change in our ‘recording systems' will not come soon or instantaneously. (USHE, 2014) The ETE faculty tuners and the state Tuning leadership were in agreement on the concept that fully integrating Tuning into higher education would be a time consuming and laborious process, possibly taking decades to complete. Moreover, the Tuning leadership and faculty team recognized that the logical conclusion of Tuning would be the systematic undermining of credit hours and grades for evaluating student learning outcomes and competencies. The following subsection discusses how the ETE Tuning faculty team was fitting their Tuning work into the existing state requirements for elementary education since the ETE college departments are responsible for preparin g the teachers for the state public education. 232 Fitting Tuning into the Existing Model As the research displays, the ETE Tuning started for the faculty members, participants of the project, with Tuning as a new idea for student evaluation through assessable learning outcomes. The ETE faculty mentally tested the new Tuning concept against their existing conceptual models for student learning and evaluation. One of the faculty tuners remembered: "My initial reaction was one of saying, ‘Okay, how does this fit with other discussions I've had?' Then, I worked that out to some similar minds" (Faculty_interview_08). This demonstrates and evidenced in the research, the team struggled with their concern that Tuning might be just another layer of requirements to be met and documented. That it might be additional outside pressure on the discipline or another attempt at standardization, or a top-down process to be imposed on the faculty by administrators. Through their "learning-in-working," the ETE faculty tuners came to realize that Tuning required a new way of articulating student learning outcomes and evaluating student achievement. Once this realization was achieved, they then had to decide how to proceed. This was a very complex and complicated process that required faculty' will, efforts, intentionality in testing and discarding ideas until the ETE Tuning team could develop enough of a conceptual framework. Through their purposive work, faculty could see how the Tuning methodology began to fit in the framework of standards and requirements governing elementary education. Institutional work analysis exposed, that the ETE Tuning team in their attempt to create new rules, norms, and procedures for ETE graduates in the state had "the potential to leverage existing sets of taken-for-granted practices, technologies and rules" (Lawrence & Suddaby, 2006, p. 225), and the faculty tuners were "able to associate the 233 new with the old in some way that eases adoption" (Ibid.). Based on the findings, eventually the ETE faculty came to Utah Effective Teaching Standards and considered these as a starting place. After some debate, they agreed that the UETS would be a suitable vehicle to develop and articulate student learning outcomes and competencies for preservice teachers. The analysis of meeting notes (USHE, April, 2012, May, 2012, April, 2013) exposed that the team intentionally worked to make sure ETE Tuning properly fit into the existing state standards. For example, Table 7 (p. 266) displays a side-by-side comparison of the preexisting UETS (right hand column) and ideas for how to modify the UETS to accommodate preservice teachers on the left hand column. The left hand column represents a work in progress, where the ETE tuners recorded their ideas for modification of the UETS. It illustrates that they were maintaining the core of the UETS as the bulk of the text is normal font, representing unchanged text and bold text representing minor changes in articulating of learning outcomes. Beyond that, the italicized text (blue in the original sample), represents ideas for how things could be conveyed to students, and the underline text (red in the original sample) represents how the teacher could evaluate the competency of the student. The underlined text signifies the artifacts that will be used to assess the students' achievement. The new text is important because as one leader said: "transparency is important. Students need to know what they are learning and what the outcomes they need to achieve" (USHE, 2013, p. 78). Institutional work analysis revealed, to a great extent the team mimicked the structure of the UETS and "stretched" the standards, applying Tuning to reach an accommodation for preservice teachers. According to Lawrence and Suddaby (2006) 234 "part of the success of mimicry in creating new institutional structures is that the juxtaposition of old and new templates can simultaneously make the new structure understandable and accessible, while pointing to potential problems or shortcomings of past practices" (p. 226). In case of ETE Tuning, one of faculty tuners pointed out: "We created the document which, as I said, was a companion document to the Utah effective teaching standards" (Faculty_interview_11). The reviewed documents revealed that the ETE Tuning initiative was a continuous process of improvement of teacher preparation programs, and the ETE Tuning work on "making implicit explicit" (McKiernan & Birtwistle, 2010) driven by faculty was an add-on for the state requirements guiding elementary education. The ETE faculty worked with a sense of agency to make sure that they had done their best to provide for well-articulated student learning outcomes while maintaining the integrity of the UETS. For example, [Redacted] asked, "What else do we need to do? What are we missing?" For the next meeting, she asked the team to read all of the standards, as revised to see if there are any redundancies. Choose the ten key issues. Select two or three main ideas under each standard. This endeavor is different from the other Tuning teams, because elementary education is a licensing community. Making sure that our objectives match the Utah Teaching Standards makes us proactive to get accredited by the State Board. (USHE, January 25, 2012) This quote from the Tuning meeting notes reflects that the ETE faculty tuners engaged in intentional efforts to fit Tuning with the existing state standards and requirements for elementary teachers. This quote also exposes the faculty's ability and will to take the initiative. Through mimicry as institutional work the team used the UETS for applying Tuning that morphed and developed as they were working at articulating a new evaluation product that manifested itself as the Utah Preservice Teacher Learning Outcomes. 235 The institutional work analysis further revealed that the faculty tuners demonstrated their intelligence and efforts in maintaining the UETS and using this institution as the vehicle for articulating student learning outcomes and competencies. The documents, including meeting notes and reports (USHE, 2013, 2014), exposed that the work of defining the core of the ETE college discipline required thorough "repairing and recreating" (Lawrence & Suddaby, 2006), and supporting of the UETS at the same time. One state Tuning leader said that in order to define and articulate student learning outcomes and competencies, faculty tuners "really had to break apart the effective teaching standards [UETS]" (State leader_individual interview_3). Breaking apart the UETS required faculty tuners: [T]o turn a mirror on what they actually do in the programs and to collect data on how these practices affect the quality of student learning…faculty and administrators cannot improve the quality of a system until they make it visible. (Haworth & Conrad, 1998, p. 171) The faculty tuners supported and "repaired" the UETS as an important institutionalized norm in a way that allowed them to expand their coverage to include preservice teachers which the Utah standards were not specifically designed to do. The ETE faculty tuners also maintained these standards by recognizing the value and authority of the USOE in deciding that the UETS would be the vehicle for developing the learning outcomes and competencies for ETE graduates or preservice teachers. Following is the discussion how the ETE faculty tuners were the expert engine of the Utah ETE Tuning process. Understanding Faculty-Driven Tuning The findings of the research exhibited that ETE Tuning work was faculty-driven at the development stage of the process. This was the core of ETE Tuning where the 236 expertise of the faculty was the strongest and critical. The findings showed that the ETE Tuning team was allowed to conduct their work with little to no direct outside influence or pressure. The ETE Tuning work was faculty-driven and was executed with robust consensus as a success. It was collaborative, collegial, expert work united by a common purpose and agenda to improve ETE in the state. The faculty tuners believed and emphasized that the work they were doing was very grassroots and faculty-based discussion. One faculty tuner described their work as: I think, Tuning is an excellent model. It's really a discussion. Certainly, you have to have that top down support, but it's very grassroots, and faculty who are independent individuals. Nonetheless, most of us like to discuss our work, like to share our ideas, and this gives us an excuse to get together and actually have those conversations, and have this be seen as productive, as even necessary to the university doing its work of educating students. I think, there's some really important implications both individually but also institutionally. (Faculty_interview_08) The faculty member thought about the importance of their Tuning work for educating students. Faculty's deliberations is the main thread of this reflection. And this also expresses the necessity of leadership support. Another faculty member reflected how it started with a concept and would end up affecting the lives of generations of Utah school children. She said: When we are thinking about students, we are thinking about those kids. Are we willing to turn this student loose on children in public schools? … Well, would I want this person to teach my grandchild? I think that underlies a lot of what we do. We are always thinking about those people, those little people and what will become of them, and how, what will happen to them and what will be best for them, and will our students be able to provide that. (Faculty_focus group interview) This explains that faculty members of the ETE programs work with a sense of agency and take very seriously their responsibility for preparing their students to become professional teachers, and faculty understand that they are driving this change. The 237 faculty also understand how important their work is for their community, the young generation, and the whole state; that changing the way preservice teachers are evaluated would ripple through their students to the elementary school classroom where it would wash up on the shore of elementary students. Overall, this demonstrates the intentionality of the ETE faculty and their thoughtful, analytical, and reflective attitude to their professional work. The documents, such as the Tuning teams' reports (USHE, 2013, 2014) described a whole Tuning process as a faculty driven or faculty led process. The terms facultydriven or faculty-led have also been used in Tuning literature (Jones, 2012; Kolb et al., 2013; McInerney, n.d.) and most of the interviews. However, the publications and reports did not clearly define the boundaries of the faculty-driven portion of the process. As a result, there exists some confusion about the meaning of the constantly repeated mantra that Tuning is a faculty driven process. Thus, there is a need to define what is meant by the term "faculty-driven process." Supported by the findings from my research, faculty-driven or faculty-led Tuning applied to this research means experts' intentional work aimed at developing and articulating student learning outcomes and demonstrable competencies for elementary teacher education graduates or preservice teachers at bachelor's and associate degree level in Utah without undue influence by outside players. The term faculty-driven, or faculty-led, refers to the ETE Tuning faculty work itself, where faculty collaborated as professionals across their field of expertise to define student learning outcomes and competencies. Beyond this point, the Utah ETE Tuning process involved an array of other players, such as department faculty members, department chairs, college deans, and 238 provosts at university level, the Board of Regents and USOE at the state level, and accrediting agencies. Consequently, the whole process from the beginning to its full implementation could not be faculty-driven. The findings of the research revealed that Utah ETE Tuning was not facultydriven as a whole process till its implementation stage. One state Tuning leader reflected her understanding on how many important actors were involved in the process, and how difficult to deal with a structure with an academic department or college: One of the things we learn which was regrettable, but I understood, it was that some faculty in Tuning, and you saw the evaluation in David's report. Some faculty did try to bring the people on board and for some that was a success, others tried once or twice and didn't go any farther than that. We deal, of course, with a human factor, and that's part of it. We deal also with what kind of structure is there within the academic department that either supports innovation or doesn't. You know, there is that element. Tuning can help in it if the person interpreting it and is able to get other faculty interested, and the chair interested. If not, then the changes are all probably in the classroom. She continued: The same thing with deans of colleges too. We brought together the deans of colleges but you have to do that over and over again. I think, we did it once a year or twice but that's not enough, that's like hearing something and saying ‘oh, yeah, great idea and getting back to what you have to do.' They all need … All of these depend on people of good will who see the value of it. And sometimes no matter how hard we try to interpret the value, it doesn't go anywhere, not because people don't recognize it, but may be deans and chief academic officers are so steeped in their other stuff that they must get done, what they are responsible for. That becomes a real back-burner issue. There is that too. It's really a crap shoot, you know, what we were able to do. (State leader_interview_3) This demonstrates that dissemination and implementation of ETE Tuning depends on many people, first of all, department chairs, college deans, who are in the powerful positions to accept the value and benefits of discipline Tuning, and to promote and implement Tuning. Through institutional work analysis, these evidences unfold the deep relations of actors and institutions, their constant interactions and interdependence; 239 faculty work within a college and university cultures and norms. Additionally, this exhibits the state Tuning leader's deep reflection and attitude on how ETE Tuning was disseminated, and regret that the ETE Tuning was not valued everywhere. This also demonstrates the state Tuning leader's ability to learn a lesson from this situation. One faculty tuner echoed: It just seemed like all the important players didn't really know about Tuning. I'm not really sure, because again, I was the most junior faculty member, but it appeared that, perhaps, Tuning had come across the dean's desk, and the dean assigned somebody and then totally forgot about it. So having all the key administrators playing a role probably would have been more helpful to have Tuning be able to move forward, and to have some understanding of how Tuning should fit, or what the vision of the state was when they wanted to tune elementary education, and what the vision was of how Tuning would fit with all of the other sorts of articulation that goes on with elementary teacher education. (Faculty_interview_11) Another faculty tuner resounded: Certainly, because of the work I do on the campus, with faculty development, I'm less distrustful of something new coming out of the pipe, and saying, ‘Okay, faculty let's do this.' But I know, many of my colleagues were distrustful that [Tuning]. ‘Is this yet another imposition? Is this yet another thing that I have to do?' Without really appreciating that it was a faculty-based discussion and that faculty were essential to this discussion. (Faculty_interview_08) All these reflections demonstrate that besides being just a faculty member at a university, one needs at least to be in a trustworthy position and better in a position of power to move the Tuning process forward. Having realized that, the state Tuning leadership team revealed their agency and organized discussion on how faculty tuners could work at the institutional level. The same faculty tuner remembered: At the statewide Tuning meetings themselves, there has been very good discussion about strategies from the state discussions to the institutional level. I don't think we've done a good job but we have tried, we have shared strategies so ‘How did you approach this? How did your department ... How did you have a conversation at the dean's level? What was the strategy that works there?' I think those meetings have been important. (Faculty_interview_08) 240 This proves that faculty members must be educated to tune and develop their agency, and effective strategies to go from cross-state Tuning developmental work to institutional level implementation. In higher education literature (Haworth & Conrad, 1997; O' Banion, 2014; Steinert, 2005; Tierney, 1998), the role of faculty in higher education changes is highly debated. For example, Green (1997) emphasized: [T]he faculty members are important actors in bringing about changes that grow out of their work as teachers and scholars. In that context, they are acutely aware of changes in their fields, of technological advancement, and of new approaches that require restructuring of their fields or even institutions. (p. 20) O' Banion (2014) echoed: [F]aculty are central to the success of the college and the success of students, and they must be the key agents of any substantive change … faculty as full partners in institutional reform to improve student success and completion … we can only transform our colleges with the full involvement of the faculty. (p. 3-4) All these say that it is not possible to reform any discipline, college, university or higher education in general without faculty expert work. Institutional work theorists (Boxenbaum & Pedersen, 2009; Hirsch & Bermiss, 2009; Trank & Washington, 2009; Zietsma & McKnight, 2009; Zilber, 2009) emphasized the role of professionals in creating, maintaining, or disrupting institutions. Related to higher education, large-scale institutional change can happen and succeed only with faculty agreement and support (Green, 1997). That said, in case of the ETE Tuning work simply producing the Utah Preservice Teacher Leaning Outcomes was not the end of the Tuning process, it must be followed by implementation, and that is where the other actors become involved and other rules and structures become important. At the implementation phase, leadership as well as faculty must take the responsibility to institutionalize the ETE Tuning product, a function that is 241 beyond the capacity of the ETE Tuning team members alone. Faculty members are not in the position of power to make the decisions at the department or college levels. So, it is relevant to apply the term ‘faculty-driven' only to the ETE Tuning work not to the whole ETE Tuning process which was multidimensional and included many elements besides faculty's articulating and writing student learning outcomes and competencies. The following subsection discusses the importance of department support for faculty tuners' work. The Importance of Having Department Support By definition, ETE Tuning as a multilayered process included the interactions within the team, between the ETE Tuning faculty team and the state Tuning leadership team, interactions with department colleagues. It also included interactions with the deans of the colleges of education and other university and college leadership, and external stakeholders such as the USOE, and accrediting agencies, and a broader educational community. The researchers (Jones, 2012; Kolb et al., 2013; McInerney, n.d.) stressed a great value of and need for administrative support for faculty tuners during development and implementation stages on all levels of the Tuning initiative. Similarly, institutional work theorists (Empson, Cleaver, & Allen; 2013; Nilsson, 2015; Rojas, 2010; Singh & Jayanti, 2013; Suddaby & Viale, 2011) indicated that professionals needed support from those with resources and power. Throughout the ETE Tuning process, the faculty tuners demonstrated their qualifications and capabilities, and through their participation all made significant contributions to the development of learning outcomes for preservice teachers in Utah. 242 The findings of my research showed, the ETE Tuning process had consistent support from the state Tuning leadership team. The research findings also demonstrated tenacious collaboration within the ETE Tuning team, where faculty tuners were encouraged to speak openly and freely at the ETE Tuning team meetings and between them. As for the interactions with departmental colleagues concerned, at those departments where Tuning had support, it made a difference. As one of the faculty tuners recalled: "I was without any power portfolio, and then a new Chair came, and she was interested in and came to some of those meetings and participated. That makes a difference" (Faculty_individual interview_8). Another faculty stressed the importance of direct participation of the department chair in the Tuning work: " I know that at Weber the Chair of the department was on the Tuning committee, and so she just implemented it, and the rest of her faculty did too" (Faculty_individual interview_1). The findings demonstrate that support from their own organizations made the faculty tuners more effective agents of change. It provided a pond into which they could "toss the pebble" of Tuning and help propagate the ripples of change through the ETE community. As stated in the report (USHE, 2013), Tuning team members "have been most effective when the discipline team member at the institution has been the chair of the department. Being the chair, gives the team member the focus and the authority necessary to motivate the department…" (p. 6). This illuminates the need for Tuning team representatives to have significant influence in their home department in order for the Tuning results to be recognized and valued in their home departments. Supported by the findings, however, the ETE Tuning faculty were not always well 243 received when reporting back to their home departments. They needed more support, time, and faculty agenda. For example, one faculty tuner emphasized: " What I needed at my institution was from my department head to care more. … Time, time, and the faculty agenda but that would be the main thing" (Faculty_individual interview_4). Just as the faculty tuners relied on the state Tuning leadership team to facilitate the development of student learning outcomes and competencies, the faculty tuners also needed departmental level support to disseminate the Tuning knowledge and then implement these changes. The report "Transparent Pathways, Clear Outcomes: Using Disciplinary Tuning to Improve Teaching, Learning, and Student Success" (Stein & Reinert, 2014) highlighted that "without intentionality of participation by departments at the front end of a new initiative, tuners unnecessarily have the additional burden of having to gain their colleagues' attention and interest as their work progresses" (p. 43). The data analysis revealed the lack of support structure for Tuning experienced by some ETE Tuning team representatives in their home departments diminished the effectiveness of the ETE Tuning project in disseminating the concept of ETE Tuning throughout the ETE higher education community. Some elements of the ETE Tuning process were not completed to the extent the ETE Tuning team had hoped because of lack of support from colleagues in their academic departments. Lack of support might be attributed to faculty ignorance of the Tuning process. In order to understand and value the Tuning process, one needs to be a part of this process. As one faculty tuner underlined: "[b]ecause my department head wasn't there it wasn't important to her" (Faculty_individual interview_04). Just as the ETE Tuning team had to learn about Tuning through their intentional participation before they could embrace it, so must department faculty go 244 through the same process before the results of state wide Tuning can be implemented at the department level. Lack of support might be also attributed to departmental faculty fear that someone was going to force this process on them. Consequently, the faculty tuners needed support from the department chairs "as full partners in institutional reform to improve student success" (O'Banion, 2014, p. 3). Additionally, the findings of the research revealed the importance of having department support from the colleagues whose power could "hold in delaying or blocking progress" (Ibid.). Connected with institutional work, implementation of new models and norms is culturally embedded, and actors can find themselves in "iron cage" (DiMaggio & Powell, 1983). The following section discusses the implications of ETE Tuning for practice, policy, and future research. The Implications of ETE Tuning: A Mechanism of Enhancing Higher Education As evidenced in the findings, the Utah ETE Tuning process provides a mechanism of enhancing higher education as a state-wide discipline level change accomplished by faculty and leadership together. This process has demonstrated how faculty were intentionally developing and articulating student learning outcomes and demonstrable competencies for all ETE programs in Utah. They were doing it based on consultations with and getting input from students, employers and other stakeholders. Although we have not seen whether Tuning has changed the ETE discipline once implemented; the process is at its early phase, however, the document analysis exposed that student learning outcomes for ETE were included in programs' syllabi, displayed in 245 university canvas systems, shaped through observation form, and portfolios. These sprouts let us say that Tuning potentially benefits students, faculty, parents, programs, and ultimately elementary school children because it provides transparent pathways through a college degree. Tuning, as a relatively new concept is truly a process of exploration, and Haunschild and Chandler (2008) stressed "processes of exploration engender dramatic institutional change and that such processes are more likely to occur under slow adaptation and underperformance in relation to specific institutional norms, rules and models" (p. 643). Given all of these, the research findings permitted me to provide some findings implications for practice, policy, and future research. This section discusses the findings implications of the ETE Tuning process for the practice of elementary teacher education, such as: the shift to outcomes, shift in culture of higher education from teaching centered education to student centered learning. It also considers the findings implications for faculty professional development and program accreditation. The section also considers the findings implications of the ETE Tuning process for educational policy. Furthermore, the section provides implications and recommendations for future research. Implications for Practice According to Evenson (2012) and McInerney (n.d.), a college discipline cannot be tuned, because Tuning focuses on student learning, transparency, and accountability, so it requires an ongoing process of evaluating what has been done and what more can be done to improve the outcomes. Educational researchers (Braskamp & Wergin, 1998; 246 Chaffee, 1998; Ewell, 1998; Tierney, 1998) have addressed several shifts in higher education in connection with the responsive paradigm. For example, the shift "from how faculty members teach to how students learn" (Keith, 1998, p. 164), that is the shift to real student-centered programs causes the shift in how we think about out teaching and what we could do in our teaching practices. As Adelman (2009) stressed, Tuning is poised to become the higher education model of collegial and collaborative faculty's writing students learning outcomes and competencies for their disciplines. The changes that arise from Tuning affects students, colleges and universities, faculty, and, eventually, public education and require new learning and new ways of thinking by all involved. Applying our "pebble in the pool" metaphor, the changes are all ripples that propagate from tossing the Tuning pebble into the pond of elementary teacher education. Evidenced in the study of the Utah ETE Tuning process, the shift in ETE from focusing on inputs to a focus on student learning outcomes and competencies is continuing and becoming intentional. For teaching practice, it means that faculty are and will continue mapping learning outcomes and competencies and related assignments, and assessment tools across their disciplines. It also means that faculty must collectively make certain that their discipline appropriately enhance students' practices (Schneider, 2013). Cochran-Smith (2006) outlined that: The "outcome question," which has emerged as central in the last decade, has to do with the expected consequences of teacher preparation as well as how, by whom, and for what purposes these outcomes are assessed. This is in keeping with general shift in the field away from focusing primarily on curriculum- or program-oriented standards and toward emphasizing instead performance-based standards and the long-term impacts of teacher preparation on K-12 students' learning. (p. xxxviii) Connected with this, ETE faculty from higher education must collegially articulate those 247 outcomes and competencies in measurable and assessable terms following by implementation across the discipline in all Utah colleges and universities. This, in turn, requires a cultural change in academic departments and colleges. Following Gaston (2008), the ETE Tuning project as a means of developing student learning outcomes, and if defined transparently in operational terms, could offer "objective benchmark criteria" (p. 2) for quality of a college degree. Getting a measureable benchmark that is definitely and transparently articulated and understood would aid both the programs and the students. From this perspective, the ETE Tuning work was extremely important in terms of getting out benefits for students in tuning the ETE college discipline. For a student, there are a number of benefits from Tuning, one of which is that students will be able to say that "I learned these competencies, I have the learning outcomes, they were assessed, and now I can do A, B, C, D, E" (State leader_individual interview_3). This reflection deals with the burning issue of transforming higher education through student learning outcomes (MacDonald, 2014). Once a student can say and demonstrate her competencies, the student understands not only what she learned, but why, and how she will benefit in the classroom and later in her teaching. The intentional focus on student learning requires professional development by the ETE faculty. Tuning as an educational reform "demands the collective intelligence and commitment of all parties who have a stake in academic programs" (Haworth & Conrad, 1997, p. 169). This includes the ETE faculty who will be, due to their positions and expertise, the principal drivers behind developing and implementing Tuning. The institutional work analysis of the research findings reminded us that "it is people, who, first and foremost, make quality happen, this principle is especially crucial for ensuring 248 stakeholder investment in the continuing improvement of academic programs" (Ibid, p. 170). The faculty are the only ones who can make the Tuning results a reality in the classroom, and they will need to be convinced of its benefits, and learn how to implement it. The two-year programs have been among the first to fully integrate Tuning into their programs. Weber State University has developed a two-year Associate of Science in Pre Education program designed to prepare students for transfer to a four-year institution to complete their Bachelor's degree with minimal to no loss of credits in the transfer process. One faculty tuner described it: It was also an implication that we could see we were aligning with others because the reality is we're not competing with other institutions, we're all working together to produce quality education in the state of Utah, and it was meant to help students when they have to transfer between institutions to finish their education. So, that we don't lose people who need to transfer for whatever reason. (Faculty_individual interview_10) This will directly benefit students who want or need to transfer between programs. Another example, Salt Lake Community College began to bring Tuning into their twoyear program. In the Tuning meeting: SLCC stated that portfolios are required for transfers so that, for instance, the U would know what it is these students have learned. The assessment pieces come from interstate consortium and also from the DQP. All artifacts have these components and show how the education courses connect to gen ed courses and then how gen ed courses connect to SLCC's mission and how they connect to other institutions. (USHE, September 27, 2014) These quotes give an account of facilitating transfers as an outwardly visible outcome that the public can see and easily understand. And if students recognize value in lower cost two-year programs before transferring to a bachelor's program, it validates the finding that Tuning has empowered students to shop for their education by seeking the 249 best educational outcomes rather than by selecting programs based on brands that may be more expensive but do not have enhanced value. The document analysis showed that Weber State University have also used the Tuning results in their Bachelor's program: Weber has embraced the Tuning outcomes. Kristin Hadley, the department chair, has made a chart that has the preservice teacher outcomes cross-referenced with the specific courses. It includes the assignments and assessments for each. These mesh well with outcomes from other organizations such as the Council for Exceptional Children (CEC) and with what people in the field want. It is nice for faculty and students at Weber to know exactly how they are using the learning outcomes (LOs). The LOs prepare students for the Utah Effective Teaching Outcomes. (USHE, 2014, p. 49) This is arguably the best example available from the Utah Tuning project of how colleges and universities can use Tuning to assure that their programs align with outside standards and also help students see why they are learning what they are learning. From an institutional perspective, the department chair moved beyond taken-for-granted norms and through cognitive efforts to institutional change, which, in turn, benefit students and faculty. Tuning benefits are experienced by faculty as well as students. Individual faculty members have seen benefits from the interaction with other departments. As one faculty tuner expressed: I am better connected to other people across the state. I know what's going on at Southern Utah University. I know what's going on in the University of Utah, not in detail, but I have a rough idea, and that's a very valuable thing. (Faculty_individual interview_04) This faculty member has recognized the benefits of interorganizational collaboration at the discipline level. Where Tuning is expected to provide overall benefits to college students, discipline, it also provides benefits to faculty members from interaction with 250 colleagues outside their university. Overall, from an institutional perspective faculty collaboration as interorganizational interactions at the college discipline level is a horizontal norming mechanism which provides benefits for student transfer. Based on the research findings, Tuning is connected with accreditation. Expressed by many participants, the Tuning initiative complements accreditation, and can be very helpful to those ETE programs that go through accreditation. Dr. William Evenson in his white paper "Relationship of Tuning and Disciplinary Accreditation" (www.tuningusa.org/library) explained that: Tuning takes discipline faculty deeper and into more explicit outcomes expectations than do existing accreditation standards. Furthermore, Tuning reports can be organized and formatted so they are useful for accreditation. There need not be duplication of effort in this process. … The Tuning process does not seek to supplant the criteria of discipline accreditation with different or expanded criteria. Rather, Tuning is complementary to discipline accreditation in that it seeks to define in specific and assessable terms the program objectives and learning outcomes, level by level. (WEE, 2010, November 7, p. 1) The understanding behind this statement is important to building acceptance for Tuning. It seeks to allay fears that Tuning might impose yet another layer of documentation requirements on already strained programs and add to the many outside-of-the-classroom obligations that programs face. The offered explanations helped tuners realize that the effort required for Tuning, when properly done, would be almost entirely directed toward program improvements and not toward additional administrative burden. The central idea of his statement is that the Tuning product aimed to state clearly in specific and assessable terms what students should know, understand, and be able to do upon graduation. Consequently, all stakeholders including students, faculty, parents, employers, and broader educational community could get a transparent meaning of a specific college degree. Additionally, imbedded in his explanations is the main aim of 251 accreditation as educational accrediting agencies evaluate college programs, not every student. Although closely connected with accreditation, Tuning is different from it. Accreditation addresses whether a program, in its totality, is deemed adequate for its purpose. Tuning, in its turn, is aimed at presenting transparent college pathway for every student in the specific discipline and degree level, and can support a department in taking tough actions on the approval process from accreditation agencies. Accreditation aims to assess the program, and, consequently, assesses only some samples of student work. On the contrary, Tuning aims to assess every graduate based on the outcomes required for a student to get a degree. This way a school district as a main employer for ETE graduates gets an assurance of what a graduate knows, understands, and is able to do. The shift in focus on student learning causes a shift in informing the culture of higher education, and this forces a rethinking, reconceptualizing of teaching, critical analysis, and development of new approaches to teaching. These changes then, require different patterns of interaction among faculty, redesign of educational structures, and changes in how the structures interact with each other both vertically and horizontally. This, in turn, demands continuous improvement of all elements of higher education and leads to faculty development in ways that have not been historically common, such as collaboration horizontally across the degree programs. Haworth and Conrad (1997) articulated the implications for faculty this way: "Faculty and administrators now must become learners as well as teachers. Without their engagement in studying, understanding, and improving student learning, the overall quality of academic programs - and the students' experiences in them - will suffer greatly" (p. 171). 252 Related to institutional work, the existing institutional structures and bureaucracies can be expected to resist the Tuning changes as something new, unfamiliar and untested. As said by one tuner: For most universities, there was just one person on the committee. They [faculty tuners] had a whole department that they had to go back and say ‘Look, here's what we've done, and here are the reasons why.' They had to convert everybody. (Faculty_individual interview_05) This faculty tuner's reflection centers on the difficulty of convincing faculty members and department chairs, who had not participated in the Tuning process, of the value and benefits of Tuning. All other educational structures can be expected to react in similar ways. Many faculty tuners were frustrated by skepticism and inertia among their colleagues as they tried to be ambassadors for Tuning with their home departments. This could be addressed by either increasing the number of representatives from each department to the Tuning team, or by considering the intradepartmental status of the ETE Tuning team representatives. For best practice results, Tuning team representatives should be either department heads or senior, well-respected members of the faculty. If this is not possible, departments should assign multiple faculty to the Tuning team. In institutional work, having more than one voice to advocate for Tuning in the department, or having an influential, respected voice would help with faculty acceptance of Tuning. Another recommendation for practice is based on the rejection of the ETE Tuning results by the Utah State Office of Education. This can be traced to a number of factors, including the constant political pressure on the public educational bodies, and the fact that Tuning was being developed contemporaneously with the Utah Effective Teaching Standards, and therefore could have been seen as a complicating factor by the USOE. 253 Although the Utah Tuning leadership attempted to engage the USOE, early in the Tuning process it lost its most influential USOE representative. As a result, when the time came to ask the USOE to accept the Utah Preservice Teacher Learning Outcomes, they declined. These types of issues could be present in any potential Tuning initiative. So, Tuning leadership must make special efforts to identify important political and regulatory bodies that have the authority to impede the process and develop specific engagement plans for interfacing with these bodies. The list of important political and regulatory bodies should be periodically reviewed and because of the importance of acceptance by the regulatory community, the engagement plans must receive high level monitoring and support. The following subsection discusses the findings implications for educational policy. Implications for Policy As this research demonstrates, ETE Tuning has been an intentional state higher education effort to advance a change in the state higher education in order to fulfill the need of enhancing quality of higher education through student learning outcomes. Based on the document analysis, meetings observation, interviews, and my personal participation in the "What is an Educated Person?" conferences in 2013, 2014, and 2015, the ETE Tuning project was a part of the state's broader plan "to improve the quality of higher education by establishing transparent and fully assessable learning outcomes and competencies for degrees, discipline by discipline" (Davis & Williams, 2012, p. 105). The Tuning 2 Project, which included ETE Tuning, worked simultaneously with other statewide higher education efforts aimed at improving higher education in Utah. Utah 254 higher education has been participating in regional and national projects, namely, the Western Interstate Passport Initiative, Degree Qualifications Profile, and the Quality Collaboratives, the Multistate Collaborative to Advance Learning Outcomes Assessment, and the Liberal Education for America's Promise initiative (Safman, 2012). For example, the Interstate Passport Initiative funded by the Carnegie Corporation of New York and coordinated by the Western Interstate commission for Higher Education included five western states: California, Hawaii, North Dakota, Oregon, and Utah, and paired colleges and universities from one state with colleges and universities from the other. The report (USHE, 2012) stated that faculty intentionally and collaboratively worked to assure that completion of the lower division general education core would transfer within the five corresponding western states. Another example, the Quality Collaboratives Initiative funded by the Association of American colleges and Universities through a Lumina Foundation grant targeted joint work between community colleges universities and was aimed to improve articulation transfer between the two- and fouryear institutions and define student learning outcomes assessment (Safman, 2012). Together, these higher education initiatives contributed to Utah higher education goals of continuous improvement through focusing on student learning. The ETE Tuning process demonstrated the features of the quality models in higher education reviewed in Chapter 2 of my dissertation. Namely, ETE Tuning embraced a focus on student experience, which is a central idea of the transformative model (Harvey & Knight, 1996). ETE Tuning also demonstrated great faculty contribution to enrich the student learning experience, which is the goal of the engagement model (Haworth & Conrad, 1997). Additionally, ETE Tuning targeted 255 student learning as the main feature of the University of learning model (Bowden & Marton, 1998). The scholars made a point that the university had only one aim, to provide learning: [T]eaching, research, and service are all supposed to yield learning: for the individuals (through knowledge being formed which is new to a particular person), for humanity (through knowledge being formed which is new in an absolute sense) and for communities (through knowledge being formed for specific purposes). (Bowden & Marton, 1998, p. viii) This statement is foundational to Tuning as Tuning seeks to focus exclusively on student learning. Keith (1998) stressed, since the public "will judge the university in terms of the quality of their relationships with the university, and the quality of the outcomes of those relationships … , colleges and universities will have to be responsive … and serviceoriented" (pp. 163-164) for students, parents, governments, businesses, nonprofit organizations, if they intend to survive and thrive. Definitely, ETE Tuning worked on articulating student centered college programs and community centered in outreach, which is a keystone of the responsive university model (Tierney, 1998). The analysis of the ETE Tuning process exhibited that Tuning process centers student learning and faculty teaching as the professional experts who are called to be in direct interactions with students, develop and articulate student learning outcomes and competencies using Tuning methodology. My single-case study of ETE Tuning in Utah exposed that faculty' role as experts is critical in developing student learning outcomes. Related to this, the findings implications on policy for teacher preparation programs fall into the following categories: programs design, measuring student outcomes, preparation program standards, and faculty composition. Faculty must consider redesigning their discipline based on expected student 256 learning outcomes where student critical skills and competencies are identified and their mastery is required for graduation. The defined core of a college discipline must contain needs of student learning, needs of children learning in elementary schools, and needs of employers. Student achievement must be subject to competency demonstration. Consequently, programs must provide focused course content to assure that all graduates attain the necessary competencies prior to graduation. Accompanying course redesign must be a change in measuring student outcomes. Critical skill competency could be evaluated on a pass/fail basis. Under this model, faculty must determine how students expect to display their competency and what evaluation standards criteria should be applied. Students must demonstrate their competencies and once successful, be deemed competent to move on to other subject matter. It also means that a student who completes the course work but fails to demonstrate the required competencies would have to repeat the course until successful. The use of pass/fail systems for evaluating learning outcomes and competencies makes that portion of the program look a lot like a professional certification. A degree could then be based on a hybrid evaluation of traditional grades in areas such as subject matter, and competency demonstration in critical teaching skills. Focusing on developing teaching skills in students may allow faculty to be hired and retained based on demonstrated skills in developing student competencies. Faculty skilled in developing classroom skills in students may be drawn from a different pool than the current faculty. One possible place where these skills could be found would be in highly successful classroom teachers themselves. This is an untested proposition that should be subject to further research. The following subsection presents the findings 257 implications for further research. Implications for Further Research Based on the findings, I also provide some implications and recommendations that could stimulate further empirical research on institutional work in tuning a college discipline. My research focused on the ETE Tuning faculty work aimed at writing criterion-referenced student learning outcomes and demonstrable competencies. At the same time, during the course of my research I viewed the very complex role that the state Tuning leadership played in the Tuning process and the importance of their role in advancing Tuning outside the higher education community. The research findings demonstrated a great need of leadership institutional work in building normative networks, advocating for Tuning, and supporting faculty in their professional innovative practices. My research led me to deep thinking about the influence of powerholders at different levels in higher education, and in case of ETE Tuning also elementary education field. Therefore, the complexity of leadership role needs to be better studied as a template for future cross state Tuning efforts. Adopting the relational approach, questions that might guide this research would be: How do multiple powerholders affect the discipline Tuning process? How do field-level conditions enable different forms of institutional work? How do the actors' leadership positions influence the Tuning process? Both the ETE Tuning faculty team and state Tuning leadership team recognize that Tuning is a nonlinear continuous process. The ultimate goals of Tuning are: "(1) to have graduates who know how to think and apply their learning, which employer surveys 258 suggest our graduates cannot do - and (2) to have clear, transparent, persuasive ways to confirm the learning that has occurred" (USHE 2014, p. 59). This statement exposes that the Tuning process is focused on preparing students to be competent teachers in a way that is measurable and verifiable, and, importantly, its full implementation will bring these benefits to students and programs across the state. However, no one should expect this process to happen by itself or overnight. From an institutional perspective, the Utah ETE Tuning process requires changes at every ETE department across the state, and acceptance by the Utah State Office of Education, and the accrediting agencies. It requires that school districts and principals accept the newly graduated teacher candidates assessed based on the Utah Preservice Teacher Learning Outcomes. Consequently, further research is needed at the implementation phase of ETE Tuning at the colleges and universities. With a total of 10 (eight public colleges and universities and two private), ETE programs expected to implement Tuning, it is likely that the seeds that have been planted at each of the departments will experience large variations in implementation pace and success rates. This research has three potential tracks to examine: the role of the faculty actors, the role of the Tuning leadership at the college and university level, and a comparison study of the pace of implementation at the department level. Because these topics all require that the departments be well down a path of implementing Tuning, this research may not be possible for several years. The first track would be research into the roles of the faculty actors, and how they transition from their roles as statewide faculty tuners to their new roles as advocates for Tuning inside their home departments. We have yet to examine how the actors do their institutional work and operate within, and influence the norms, rules, and structures of 259 their departments and universities. This research could be guided by asking: Who participates in ETE Tuning at the college or university department level? What controls the pace of Tuning at the department level? How do the faculty members drive Tuning work? How does faculty's position mediate the relationship with institutional environment? The second track should be to examine the role of Tuning leadership at the college and university level. The leadership will have much different composition than it had at the state level and will have different responsibilities. Similar to the future research of the faculty role, the research is needed to explore the institutional work of leadership actors within the norms, rules and structures of the colleges or universities. Potential research questions are: How are Tuning leaders chosen? Who serves as Tuning leaders? What practices do college or university Tuning leaders accomplish? The third track would be a plenary comparison study of implementation progress at all of the Utah ETE departments. It would examine rapid adopters and slow adopters, and ask question about the efficacy of Tuning in each instance and seek to understand the virtues and pitfalls of rapid and slow adoption. Possible research questions would be: What institutional characteristics are different between slow adopters and rapid adopters? How does institutional work create, maintain, and/or disrupt institutions within each department? Which path of adoption, rapid or slow, leads to the most robust implementation of Tuning? Research is also needed to study the Tuning process in other disciplines, and how those Tuning processes are similar or differed from ETE Tuning, and what factors make them be similar or different. We also need to explore how highly regulated institutions 260 (e.g., elementary education and elementary teacher education as connected with public education) respond the emerging external changes. Further research is needed to examine the Tuning and Degree Qualification Profile processes, and how these two intentional efforts in improving higher education complement each other (e.g., at Salt Lake Community College). Utah higher education is experiencing a number of initiatives aimed at improving the quality of undergraduate higher education in the state. Connected with this, further research is needed to explore how Tuning fits within this family of initiatives. Adopting a discourse approach, the researchers should focus on the role of language and discursively mediated experience (Alvesson & Karreman, 2000; Marshak & Grant, 2008) of Tuning as intentional work aimed at creating new institutions, maintaining necessary ones, and disrupting those that contradict. The researchers' attention should be on how Utah ETE Tuning was framed and talked about because this filled in and signified, and shaped how faculty thought about and responded to the Tuning process and practice. This implied the possibility of potential multiple realities: different stories, different narratives, and different cognitive constructs in Utah Tuning. The discourse analysis of Utah ETE Tuning would create a thorough knowledge of, and experience with, the corpus, which in turn, facilitate empirical generalizations of the Tuning process and practice (Huckin, 2002). A range of discursive elements that constitute the linguistic and symbolic life of organizations: names, roles, strategies, products, plans, ideas, stories, places, people, concepts would provide new insight into Utah Elementary Teacher Education Tuning. 261 Conclusion American higher education is "straining to respond to changing an environment" (Eckel, 2000, p. 15). The Utah System of Higher Education has adopted Tuning as an integral part of their response to this challenge. These research findings demonstrated Tuning as transformative to higher education as it changes the focus from teaching activities to learning outcomes which are called to be transparent, measurable, assessable, and matched to employer needs. This puts the student at the center of educational practice and educational policy. The research literature in higher education (Adelman, 2008b, 2009; Birtwistle & McKiernan, 2008; Evenson, 2012; Gaston, 2010; McInerney, n.d.) stressed a huge need to shift the focus of faculty members, departments, colleges and universities, professional associations, and accrediting organizations from what is taught to what students learn. Importantly, the ETE Tuning process embraced the concepts of "what" (focus on student learning) and "why" (higher education must prepare highly competent and skillful professionals) of change, and expanded on these concepts to include the "how" (faculty writing student learning outcomes with overall support of the state leadership) of change. The research findings indicated that ETE Tuning enveloped the ideas of the quality models of higher education and went a step beyond to show how to articulate education quality in terms of learning outcomes and implement it in elementary teacher education. Institutional work analysis of the ETE Tuning process demonstrated a critical role of the ETE faculty members as professional experts in developing student learning outcomes and competencies for college graduates. Institutional work analysis also exposed the role of the state Tuning leadership team as agents of change. Both teams, 262 ETE faculty tuners and state Tuning leadership, complimented each other in their purposive work aimed to create, maintain, and disrupt institutions. The state Tuning leadership team acted as entrepreneurs and advocates with outside agencies and broader educational community and worked to establish normative networks. The ETE faculty tuners, through their participatory learning and collaboration defined the discipline core in learning outcomes and competencies for preservice teachers. Both practices were necessary and neither, alone was adequate to complete Tuning. Intuitional work analysis revealed that change required disrupting current institutions, and the ETE Tuning process disrupted one of the most firmly entrenched institutions in higher education, the silos that surround individual ETE departments. Only by breaking down silos could educators from across the state come together to develop a consensus for defining learning outcomes and competencies. The faculty respected the authority of the USOE by adopting the UETS as the starting point for their work. The faculty tuners came to appreciate the interdepartmental collaboration and were looking for additional cooperation once the statewide process came to a close. This became especially acute as ETE faculty tuners came up against resistance in their home departments as they tried to advocate for Tuning. Colleges and universities that assigned department heads to participate on the faculty Tuning team were the most receptive of the Tuning process and results. Applying a "tossing a pebble into a pond" metaphor, the faculty tuners received their ripple from the state Tuning leadership team and sent it out through their intentional work at creating and articulating student learning outcomes for preservice teachers, and through their work with their colleagues in their home departments. If fully 263 implemented, the ETE Tuning will continue "to ripple" through ETE syllabi, through preservice teachers outcomes and competencies, into the elementary classroom and into improvements in elementary education. Similarly, the state Tuning leadership team sent their first ripple out to the faculty tuners but then continued working on institutions (sending ripples) through elementary education departments at the colleges and universities and through accrediting agencies, and wider educational community. 264 Outside the Tuning process there were main stakeholders who determined the rules, policies, standards, requirements Inside the Tuning process Internal factors of the Utah ETE Tuning process: Elementary The state Tuning leadership team work Faculty's expert work aimed at developing student learning outcomes. education Featured factors: Accrediting agencies governing Discipline and degree specific requirements and Learning-driven process The USOE key role standards - Student learning-centered Faculty-learning (faculty professional development) Faculty teaching Faculty collaboration and collegiality External factors of the Utah ETE Tuning process Figure 10. The ETE Tuning as a Complex and Complicated Process. School principals Who? The ETE Tuning faculty team External factors Who? What? How? The state Tuning leadership team Role of the ETE Tuning What? Role of the state Tuning faculty team: External factors professional experts leadership: ETE Tuning as Institutional work entrepreneurs Constructing networks How? Defining composition of the ETE Tuning team Participatory learning Defining discipline core Facilitating Educating Advocacy Fitting Tuning into the state requirements Direction to and connection with the ETE departments in Utah colleges and universities Driving Tuning work 265 Figure 11. Tuning the ETE Discipline at the State Level as Institutional Work. 266 Table 7. Comparison Sample of UPTLOs With UETS. Utah Preservice Teacher Learning Outcomes Learning Environments The teacher candidate: 1. Develops learning experiences that engage and support students as self-directed learners who internalize classroom routines, expectations, and procedures. (classroom management plans) 2. Collaborates with students to establish a positive learning climate of openness, respectful interactions, support, and inquiry. (class meetings, morning meeting, webpages, suggestion boxes) (student teaching observations) 3. Describes, evaluates, and/or uses a variety of research-based and theoreticallygrounded classroom management strategies to effectively maintain a positive learning environment. (classroom management plan) 4. Understands how to equitably engage students in learning by organizing, allocating, and managing the resources of time, space, and attention. (calling on students in systematic and fair ways, for example) (case study of a classroom) 5. Demonstrates ability to extend the learning environment using technology, media, and local and global resources. (lesson plans, assignments in technology class) f. Encourages students to use speaking, listening, reading, writing, analysis, synthesis, and decision-making skills in various real-world contexts. Utah Effective Teaching Standards Standard 3: Learning Environments The teacher works with learners to create environments that support individual and collaborative learning, positive social interactions, active engagement in learning, and self-motivation. THE TEACHER: a. Develops learning experiences that engage and support students as self-directed learners who internalize classroom routines, expectations, and procedures. b. Collaborates with students to establish a positive learning climate of openness, respectful interactions, support, and inquiry. c. Uses a variety of classroom management strategies to effectively maintain a positive learning environment. d. Equitably engages students in learning by organizing, allocating, and managing the resources of time, space, and attention. e. Extends the learning environment using technology, media, and local and global resources. f. Encourages students to use speaking, listening, reading, writing, analysis, synthesis, and decisionmaking skills in various real-world contexts. Sources: The data in column one are from "Meeting notes of the ETE Tuning faculty meeting" (USHE, May 21, 2012). The data in column two are from the Utah Effective Teaching Standards (http://www.uen.org/k12educator/uets/). APPENDIX A ENGAGEMENT THEORY OF PROGRAM QUALITY Figure 12. Engagement Theory of Program Quality. Reprinted from Emblems of quality in higher education: Developing and sustaining high-quality programs, by J. G. Haworth and C. F. Conrad, 1997, Needham Heights, MA: Allyn & Bacon. Copyright 1997 by Allyn and Bacon. Reprinted with permission. 268 APPENDIX B INSTITUTIONS Figure 13. Institutions. Based on the definition of institutions (Scott, 1995, p. 33-34). 270 APPENDIX C THE TUNING AND BOLOGNA PROCESS RELATIONSHIPS Figure 14. The Tuning and Bologna Process Relationships. 272 APPENDIX D NATIONAL QUALIFICATIONS FRAMEWORKS 274 Table 8. National Qualifications Frameworks. NQF / Country The Irish national qualification framework / Ireland The Scottish national qualification framework / Scotland The Dutch national qualification framework / the Netherlands The French national qualification framework / France The German national qualification framework / Germany Main Features a comprehensive vertical approach; ten levels of education: from elementary school to doctoral work, defined in terms of broad outcomes on a grid of knowledge, know-how and skill, and competence comprehensive from kindergarten through the doctoral level; a combination of philosophical and technical statements; 12 levels of qualifications from elementary school through the doctorate under five broad categories: knowledge and understanding (mainly subject based), practice (applied knowledge and understanding), generic cognitive skills (e.g., evaluation, critical analysis), communication, numeracy, and IT skills references labor market positions and tasks; a de facto extension of existing accreditation standards; each occupation ideally establishes a qualifications dossier. reaches into 19 specific applied disciplines that lead to licensure occupations; "objectives" under three headings: knowledge and understanding, skills and abilities, and judgment and approach; a more parsimonious articulation of how university students must demonstrate knowledge through instrumental competencies, systemic competencies, and communicative competencies; indicates the length of the program in terms of credits and enrolled time, preconditions for admission, subsequent educational opportunities, and special rules for recognition of nonformal education by examination; The Swedish requires every degree program to undergo a central registry review, national and the program dossier is made public via the Internet; the dossier qualification includes a description of the competencies, aptitudes and knowledge framework / associated with the qualification and necessary in the work for Sweden which the study qualifies the student The UK national through its Quality Assurance Agency, promulgates program qualification specification" and discipline-specific benchmarking" structures framework / the rest of UK Source: Adelman, C. (2009). The Bologna Process for U.S. eyes: Re-learning higher education in the age of convergence. Washington, DC: Institute for Higher Education Policy. pp. 31-45. Adapted with permission. APPENDIX E DYNAMIC RELATIONSHIPS AMONG TUNING STAKEHOLDERS 276 Figure 15. Dynamic Relationships Among Tuning Stakeholders. APPENDIX F INTERVIEW PROTOCOL 278 Date: ________________ Interviewee: ___________________ College/University: _______________ Introduction and Purpose: Explanation: I am a Ph.D. student in the Educational Leadership and Policy Department at the College of Education of the University of Utah currently doing my dissertation research project. This interview is a part of data collection procedure aimed at obtaining meaningful information through gathering the participants' perceptions of the discipline tuning experience in their own words. Thank you for your participation. I believe your input will be valuable to this research. The purpose of the study is to examine the Tuning process for baccalaureate Elementary Teacher Education (ETE) in Utah, that is, to explore the process of developing learning outcomes and competencies for ETE and analyze the role of faculty and the role of the Utah State Board of Regents. Confidentiality statement: The information you provide will be kept confidential to the greatest degree possible. The consent form explains all necessary details and assurances for voluntary participation and confidentiality. You have the right to withdraw from the study at any point in time, without any negative consequences. If you asked to remove some data from the dissertation report, your request would be accepted and satisfied. PreInterview Notes: (Description of setting or any contextual issues that may influence the interview) ___________________________________________________________________ ___________________________________________________________________ 279 [e.g., "The questions in this semistructured interview are intended to guide a natural conversation while also ensuring that each topic area is covered. If the answer to any question has already been addressed in an earlier part of the interview, I will continue to the next question."] Research Questions -RQ/ Main Protocol Questions -MPQ /Prompts Research Questions / Main Protocol Questions /Prompts 1. How have faculty been engaged in the Utah Tuning project? 1. What role have faculty played in tuning Elementary Teacher Education in Utah?  What has been your role (e.g., participant, leadership, facilitator, and liaison, etc.) in the state, or your university, and your program's tuning efforts? How did you play your role? What facilitated this role?  How did the Tuning team develop student learning outcomes and competencies? How was this communicated to/with faculty in ETE programs? What strategies did the Utah tuning team use to develop student learning outcomes and competencies for ETE?  How did the faculty develop student learning outcomes?  What challenges did your Tuning team face in developing the ETE student learning outcomes and competencies?  What is the implication of Tuning for you and your department/program? Which competencies are necessary to teach ETE as a tuned discipline (design/delivery/evaluation)? How will these be developed?  How did the team approach Tuning ETE? 2. How has the Tuning project influenced Elementary Teacher Education in Utah? Type of Question Types of Interviews: Individual (Ind.) /Focus Group (FG) RQ MPQ Ind. prompt Ind. prompt Ind. prompt Ind. prompt Ind. FG RQ 280 Research Questions / Main Protocol Questions /Prompts 2. What influence has the Tuning project had on Elementary Teacher Education in Utah?  What new skills, strategies, knowledge, and competencies have you acquired as a faculty member in ETE tuning?  How have the indicators of quality of a college degree in ETE changed from the tuning process?  How has ETE Tuning initiative been useful in program development and /or refinement? What made this Tuning initiative so useful?  Please tell me how Tuning has made a difference in how course(s) are designed, taught, assessed, to your interaction with students, your discipline/department, and your institution.  How has the Tuning process affected your understanding of the ETE discipline? (e.g., curriculum, syllabus, vocabulary, assessment, etc.) 3. a) Who provides the leadership direction for tuning Elementary Teacher Education in Utah? 3. What are the roles of the various Tuning actors (USHE, universities, colleges, faculty, others)?  What support did faculty need to tune the ETE discipline? (e.g., appreciation of their work, discussion at the department, etc.)  What support did faculty get from USHE, universities, colleges, faculty, others to tune the ETE discipline? (e.g., some training, joint meetings, department chairs' approval, political role to help faculty, programs mediate changes that resulted from the tuning process, etc.)  What other roles besides support and conditions were necessary? Type of Question MPQ Types of Interviews: Individual (Ind.) /Focus Group (FG) Ind. prompt Ind. prompt Ind. FG FG FG RQ MPQ Ind. prompt Ind. prompt Ind. 281 Research Questions / Main Protocol Questions /Prompts Type of Question  What were the conditions that helped you to do the ETE Tuning work?  What organizational and institutional changes were needed to implement ETE tuning? What was needed but not present? 3. b) What factors have advanced the Utah Tuning project? 4. What factors have been used to advance the Utah Tuning project?  What did you think when changes from Tuning were introduced to you? How did you respond when you were informed about changes that emerged from the Tuning project?  What difficulties did you have coping with the proposed (Tuning) changes/opportunities (e.g., changes to curriculum, to your teaching, to your expectations of student learning)? How were these challenges addressed?  What are the most important elements of ETE Tuning process that need to be focused on by faculty, department chair, or anybody else to advance the Utah Tuning project?  What is the current state of the ETE Tuning process in Utah or in your department?  What additional comments regarding the ETE Tuning process do you have? Types of Interviews: Individual (Ind.) /Focus Group (FG) FG. FG. RQ MPQ Ind. prompt Ind. prompt FG. FG. APPENDIX G INFORMED CONSENT STATEMENT: TUNING ELEMENTARY TEACHER EDUCATION IN UTAH 283 BACKGROUND You are being asked to take part in a research study. Before you decide it is important for you to understand why the research is being done and what it will involve. Please take time to read the following information carefully. Ask me if there is anything that is not clear or if you would like more information. Take time to decide whether you want to volunteer to take part in this study. The purpose of the study is to examine the Tuning process for baccalaureate Elementary Teacher Education (ETE) in Utah, that is, to explore the process of developing learning outcomes and competencies for ETE and analyze the role of faculty and the role of the Utah State Board of Regents. STUDY PROCEDURE Your participation in this study requires an interview during which you will be asked questions about your opinions and attitude regarding your experience in the Utah ETE Tuning process. The duration of the interview will be approximately 60 minutes. With your permission, the interview will be audiotaped and transcribed. The purpose of this procedure is to capture and maintain an accurate record of the discussion. Your name will not be used at all. On all transcripts and data collected you will be referred to only by way of a pseudonym (which you will select). This study will be conducted by Natalia Ralyk, a doctoral candidate in the Department of Educational Leadership and Policy at the University of Utah. The interview will be undertaken at a time and location that is mutually suitable. As part of this study you will be asked to take part in a focus group. It will take you approximately 2 hours to complete this study. Questions will be asked about 284 "'Tuning" Elementary Teacher Education in Utah". The following procedures are considered experimental. RISKS The risks of this study are minimal. You may feel upset thinking about or talking about personal information related to "'Tuning' Elementary Teacher Education in Utah". Participation in this study carries the same amount of risk that individuals will encounter during a usual classroom activity. If you feel upset from this experience, you can tell the researcher, and I will tell you about resources available to help. BENEFITS There are no direct benefits for taking part in this study. However, this research will potentially contribute to the theory and practice of Tuning and learning-centered teaching and competency-based needs for higher education. This study may help develop a greater understanding of Tuning Elementary Teacher Education in Utah Colleges and Universities in the future. CONFIDENTIALITY Under no circumstances whatsoever will you be identified by name in the course of this research study, or in any publication thereof. Every effort will be made that all information provided by you will be treated as strictly confidential. All data will be coded and securely stored, and will be used for professional purposes only. Any information derived from the research that personally identifies me will not be voluntarily released or disclosed without my separate consent, except as specifically required by law. The researcher will keep all research records that identify you private to the extent allowed by law. Records about you will be kept locked in a filing cabinet and on private 285 computer protected with a password. Only those who work with this study will be allowed access to your information. Your pseudonym name will be kept with your responses from the interview and focus group. In publications, only your pseudonym name will be used. This research study is to be submitted in partial fulfillment of requirements for the degree of Doctor of Philosophy at the College of Education, University of Utah, Salt Lake City, Utah. The results of this study will be published as a dissertation. In addition, information may be used for educational purposes in professional presentations and/or educational publications. Person to Contact If at any time you have any questions regarding the research or your participation, you can contact the researcher, Natalia Ralyk, who will answer your questions, who may be reached during 7 am - 11 pm. The researcher's phone number is [telephone], email [e-mail] . You may also contact the researcher's faculty advisor, Dr. Andrea Rorrer, at [telephone] during 9 am - 5 pm. Institutional Review Board: Contact the Institutional Review Board (IRB) if you have questions regarding your rights as a research participant. Also, contact the IRB if you have questions, complaints or concerns which you do not feel you can discuss with the investigator. The University of Utah IRB may be reached by phone at (801) 5813655 or by e-mail at [e-mail]. Research Participant Advocate: You may also contact the Research Participant Advocate (RPA) by phone at [phone] or by email at [e-mail]. 286 VOLUNTARY PARTICIPATION Your participation in this research is voluntary. You may refuse to participate or withdraw from participation at any time without any future jeopardy. The researcher can withdraw you from the research at her professional discretion. If, during the course of the study, significant new information that has been developed becomes available that may relate to your willingness to continue to participate, the researcher will provide this information to you. If you volunteer to participate, please, sign this consent form, scan it, and send it to my email address: [e-mail] in a week since you get this consent document. COSTS AND COMPENSATION TO PARTICIPANTS There is no financial remuneration for your participation in this study. CONSENT By signing this consent form, I confirm I have read the information in this consent form and have had the opportunity to ask questions. I will be given a signed copy of this consent form. I voluntarily agree to take part in this study. ___________________________________ Printed Name of Participant ___________________________________ Signature of Participant ___________________________________ Printed Name of Person Obtaining Consent ___________________________________ Signature of Person Obtaining Consent ______________________ Date ______________________ Date APPENDIX H TUNING REPORTS 288 ICHE. (2010). Tuning USA Indiana final report. Indiana Commission for Higher Education. Indianapolis, IN: Author. IEBC. (2012). Tuning American higher education: The process. Institute for EvidenceBased Change. CA: Author. MNOHE. (2010). Tuning USA Minnesota final report. The Minnesota Office of Higher Education. St. Paul, MN: Author. THECB. (2011). Making opportunity affordable in Texas: A student-centered approach. Texas Higher Education Coordinating Board. Austin, TX: Author. Stein, R., & Reinert, L. (2014). Transparent pathways, clear outcomes: Using disciplinary tuning to improve teaching, learning, and student success. Midwestern Higher Education Compact. APPENDIX I UTAH TUNING REPORTS 290 USHE. (2009). Tuning USA Utah interim report. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2011). Grantee final narrative report outline. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2011). Tuning USA Utah final report. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2012). Grantee interim narrative report. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2013). Grantee interim narrative report. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2014). Grantee final narrative report. Utah State Board of Regents. Salt Lake City, UT: Author. For the full texts of the reports see http://utahtuning.weebly.com/utah-tuning-reports.html APPENDIX J ETE TUNING MEETING NOTES LIST 292 Utah System of Higher Education. (2011, October 21). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2011, December 2). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2012, February 10). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2012, April 2). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2012, May 8). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2012, May 21). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2012, November 30). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2013, January 25). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2013, February 22). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2013, April 3). Elementary education Tuning team meeting. The Board of Regents, Utah. Utah System of Higher Education. (2013, May 6). Elementary education Tuning team meeting. The Board of Regents, Utah. APPENDIX K ELEMENTARY EDUCATION MAJOR'S MEETINGS NOTES LIST 294 Utah System of Higher Education. (2012, September 27). Major's meeting agenda: Elementary education. The Board of Regents, Utah. Utah System of Higher Education. (2012, September 27). Elementary education major's meeting. The Board of Regents, Utah. Utah System of Higher Education. (2013, September 27). Elementary education major's meeting. The Board of Regents, Utah. Utah System of Higher Education. (2013, February 22). Elementary education major's meeting. The Board of Regents, Utah. Utah System of Higher Education. (2014, September 27). Elementary education major's meeting with Tuning group. The Board of Regents, Utah. REFERENCES Adelman, C. (2008a). Accountability ‘Light': Our version is going the way of the dollar vs. the euro. Liberal Education, 94(4), 8-9. Adelman, C. (2008b). The Bologna club: What U.S. higher education can learn from a decade of European reconstruction. Institute for Higher Education Policy. Retrieved from http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED501332 Adelman, C. (2009). The Bologna process for U.S. eyes: Relearning higher education in the age of convergence. Washington, DC: Institute for Higher Education Policy. www.ihep.org/Research/GlobalPerformance.cfm. Adler, P. & Adler, P. (1994). Observational techniques. In N. Denzin & Y. Lincoln (Eds.), Handbook of Qualitative Research (pp. 377-392). Thousand Oakes, CA: Sage. Altbach, P. G. (2005). Harsh realities: The professoriate faces a new century. In P.G. Altbach, R.O. Berdahl, & P.J. Gumport (Eds.), American higher education in the twenty-first century (pp. 287-314). Baltimore: The John Hopkins University Press. Alvesson, M. & Karreman, D. (2000). Taking the linguistic turn in organizational research: challenges, responses, consequences. Journal of Applied Behavioral Science, 36(2), 136-158. American Educational Research Association. (1992). Ethical standards of the American Educational Research Association. An Introduction to LEAP (AAC&U). (n.d.). Retrieved November 6, 2013, from http://www.aacu.org/leap/documents/Introduction_to_LEAP.pdf Argote, L. (2011). Organizational leaning research: Past, present, and future. Management Learning, 42(4), pp. 439-446. Argyris, C. & Schön, D. (1996). Organizational learning II. Reading, MA, AddisonWesley. Association of American Colleges, (1992). Program review and educational quality in the major. Project on liberal learning, study-in-depth, and arts and science major. 296 Washington, D.C. Astin, A. (1993). What matters in college: Four critical years revisited. San Francisco: Jossey-Bass. Ball, Ch. (1985). Essays in Higher Education. The Society for Research into Higher Education & NFER-Nelson, Surrey, Guildford, 1985. Bakhtin, M. (1981). The dialogic imagination. Austin, TX, University of Austin Press. Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), pp. 1-26. Baron, J. N., Dobbin, F. R. & Jennings, P. D. (1986). War and peace: the evolution of modern personnel administration in U.S. industry. American Journal of Sociology, 92, 350-383. Barr, H. (1996). Ends and means in interprofessional education: Towards a typology. Education for Health, 9, 341 -352 Battilana, J., & D'Aunno, T. A. (2009). Institutional work and the paradox of embedded agency. In T. B. Lawrence, R. Suddaby, & B. Leca (Ed.), Institutional work: Actors and agency in institutional studies of organizations (pp. 31-58). Cambridge, UK: Cambridge University Press. Battilana, J., Leca, B., & Boxenbaum, E. (2009). How actors change institutions: Towards a theory of institutional entrepreneurship. The Academy of Management Annals, 3(1), 65-107. Benjamin, R. & Carroll, S. (1998). The implications of the changed environment for governance in higher education. In W. Tierney, The responsive university: Restructuring for high performance (pp. 92-119). Baltimore: The Johns Hopkins. Birtwistle, T., & McKiernan, H. H. (2008). The changing landscape of higher education: An analysis of how national change might be brought about in American higher education compared with the Bologna signatory states. Education and the Law, 20(4), 317-336. Bloomberg, L. D. & Volpe, M. (2012). Completing your qualitative dissertation: a road map from beginning to end. 2nd ed., Thousand Oaks, CA: Sage. Bogdan, R. C., & Biklen, S. K. (1998). Qualitative research for education: An introduction to theory and methods. Boston: Allyn & Bacon. Bologna. (2009). Overarching framework for qualifications of the EHEA. Retrieved from http://www.ehea.info/article-details.aspx?ArticleId=67 297 Bolman, L. G. & Deal. T. E. (2003). Reframing organization: Artistry, choice, and leadership. San Francisco: Jossey-Bass. Boreham, N., & Morgan, C. (2004). A sociocultural analysis of organizational learning. Oxford Review of Education, 30(3), 307-325. Bowden, J. & Marton, F. (1998). The University of learning-beyond quality and competence in higher education. London, Kogan Page. Boxenbaum, E. & Pedersen, J. S. (2009). Scandinavian institutionalism - a case of institutional work. In T. Lawrence, R. Suddaby, & B. Leca, Institutional work. (pp. 178-204). Cambridge, UK: Cambridge University Press. Braskamp, L. & Wergin, J. (1998). Forming new social partnerships. In W. Tierney, The responsive university: Restructuring for high performance (pp. 62-91). Baltimore: The Johns Hopkins. Brown, J. S., & Duguid, P. (1991). Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization Science, 2(1), 40-57. Brown, J.S. & Duguid, P. (2000). The social life of information. Cambridge, MA: Harvard Business School Press. Carey, K. (2010, June 06). The Chimera of College Brands. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/The-Chimera-of-CollegeBrands/65764/ Chaffee, E. (1998). Listening to the people we serve. In W. Tierney, The responsive university: Restructuring for high performance (pp. 13-37). Baltimore: The Johns Hopkins. Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA: Sage. Clark, B. R. (1998). Creating entrepreneurial universities: Organizational pathways of transformation. London: IAU Press. Clark, B. R. (2004). Delineating the character of the entrepreneurial university. Higher Education Policy, 17, 355-370. Cochran-Smith, M. (2006). Policy, practice, and politics in teacher education: Editorials from the Journal of Teacher Education. The American Association of Colleges for Teacher Education. Czarniawska, B. & Sevon, G. (1996). Introduction. In B. Czarniawska and G. Sevon 298 (ed.), Translating organizational change (pp. 1-12). Berlin: Walter de Gruyer. Dacin, M. T., & Dacin, P. A. (2008). Traditions as institutionalized practices: implications for deinstitutionalization. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (eds.), Handbook of Organizational Institutionalism, pp. 327-351. London: Sage. Dacin, M.T., Goodstein, J. & Scott, W.R. (2002). Institutional theory and institutional change: introduction to the special research forum. Academy of Management Journal, 45, 45-56. Dacin, M. T., Munir, K., & Tracey, P. (2010). Formal dining at Cambridge colleges: Linking ritual performance and institutional maintenance. Academy of Management Journal, 53, 1393-1418. Davis, G. F. (1991). Agents without principles? The spread of the poison pill through the intercorporate network. Administrative Science Quarterly, 36, 583-413. Davis, G. F., & Anderson, P. J.J., (2008). Social Movements and Failed Institutionalization: Corporate (Non) Response to the AIDS Epidemic. In S. R. Clegg, C. Hardy, T. B. Lawrence & W. R. Nord (eds.), Handbook of Organization Studies, pp. 371-388. London: Sage. Davies, R., & Williams, D. (2012). Utah Tuning 2 project: Interim evaluation report. August 2011-May 2012. Brigham Young University. Provo, UT. Davies, R., & Williams, D. (2013). Utah Tuning 2 project: Interim evaluation report. August 2012-May 2013. Brigham Young University. Provo, UT. Davies, R., & Williams, D. (2014). Utah Tuning project: Final evaluation report. Brigham Young University. Provo, UT. Denzin, N. K., & Lincoln, Y. S. (1994). Introduction: Entering the field of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 1-18). Thousand Oaks, CA: Sage. DiMaggio, P. (1988). Interest and agency in institutional theory. In L. Zucker (ed.) Institutional patterns and organization (pp. 3-21). Cambridge, Ma: Ballinger Press. DiMaggio, P. (1991). Constructing an organizational field. In W. Powell and P. DiMaggio (Eds.), The new institutionalism in organizational analysis (pp. 267292). Chicago: University of Chicago Press. DiMaggio, P. & Powell, W. W. (1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American 299 Sociological Review, 48, 147-160. Dorado, S. (2013). Small groups as context for institutional entrepreneurship: An exploration of the emergence of commercial microfinance in Bolivia. Organization Studies, 34, 533-557. Duke, Ch. (1992). The university of learning: Towards a new paradigm? The Society for Research into Higher Education & Open University Press. Edmunds, UK: St Edmundsbury Press. EAC (2010). Cooperation in higher education between the United States and the European Union to produce a robust methodology to evaluate the application of the Tuning approach. European Commission on Education and Culture. Retrieved from http://tuningacademy.org/wp-content/uploads/2014/02/Tuning-EU-USStudy_Executive-Summary.pdf Eckel. P. D. (2000). The role of shared governance in institutional hard decisions: Enabler or antagonist? The Review of Higher Education, 24(1), 15-39. Elo, S. & Kyngas, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107-115. Emirbayer, M. & Mische, A. (1998). What is agency? American Journal of Sociology, 103(4), pp. 962-1023. Empson, L., Cleaver, I., & Allen, J. (2013). Managing partners and management professionals: Institutional work dyads in professional partnerships. Journal of Management Studies, 50(5), 808-844. Erickson, F. (1986). Qualitative methods in research on teaching. In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 119-161). New York: Macmillan. EUA Tends II. (2001). Towards the European higher education area: survey of main reforms from Bologna to Prague. Retrieved from http://www.eua.be/eua/jsp/en/upload/OFFDOC_BP_trend_II.1068715483262.pdf EUA Trends III. (2003). Bologna four years after: Steps toward sustainable reform in higher education of Europe. Retrieved from http://www.ond.vlaanderen.be/hogeronderwijs/bologna/documents/EUA_Trends_ Reports.htm Evenson, W. (2010, November 7). Relationship of tuning and disciplinary accreditation. WEE [white paper]. Available at (www.tuningusa.org/library) Evenson, W. E. (2012). Strengthening student learning through "Tuning". A Journal of Science, Technology, Ethics, and Policy, 18-24. 300 Ewell, P. (1998). Achieving high performance: The policy dimension. In W. Tierney, The responsive university: Restructuring for high performance (pp. 120-161). Baltimore: The Johns Hopkins. Fattah, G. (2011, September 12). Giving higher education a tune-up: Utah takes steps to ensure students have right skills for workforce. The Deseret News, p. 3. Fiss, P. C. (2008). Institutions and Corporate Governance. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (eds.), Handbook of Organizational Institutionalism, pp. 389-410. London: Sage. Fligstein, N. (2001). Social skill and the theory of fields. Sociological Theory, 19, 105125. Fontana, A. & Frey, J. H. (2003). The interview: From structured questions to negotiated text. In N. K. Denzin & Y. S. Lincoln (Eds.), Collecting and interpreting qualitative materials (2nd ed., pp. 61-106). Thousand Oaks, CA: Sage Publications. Ford, J. D. & Ford, L. W. (1995). The role of conversations in producing intentional change in organizations. Academy of Management Review, 20(3), 541-570. Foster, A. (2004). A nonlinear model of information-seeking behavior. Journal of the American Society for Information Science & Technology, 55(3), 228-237. Framework. (2005). Overarching framework for qualifications of the EHEA. Retrieved from http://www.ehea.info/article-details.aspx?ArticleId=67 Gaston, P. (2008). Bologna: A challenge for liberal education and an exceptional opportunity. Liberal Education, 94(4). Retrieved from http://www.aacu.org/liberaleducation/le-fa08/le-fa08_Gaston.cfm Gaston, P. (2010). The challenge of Bologna: What the United States higher education has to learn from Europe, and why it matters that we learn it. Sterling, VA: Stylus Publishers. Glaser, B. & Strauss, A. (1998). The discovery of grounded theory: Strategies for qualitative research. New York: Aldine De Gruyter. Gonzalez, J. (2004). The TUNING methodology, Retrieved from http://www.sefi.be/technotn/wp-content/uploads/the-tuning-methodology-juliagonzalez.ppt. Gonzales, J., & Wagenaar, R. (2008). Introduction to the third cycle (doctoral) studies as part of the Tuning "Process." Retrieved from http://www.unideusto.org/tuningeu/images/stories/Third_cycle/INTRODUCTION 301 _TO_THIRD_CYCLE.pdf Goodall, H. L. (2000). Writing the new ethnography. Lanham, MD: AltaMira. Green, M. F. (1997). Transforming higher education: Views from leaders around the world. American Council on education: Oryx Press series on higher education. Greenwood, R. & Hinings, C.R. (1996). Understanding radical organizational change: Bringing together the old and the new institutionalism, Academy of Management Review, 21, 1022-1054. Greenwood, R., Oliver, C., Suddaby, R., & Sahlin, K. (Eds.). (2008). The Sage handbook of organizational institutionalism. London: Sage Publications. Greenwood, R., Suddaby, R. & Hinings, C.R. (2002). Theorizing change: The role of professional associations in the transformation of institutionalized fields. Academy of Management Journal, 45, 58-80. Guba, E. G. & Lincoln, Y. S. (1981). Effective evaluation. San Francisco: Jossey-Bass. Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigm in qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 105117). Thousand Oaks, CA: Sage. Gvaramadze, I. (2008). From quality assurance to quality enhancement in the European higher education area. European Journal of Education, 43(4), 443-455. Hardy, C., & Maguire, S. (2008). Institutional entrepreneurship. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (eds.), Handbook of Organizational Institutionalism, pp. 198-217. London: Sage. Hargrave, T. J., & Van de Ven, A. (2009). Institutional work as the creative embrace of contradiction. In T. B. Lawrence, R. Suddaby, & B. Leca (Ed.), Institutional work: Actors and agency in institutional studies of organizations, pp. 120-140. Cambridge, UK: Cambridge University Press. Harvey, L. & Knight, P. (1996). Transforming higher education. Buckingham: Open University Press. Harsh, Sh. (2010, October). Gaining perspective on a complex task: A multidimensional approach to capacity building. Appalachia Regional Comprehensive Center (ARCC) at Edvantia. Hart Research. (2013). It takes more than a major: Employer priorities for college learning and student success. An Online Survey Among Employers Conducted On Behalf of: The Association of American Colleges And Universities by Hart 302 Research Associates. Hart Research Associates. Hatch, J. A. (2002). Doing qualitative research in education settings. Albany: State University of New York Press. Haveman, H.A. (1993). Follow the leader: Mimetic isomorphism and entry into new markets. Administrative Science Quarterly, 38, 593-627. Haunschild, P., & Chandler, D. (2008). Institutional-level learning: Learning as a source of institutional change. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (eds.), Handbook of Organizational Institutionalism, pp. 624-649. London: Sage. Haworth, J. G. & Conrad, C. F. (1997). Emblems of quality in higher education: Developing and sustaining high-quality programs. Needham Heights, MA: Allyn & Bacon. Hinings, C. R. & Greenwood, R. (1988). The normative prescription of organizations. In L. Zucker (ed.), Institutional patterns and organizations: Culture and environment (pp. 53-70). Cambridge, MA.: Balinger. Hirsch, P. & Bermiss, Y. S. (2009). Institutional "dirty" work: preserving institutions through strategic decoupling. In T. Lawrence, R. Suddaby, & B. Leca, Institutional work. (pp. 262-283). Cambridge, UK: Cambridge University Press. Holliday, W. (2011, March 30-April 2). Harmonic convergence: Using the Tuning process to build relationships and transform information literacy, Paper presented at the conference, Philadelphia, PA. Horn, I. S. (2005). Learning on the job: A situated account of teacher learning in high school mathematics departments. Cognition and Instruction, 23(2), 207-236. Huckin, T. (2002). Critical discourse analysis and the discourse of condescension. In E. Barton & G. Stygall (Eds.), Discourse studies in composition. (pp. 155- 176). Hampton. ICHE. (2010). Tuning USA Indiana final report. Indiana Commission for Higher Education. Indianapolis, IN: Author. IEBC. (2012). Tuning American higher education: The process. Institute for EvidenceBased Change. CA: Author. Retrieved from http://www.iebcnow.org/IEBCPublicFiles/iebc.public/e8/e86cc217-d67c-4de1b618-77c6986e5537.pdf Jepperson, R. (1991). Institutions, institutional effects, and institutionalization. In W. Powell and P. DiMaggio (Eds.), The new institutionalism in organizational theory (pp. 143-163). Chicago, Il.: University of Chicago Press. 303 Jones, N. (2012). "Tuning" the disciplines. Liberal Education, 98(4), 1-25. Johnstone, B. (2008). Discourse analysis. Malden, MA: Wiley-Blackwell. Keith, K. (1998). The responsive university in the twenty-first century. In In W. Tierney, The responsive university: Restructuring for high performance (pp. 162-172). Baltimore: The Johns Hopkins. Keohane, R.O. (1989). International institutions and state power: Essays in international relations theory. Boulder, CO: Westview. Kezar, A. J. (2001). Understanding and facilitating organizational change in the 21st century: Recent research and conceptualizations. ASHE-ERIC Higher Education Report, 28(4). Jossey-Bass Higher and Adult Education Series. Retrieved from http://eric.ed.gov/?id=ED457711 Kolb, M., Kalina, M. & Chapman, A. (2013). Tuning toward completion. New Directions for Community Colleges, 113, 57-65. Kraatz, M. S. (2009). Leadership as institutional work: a bridge to the other side. In T. B. Lawrence, R. Suddaby, & B. Leca (Ed.), Institutional work: Actors and agency in institutional studies of organizations, pp. 59-91. Cambridge, UK: Cambridge University Press. Kuh, G. (1981). Indices of quality in the undergraduate experience. (AAHE-ERIC Higher Education Research Report No. 4). Washington, D.C.: American Association for Higher Education. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press. Lawrence, T. B., & Suddaby, R. (2006). Institutions and institutional work. In S. R. Clegg, C. Hardy, T. B. Lawrence & W. R. Nord (eds.), Handbook of Organization Studies, pp. 215-254. London: Sage. Lawrence, T., Leca, B., & Zilber, T. (2013). Institutional work: Current research, new directions and overlooked issues. Organization Studies, 34(8), 1023-1033. Lawrence, T. B., Suddaby, R., & Leca, B. (2009). Introduction: Theorizing and studying institutional work. In T. B. Lawrence, R. Suddaby, & B. Leca (Eds.), Institutional work: Actors and agency in institutional studies of organizations (pp. 1-27). Cambridge, UK: Cambridge University Press. Leblebici, H., Salancik, G. R., Copay, A. & King, T. (1991). Institutional change and the transformation of interorganizational fields: An organizational history of the U.S. radio broadcasting industry. Administrative Science Quarterly, 36, 333-363. 304 Lincoln, Y. S. & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage. Lofland, J., & Lofland, L. H. (1995). Analyzing social settings: A guide to qualitative observation and analysis (3rd ed.). Belmont, CA: Wadsworth. Lumina Foundation Strategic Plan 2013-2016 | Lumina Foundation. (n.d.). Retrieved November 6, 2013, from http://www.luminafoundation.org/goal_2025.html MacDonald, W. (2014, Fall). Transforming public higher education through student learning outcomes. Public Purpose Magazine. Retrieved from http://news.ets.org/blog/transforming-public-higher-education-through-studentlearning-outcomes Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass Publishers. Marshak R. J. & Grant, D. (2008). Organizational discourse and new organization development practices. British Journal of Management, 19, 7-19. Marshall, C. & Rossman, G. B. (1995). Designing qualitative research (2nd ed.). Thousand Oaks, CA: Sage. Marshall, C. & Rossman, G. (1999). Designing qualitative research. Thousand Oaks, CA: Sage. Marshall, C. & Rossman, G. (2011). Designing qualitative research (3rd ed.). London: Sage Publications. McInerney, D. J. (n.d.) "Accounting 101": How disciplines can answer the "so what?" question about their work and their contributions to higher education. Retrieved from http://utahtuning.weebly.com/index.html. McKiernan, H. H. & Birtwistle, T. (2010). Making the implicit explicit: Demonstrating the value added of higher education by a qualifications framework. Journal of College and University Law, 36(2), 510-564. Merriam, S. B. (1988). Case study research in education: A qualitative approach. San Francisco: Jossey-bass. Merriam, S. B. (1998). Qualitative research and the case study application in education. San Francisco: Jossey-bass. Mertens, D. M. (1998). Research methods in education and psychology: Integrating diversity with qualitative & quantitative approaches. Thousand Oaks, CA: Sage. Meyer, J. (1977). The effects of education as institution. The American Journal of 305 Sociology. 83(1), 53-77. Meyer, J.W., Boli, J., Thomas, G.M. & Ramirez, F.O. (1997). World society and the nation-state. American Journal of Sociology, 103(1), 144-181 Meyer, J. & Rowen, B. (1977). Institutionalized organizations: Formal structure as myth and ceremony. The American Journal of Sociology. 83(2), 340-363. Miles, M. B., & Huberman, A.M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks: Sage. Miles, M. B., Huberman A. M., & Saldana J. (2014). Qualitative data analysis: A methods sourcebook. (3rd ed.). Arizona State University, CA: Sage. MNOHE. (2010). Tuning USA Minnesota final report. The Minnesota Office of Higher Education. St. Paul, MN: Author. Morrow, S. L. & Smith, M. L. (2000). Qualitative research methods. In S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed., pp. 199-230). New York: Wiley. Mumby, D. K. & Clair, R. P. (1997). Organizational discourse. In T. A. Van Dijk (ed.), Discourse as social interaction. London: Sage. Musolf, G. R. (2001). John Dewey's Social psychology and neopragmatism: Theoretical foundations of human agency and social reconstruction. The Social Science Journal, 38, 277-295. Nilsson, W. (2015). Positive institutional work: Exploring institutional work through the lens of positive organizational scholarship. Academy of Management Review, 40(3), 370-398. O' Banion, T. (2014, February) Why education reform fails. Retrieved from http://www.wiche.edu/info/waccal/annual2014/meetingMaterials/whyEducationR eformFails.pdf OECD. (1998). Reviews of national policies for education: Russian Federation. Paris: Author. OECD. (1999). Tertiary education and research in the Russian Federation. Paris: Author. Oliver, C. (1991). Strategic responses to institutional processes. The Academy of Management Review, 16(1), 145-179. Oliver, C. (1992). The antecedents of deinstitutionalization. Organization Studies, 13(4), 306 563-588. Patton, M. Q. (1985). Logical incompatibilities and pragmatism. Evaluation and Program Planning, 8, 307-308. Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage. Patton, M. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage. Peshkin, A. (1988). In search of subjectivity - one's own. Educational Researcher, 17(7), 17-22. Polkinghorne, D. E. (1995). Narrative configuration in qualitative analysis. Qualitative Studies in Education, 8(1), 5-23. Powell, W.W. & DiMaggio, P. (1991). The new institutionalism in organizational analysis. Chicago: University of Chicago. Rao, H., & Kenney M. (2008). New forms as settlements. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (eds.), Handbook of Organizational Institutionalism, pp. 352-370. London: Sage. Roberts, P. W. (2008). Charting progress at the nexus of institutional theory and economics. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (eds.), Handbook of Organizational Institutionalism, pp. 560-572. London: Sage. Rogers, E. M. (2003). Diffusion of innovation. New York: Free Press. Rojas, F. (2010). Power through institutional work: Acquiring academic authority in the 1968 Third World Strike. Academy of Management Journal, 53, 1263-1280. Rorrer, A. K. (2001). Leadership and equity: From reproduction to reconstruction. An institutional analysis. (Unpublished doctoral dissertation). University of Texas, Austin. Rossman, G. B. & Rallis, S. F. (2011). Learning in the field: An introduction to qualitative research (3rd ed.). Thousand Oaks, CA: Sage. Roulston, K. (2010). Reflective interviewing: A guide to theory and practice. London: Sage. Sahlin, K., & Wedlin, L. (2008). Circulating ideas: Imitation, translation and editing. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (Eds.), Handbook of Organizational Institutionalism, pp. 218-242. London: Sage. 307 Saldana, J. (2009). The coding manual for qualitative researchers. Los Angeles, CA: Sage. Saldana, J. (2013). The coding manual for qualitative researchers. (2nd ed.). London: Sage. Safman, Ph. (2012). How we strengthen higher education: The convergence of the interstate passport initiative, Tuning USA, and the quality collaboratives. In Grantee interim narrative report. pp. 123-125. Utah State Board of Regents. Salt Lake City, UT: Author. Schatzki, T. R., Knorr Cetina, K. & Von Savigny, E. (2001). The practice turn in contemporary theory. London: Routledge. Smith, M. L. (1980). Solving for some unknowns in the personal equation [CIRCE Occasional Paper]. Urbana, IL: University of Illinois. Schneiberg, M. & Clemens, E. (2006). The typical tools for the job: Research strategies in institutional analysis. In W. W. Powell and D. L. Jones (Ed.), How institutions change. Chicago: University of Chicago Press. Schneiberg, M., & Lounsbury, M. (2008). Social movements and institutional analysis. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (Eds.), Handbook of Organizational Institutionalism, pp. 650-672. London: Sage. Schneider, C. G. (2013). If competency is the goal, then students' own work is the key to reaching it. Liberal Education, 99(4). Scott, W. R. (1995). Institutions and organizations. Thousand Oaks, CA: Sage. Scott, W. R. (2001). Institutions and organizations (2nd ed.). Thousand Oaks, CA: Sage. Scott, R. W. & Christensen, S. (1995). The institutional construction of organizations: International and longitudinal studies. Thousand Oaks, CA: Sage. Selznick, P. (1957). Leadership in administration: A sociological interpretation. Evanston, IL: Row, Peterson. Senge, P. (1994). The fifth discipline. New York, Doubleday. Sjoberg, G., Williams, N., Vaughn, T., & Sjoberg, A. (1991). The case study approach in social research: Basic methodological issues. In Feagin, J., Orum, A., & Sjoberg, G. (Eds.), A case for the case study. Chapel Hill, NC: The University of North Carolina Press. Singh, J., & Jayanti, R. K. (2013). When institutional work backfires: Organizational 308 control of professional work in the pharmaceutical industry. Journal of Management Studies, 50(5), 900-929. Srikanthan, G., & Dalrymple, J. F. (2002). Developing a holistic model for quality in higher education. Quality in Higher Education, 8(3), 215-224. Stake, R. (1994). Case Studies. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research (236-247). Thousand Oaks, CA: Sage. Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: Sage. Steinert, Y. (2005). Learning together to teach together: Interprofessional education and faculty development. Interprof Care, 1, 60-75. Strauss, A. & Corbin, J. (1998). Grounded theory methodology: An overview. In N. Denzin & Y. Lincoln (Eds.), Strategies of Qualitative Inquiry (pp. 158-183). Thousand Oaks, CA: Sage. Suddaby, R., & Viale, T. (2011). Professionals and field-level change: Institutional work and the professional project. Current Sociology, 59, 423-442. Tesch, R. (1990). Qualitative research: Analysis types & software tools. Bristol, PA: Falmer Press. Tolbert and Zucker (1996). Tolbert, P. and Zucker, L.G. (1996). The institutionalization of institutional theory. In S. R. Clegg, C. Hardy and W. R. Nord (Ed.), The handbook of organization studies (pp. 175-190). London: Sage. Terry, L. S. (2008). The Bologna Process and its impact in Europe: It's so much more than degree changes. Vanderbilt Journal of Transnational Law, 41, 107-228. THECB. (2011). Making opportunity affordable in Texas: A student-centered approach. Texas Higher Education Coordinating Board. Austin, TX: Author. Tierney, B. (1998). Responsive university: Restructuring for high performance. Baltimore and London: John Hopkins University Press. Trank, Ch. & Washington, M. (2009). Maintaining an institution in a contested organizational field: the work of the AACSB and its constituents. In T. Lawrence, R. Suddaby, & B. Leca (Eds.), Institutional work. (pp. 236-261). Cambridge, UK: Cambridge University Press. Tuning. (2004). Tuning educational structures in Europe: What is Tuning? Retrieved November 7, 2012, from http://tuning.unideusto.org/tuningeu/ Tuning. (2006). Tuning educational structures in Europe. Retrieved November 7, 2012, 309 from http://www.ihep.org/assets/files/gcfp-files/Tuning_Objectives.pdf Tuning Glossary. (2013). Retrieved from http://utahtuning.weebly.com/tuningglossary.html Tuning USA, available at http://tuningusa.org/About/What_is_Tuning.aspx USHE. (2009). Tuning USA Utah interim report. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2011). Grantee final narrative report outline. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2011). Tuning USA Utah final report. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2012). Grantee interim narrative report. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2013). Grantee interim narrative report. Utah State Board of Regents. Salt Lake City, UT: Author. USHE. (2014). Grantee final narrative report. Utah State Board of Regents. Salt Lake City, UT: Author. Utah Tuning. http://utahtuning.weebly.com/index.html Wagenaar, R. (March, 2002). Tuning Educational Structure in Europe. Presentation for the pre Berlin Conference Working on the European Dimension of Quality, Amsterdam, March 12-13, 2002, available at http://www.ihep.org/assets/files/ gcfp-files/Tuning_Educational_Structures_in_Europe_Presentation.pdf Walsh, D. J, Tobin, J. J. & Graue, M. E. (1993). The interpretive voice: Qualitative research in early childhood. In B. Spodek (Ed.), Handbook of research on the education of young children (pp. 464-476). New York: Macmillan. WASC Senior College and University Commission. (2013). Degree programs: Meaning, quality, and integrity of the degree. 2013 Handbook of accreditation (pp. 29-30). Retrieved from http://www.wascsenior.org/files/2013_handbook_of_accreditation_1.pdf Weick, K. E. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage. Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press. 310 Wenger, E. (2009). Communities of practice and social learning system: the career of concept. Yin, R. (1984). Case study research: Design and methods. Thousand Oaks, CA: Sage Publications. Yin, R. (1994). Case study research: Design and methods. Thousand Oaks, CA: Sage Publications. Yin, R. (2014). Case study research: Design and methods. Los Angeles: Sage Publications. Zemsky, R. (2009a, September 4). The don'ts of higher ed reform. Inside Higher Education. Retrieved from http://www.insidehighered.com/views/2009/09/04/donts-higher-ed-reform Zemsky, R. (2009b). Making reform work: The case for transforming American higher education. Rutgers University Press. Zietsma, C. & McKnight, B. (2009). Building the iron cage: institutional creation work in the context of competing proto-institutions. In T. Lawrence, R. Suddaby, & B. Leca (Eds.), Institutional work. (pp. 143-177). Cambridge, UK: Cambridge University Press. Zgaga, P. (2003). Bologna Process between Prague and Berlin: Report to the ministers of education of signatory countries. Retrieved from http://www.ehea.info/Uploads/Related%20EU%20activities/Report-fromPraguetoBerlin-Sept2003.pdf Zilber, T. (2002). Institutionalization as an interplay between actions, meanings and actors: The case of a rape crisis center in Israel. Academy of Management Journal, 45, 234-254. Zilber, T. (2008). The work of meanings in institutional processes and thinking. In R. Greenwood, C. Oliver, K. Sahlin & R. Suddaby (Eds.), Handbook of Organizational Institutionalism, pp. 151-170. London: Sage. Zilber, T. (2009). Institutional maintenance as narrative acts. In T. Lawrence, R. Suddaby, & B. Leca, Institutional work, pp. 205-235. Cambridge, UK: Cambridge University Press. Zilber, T. (2013). Institutional logics and institutional work: Should they be agreed? In M. Lounsbury, E. Boxenbaum (ed.). Institutional logics in action, Part A Research in the Sociology of Organizations, 39(A). Emerald Group Publishing Limited, pp. 77-96. 311 Zucker, L. (1988). Institutional patterns and organizations: Culture and environment. Cambridge, MA: Ballinger Publishing Company. Zundel, M., Holt, R., & Cornelissen, J. (2012). Institutional work in the wire: An ethological investigation of flexibility in organizational adaptation. Journal of Management Inquiry, 22(1), 102-120.