Agile in the Wild - An in-Depth look into EAE Student Development Practice

There have been works produced on Agile techniques and their e#11;ectiveness in terms of both actual project progress as well as human factors, especially in industry. There are many training programs that companies enroll their employees in, such as ASPE. In these training programs, individuals are taught Agile in a variety of ways including Scrum and Extreme Programming (XP). However, there is a noticeable lack of focus on what Agile looks like from a grounded perspective; in other words, from the perspective of someone participating in the pro- cess. This absence of detailed, on-the-ground study is a missed opportunity. In this work, I study EAE Student Game Development Practices from a grounded perspective, in the context of the EAE Undergraduate Capstone Game named Fast Travel: Loot Delivery, on which I am an engineer. The Capstone class is structured such that students are expected to develop their games using the Scrum Agile de- velopment process. The syllabus states that one of the learning outcomes for the semester is: \Students will learn the scrum agile process and how to apply it in the development of a software product." I am interested in analyzing the e#11;ect that using Scrum has on the overall development process and its people. In the course of this study, my goal was to improve individual awareness of Agile methods and techniques, improve the overall game development process for my team, and critically evaluate the way Scrum is taught and structured in the EAE Capstone course.

AGILE IN THE WILD - AN IN-DEPTH LOOK INTO EAE STUDENT DEVELOPMENT PRACTICE by Jacob Sivo A Senior Honors Thesis Submitted to the Faculty of The University of Utah In Partial Fulfillment of the Requirements for the Honors Degree in Bachelor of Science In Computer Science Approved: Rogelio E. Cardona-Rivera, PhD Thesis Faculty Supervisor Mike Kirby, PhD Associate Director, School of Computing Erin Parker, PhD Honors Faculty Advisor Sylvia D. Torti, PhD Dean, Honors College May 2019 Copyright c 2019 All Rights Reserved ABSTRACT There have been works produced on Agile techniques and their effectiveness in terms of both actual project progress as well as human factors, especially in industry. There are many training programs that companies enroll their employees in, such as ASPE. In these training programs, individuals are taught Agile in a variety of ways including Scrum and Extreme Programming (XP). However, there is a noticeable lack of focus on what Agile looks like from a grounded perspective; in other words, from the perspective of someone participating in the process. This absence of detailed, on-the-ground study is a missed opportunity. In this work, I study EAE Student Game Development Practices from a grounded perspective, in the context of the EAE Undergraduate Capstone Game named Fast Travel: Loot Delivery, on which I am an engineer. The Capstone class is structured such that students are expected to develop their games using the Scrum Agile development process. The syllabus states that one of the learning outcomes for the semester is: “Students will learn the scrum agile process and how to apply it in the development of a software product.” I am interested in analyzing the effect that using Scrum has on the overall development process and its people. In the course of this study, my goal was to improve individual awareness of Agile methods and techniques, improve the overall game development process for my team, and critically evaluate the way Scrum is taught and structured in the EAE Capstone course. ii TABLE OF CONTENTS ABSTRACT ii INTRODUCTION 1 METHODS 8 RESULTS 15 LIMITATIONS 28 DISCUSSION 29 REFERENCES 32 iii 1 INTRODUCTION As software development grew over time into the massive industry it is today, it was the same way many other large scale projects in the corporate world were. The project would be planned out, developed until completion, then tested. Only once it had passed all tests would the product be released. This this development method is known as the ”Waterfall Method”. One of the most famous descriptions of this style was written by Winston W. Royce, and consisted of seven stages, from deciding upon the system requirements to the operation of the published system. The steps come one after the other in strict order, similarly to a waterfall. It is a rigid and inflexible system, with heavy emphasis on documentation and an analysis of the product at each development level before the process can move to the next. This worked perfectly well for things like helicopters and bridges, but software is a different beast. Royce noted the method’s deficiencies. ”I believe in this concept, but the implementation described above is risky and invites failure.”1 Software development needed change. As a newer industry, it attracted mavericks, who were not satisfied with the state of things and wanted to establish a new way of doing things that better suited the task at hand. In 2001, a group of software engineers met to do just this. They created the Manifesto for Agile Software Development, a guide to a new way of creating software. It sought to address the problems inherent in the Waterfall structure. The creators specified what their new system would value.2 Individuals and interactions over processes and tools 1 Royce,William W. “Managing The Development Of Large Software Systems” IEEE WESCON, Aug. 1970, pp.328–338, www-scf.usc.edu/ csci201/lectures/Lecture11/royce1970.pdf 2 Beedle, Mike, et al. “Manifesto for Agile Software Development.” Manifesto for Agile Software Development, 2001, agilemanifesto.org/. 2 Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan Agile Software development had been born. Development would now be carried out in teams who worked closely together on the same project. Developers would interact with the customer during the development process and, most importantly, have working software to show them. They would develop, test and release all at the same time, rather than setting each a slot in a long schedule, and risk customer disillusionment and code being found insufficient long after the actual coding phase was over. This new development structure would hopefully address all of the problems that Waterfall struggled with. However, precisely how this process would be done was not specified. As a result, many forms of Agile software development have been created, each a variation on the common theme, and each with their own benefits and drawbacks. There is no ”best” method in the same way that there is no ”best” programming language. A team should seek out what works best for them, Extreme Programming, Kanban, Lean Software Development, and many others. In the EAE Undergraduate Thesis course, the chosen method was Scrum. Scrum was first given form by Hirotaka Takeuchi and Ikujiro Nonaka in an article for Harvard Business Review in 1986. The term ”Scrum” was first introduced in this article, as a part of an extended analogy to a rugby game. ”The traditional sequential or “relay race” approach to product development—exemplified by the National Aeronautics and Space Administration’s phased program planning (PPP) system—may conflict with the goals of maximum speed and flexibility. Instead, a holistic or “rugby” approach—where a team tries to go the distance as a unit, passing the ball back and 3 forth—may better serve today’s competitive requirements.”3 . Scrum was given a more formal structure and terminology by Jeff Sutherland and Ken Schwaber, who developed the method over the years, culminating in the current ”Scrum Guide”. The most up to date version, available via the website Scrum.org, was used for this study. Scrum development involves assigning certain roles within the development team, using several organizational techniques and following the ”Sprint” development cycle.4 The Scrum Team The Product Owner - This team member represents the customer’s perspective in development, and forms a bridge of communication between the team and the clients. The Scrum Master - This team member makes sure the Scrum Framework is being adhered to and works to remove obstacles and address issues blocking the Development Team The Development Team - These are the Team members working directly with the product, and are in charge of developing, testing and releasing product increments. The Scrum Artifacts The Product Backlog - This is an ordered list of everything that is known to be needed in the product. The Sprint Backlog - This is the set of Product Backlog items selected for the Sprint, plus a plan for delivering the product Increment and realizing the Sprint Goal. The Increment - This is the sum of all the Product Backlog items completed during a Sprint and the value of the increments of all previous Sprints. 3 Takeuchi, Hirotaka, and Ikujiro Nanaka. “The New New Product Development Game.” Harvard Business Review, Jan. 1986, hbr.org/1986/01/the-new-new-product-development-game. 4 Schwaber, Ken, and Jeff Sutherland. “The Scrum Guide TM.”Scrum Guides, Nov. 2017, www.scrumguides.org/scrum-guide.html. The Scrum Events The Sprint - A time-box of one month or less during which a ”Done”, usable, and potentially releasable product Increment is created. Sprint Planning - The meeting at which the work to be performed in the Sprint is planned. The Daily Scrum - A time-boxed event held every day of the Sprint where the Development Team plans work for the next 24 hours. The Sprint Review - A meeting held at the end of the Sprint to inspect the Increment and adapt the Product Backlog, and for the Scrum Team and stakeholders to collaborate about what was done in the Sprint. The Sprint Retrospective - A meeting providing an opportunity for the Scrum Team to inspect itself and create a plan for improvements to be enacted during the next Sprint. Many companies throughout all industries have begun to adopt this basic structure. According to the Project Management Institute’s 2018 report, 46% of ”...4,455 project management practitioners, 447 senior executives, and 800 project management office (PMO) directors...” studied stated that their organizations used Scrum at least sometimes5 , There exist a multitude of training programs that aim to bring both employees and managers up to speed on the new methods. Courses such as ASPE’s Certified ScrumMaster Workshop offer training in certain roles as well as the general history and methodology. This can be very expensive, with the ASPE CSM course costing $1,350 for a two day course6 . Companies would rather have their employees 5 “Success in Disruptive Times.” PMI, PMI’s Pulse of the Profession, www.pmi.org//media/pmi/documents/public/pdf/learning/thought-leadership/pulse/pulse-of-the-profession2018.pdf 6 “Certified Scrum Master Workshop — CSM Training Course.” Certified Scrum Master Workshop — CSM Training Course, ASPE, aspetraining.com/courses/certified-scrummaster-workshop-csm. 5 know how to do Scrum, rather than pay those high fees. Indeed some universities are beginning to offer in-class training, and the University of Utah is one of these. I have encountered a basic Scrum teaching setting in classes I have taken, often as a last-two-weeks lecture series. However, the first formal effort to have students develop a project with a Scrum framework was the EAE Undergraduate Thesis Course, where students are organized into teams and develop a game over the course of two semesters, from proposing ideas to publishing the game. In the class Syllabus for the first semester, it is stated that one of the desired learning outcomes is that the “Student will learn the scrum agile process and how to apply it in the development of a software product.”7 . It was with this goal in mind that the class was structured. A portion of the first semester was taken up with basic exercises and a quick reintroduction to Scrum, followed by an initial development phase of a number of prototypes using smaller teams. Once the final set of games was decided upon, the students were redistributed into their final teams and began work on their games. The class formally meets twice a week, normally a workday, but sometimes for guest lectures or the class-wide Sprint Reviews where the groups present on their changes to their peers and faculty. At the end of every sprint, which are usually 2-3 weeks long, the students fill out a Sprint Review form, which is designed to assess their team members and their progress. They then submit a working build of their game for grading and present their progress to the class. The Sprint Review form was replaced by a monthly review form in the second semester. The learning goal for students to learn how to use Scrum was also dropped. The teams are encouraged by professors and the sprint review sessions and forms to adhere to a Scrum framework, but there is no formal requirement and no method for professors to be sure whether teams are following the framework or not. The idea is for students to be given the basics, and 7 ”Syllabus”. Syllabus for EAE 4500-001 Fall 2018 Senior https://canvas.utah.edu/courses/511578/assignments/syllabus. Password Protected Project I. 6 then to learn to utilize and implement Scrum on their own, but I hypothesize this to be insufficient, especially given the more strict Scrum adherence found in the workplace, such as that I personally experienced in an internship with Northrop Grumman. My team, Souper Chef, has been developing our game Fast Travel. We have largely stuck to a very informal version of Scrum, with individuals taking up the equivalents of the Scrum roles ad hoc. The game has been playable at the end of every sprint period, but no effort was made to maintain a backlog, track changes, hold retrospectives and reviews, and the Scrum stand-up meeting was sometimes skipped in favor of working. This was the pattern until the introduction and adoption of a more formal structure in the course of this study. I was concerned with this because it seemed as though we were not receiving a proper education in what Scrum actually feels like, and how it should be properly run. I believe that we are not being properly trained for the way things are run in the corporate world and that if we continued with this vague, loose pattern, that we would not develop a proper appreciation for the benefits and downsides of the Scrum method. I therefore decided to introduce a more formal structure into the team’s work, and study the effects of the sudden imposed formality on our productivity and morale. I was concerned also with the lack of comparable data to the kind this study would generate. In a review of Agile studies by Tore Dyba and Torgeir Dingsoyr, they state: “Our review identified 1,996 studies ... 36 of which we found to be research studies of acceptable rigor, credibility, and relevance ...”. Of the 36 studies that were accepted, 33 dealt with XP, and none dealt with Scrum. A subset of those studies dealt with developer perception of agile developments, and those that studied university students did not cite statistics, but asked students opinions at a random point in the 7 process8 . I believe that a detailed, on-the-ground study of not just the process, but the integration of these new Scrum methods on a heretofore casual team can provide vital information for educators and employers seeking how to best teach Scrum, and to other development teams seeking a style of development that suits them. The fact that this study is conducted by a member of the team which is experiencing the change will provide useful perspective to the data. As the project nears completion, the data gathered on student experience, both in past Agile Scrum education and in the process of completing the game, will add to the growing body of knowledge about how Scrum actually works in a student environment. It will also help assess the way Scrum is being taught at the University so as to improve future classes and curricula. In an Agile world, students need and deserve to be as prepared as possible to face the real world environments they will find themselves in post-graduation to pass on that experience to others and further our ability to develop quality software with maximum efficiency and minimum stress on the Development teams. 8 Dyba, T., and T. Dingsoyr. ”What Do We Know about Agile Software Development?” IEEE Software 26, no. 5 (September/October 2009): 6-9. doi:10.1109/ms.2009.145. 8 METHODS My team, from the start, was working in a pseudo-Scrum format. Therefore, a more formal Scrum format could be built on what we already had. It was necessary to introduce a formal, working definition of Scrum to the group. Before this was done, I tested the group on their prior knowledge of Scrum, then polled the group after each sprint to assess progress and changes in opinion about Scrum and the project. All polls were created using Google Forms and contained a variety of questions of different types. My initial poll, designed to assess our group’s prior knowledge and teaching experience of Scrum, was the first data collection point of the study. The goal was to gather information about how rigorously the University teaches Scrum, not just to the computer scientists, but to the designers and artists as well. In the corporate world, teams can consist of people from various backgrounds, and it is important for a team to be on the same footing. This Prior Knowledge poll had two sections. The first section sought to establish the level of education the subject had received on Scrum in a classroom environment. The second section, consisting of short answer questions, asked the subject to define all of the basic Scrum terms listed in the Introduction section, and to describe some aspects in more detail. This would establish the basic knowledge of the group for data collection purposes, and to guide me in how to introduce the formal elements based on that knowledge. QUESTIONS 1. I know what Scrum Development is. [Y/N] 2. I have been taught Scrum in the classroom. [Y/N] 3. If yes, I have been educated about Scrum in: [1,2,3,3+ classes] 4. I know Scrum well. [Strongly Disagree - Strongly Agree] 9 5. I have practiced Scrum development before. [Y/N] 6. If yes, I have practiced it formally, following established Scrum techniques/structure. [Strongly Disagree - Strongly Agree] 7. What are the 5 Scrum Events? [Short Answer] 8. What are the 3 Scrum Artifacts? [Short Answer] 9. What is a Sprint? [Short Answer] 10. What is the role of a Scrum Master? [Short Answer] 11. What are the two questions answered in a Sprint Meeting? [Short Answer] 12. What are the 8 Elements included in a Sprint Review? [Short Answer] 13. What are the 3 things discussed in a Sprint Retrospective? [Short Answer] 14. What is a Product Backlog? [Short Answer] 15. What is a Sprint Backlog? [Short Answer] 16. What is an Increment? [Short Answer] These questions were chosen to give the subjects a basic quiz in the elements that would be present in the new Scrum process being introduced. The subjects were encouraged to research the terms on their own after submitting their answers to aid in the adoption of the new techniques. At the first Sprint meeting after sending out the Prior Knowledge poll, I established our formal structure going forwards. I would function as the Scrum Master. I would handle the product backlog and manage our sprint backlog via Trello, an Agile planning tool. 10 Figure 1: Trello Board as of 4/5/2019 I would begin stand-up meetings and handle running all reviews, planning meetings and retrospectives. I would also continue my role as a member of the Development Team. Due to the nature of our game, being made for a project and not a particular customer in mind, the Product Owner role was informally handled by all team members working with play test data. I also explained the various roles and what was required of the team at each meeting. Additionally, a Project Backlog was established for the remaining tasks for the semester, as there was not one before, and then the first Sprint Backlog was constructed. At an ordinary Sprint Meeting, I would have the team make any last minute additions to the Product Backlog before the official planning began. We would perform a stand-up meeting, and would then start the meeting proper. First, we would go over all of the remaining elements in the Product Backlog and discuss each one, adding new tasks and clarifying existing tasks as needed. Once all the tasks had been discussed, we went through the list again, assigning tasks to the people that volunteered for them. At the bi-weekly class meetings we performed our Sprint meetings, which could not 11 be performed daily due to scheduling difficulties inherent in the lives of students. We would run a stand up and discuss what we had completed and what was causing problems, then would work for the remaining hour of class before determining what we would be working on up to the next class meeting. Out-of-class communication on tasks was done through Discord. On the day the Sprint concluded we would meet as a class and perform a class-wide pseudo-Sprint Review where all teams would display what changes and improvements had been made over the course of the sprint. After this meeting concluded, our team would meet. If a significant portion of the team was there, we would perform our Sprint Review and Sprint Retrospective as one meeting. If not, we would delay it until the next meeting, and would either use the rest of class as a work day or segue into the Sprint Planning meeting, depending on time constraints. In the Sprint Review, I would lead the team in reviewing our Sprint Backlog, then move all completed tasks into a separate backlog that would record what was finished that Sprint. After this, I would have the team discuss what went well and what went poorly during the Sprint, and finish with a general appraisal of how much progress we had made. The Sprint Retrospective was often combined with this meeting, and we discussed what things could be improved, and I asked team members to contribute any improvements they thought ought to be implemented, and we would shortly discuss how we would do that. After each Sprint was complete, I would send out a post-Sprint form to the team to assess how the team felt about the Scrum Process, the team’s progress and their own progress. The goal was to see if there were improvements in these areas, and whether the adoption of Scrum had a positive effect on the team. QUESTIONS 1. I am satisfied with the amount of work I completed this sprint. [Strongly Disagree 12 - Strongly Agree] 2. I am not satisfied with the amount of work I completed this sprint. [Strongly Disagree - Strongly Agree] 3. The team completed more work than the last sprint. [Strongly Disagree - Strongly Agree] 4. The team completed more work than any previous sprint. [Strongly Disagree Strongly Agree] 5. I am satisfied with the amount of work the team completed this sprint. [Strongly Disagree - Strongly Agree] 6. I am not satisfied with the amount of work the team completed this sprint. [Strongly Disagree - Strongly Agree] 7. The team completed more work than an average sprint last semester. [Strongly Disagree - Strongly Agree] 8. I completed more work than an average sprint last semester. [Strongly Disagree Strongly Agree] 9. I feel positive about the future of the project. [Strongly Disagree - Strongly Agree] 10. I feel negative about the future of the project. [Strongly Disagree - Strongly Agree] 11. I enjoyed the sprint meeting. [Strongly Disagree - Strongly Agree] 12. I did not enjoy the sprint meeting. [Strongly Disagree - Strongly Agree] 13. I feel positive about Scrum techniques and practices. [Strongly Disagree - Strongly Agree] 14. I feel negative about Scrum techniques and practices. [Strongly Disagree Strongly Agree] 15. Comments/Suggestions These questions were designed to assess the state of the team, project, and Scrum 13 adoption, and repetition was included to ensure consistency of data. This is especially important, as all data collection in this study is retrospective by necessity. Additional data is provided by looking at our art assets stored on Google Drive and recording what assets, and how many, were uploaded on what dates. Similarly, the project GitHub shows dates for all of the commits and branch publishes in the history of the project, allowing comparisons between pre and post-introduction of the formal Scrum elements. These are valuable data as they are disconnected from personal memory and allow a hard numbers look into the changes, if any, in efficiency and volume of work completed. Due to the highly subjective nature of this sort of study, it is important to have some hard data on the ongoing process to compliment the opinions of the team. Figure 2: Google Drive as of 4/5/2019 14 Figure 3: GitHub Statistics as of 4/5/2019 15 RESULTS Strongly Disagree - Strongly Agree is notated as 1 - 5, the ”Years” are left off, and for the Short Answer sections, ”Partial” covers all non-complete correct answers. SCRUM KNOWLEDGE POLL Yes 90% Yes 100% No 10% Figure 4: Question 1 / Question 2 2 20% 1 80% 3 - 3+ 0% 3 60% 2 20% Figure 5: Question 3 / Question 4 1 10% 5 0% 4 10% 16 2 20% 1 20% Yes 60% 5 10% 4 0% 3 50% No 40% Figure 6: Question 5 / Question 6 Incorrect 60% Partial 40% Incorrect 50% Partial 20% Figure 7: Question 7 / Question 8 Correct 30% 17 Incorrect 60% Partial 90% Correct 10% Partial 40% Figure 8: Question 9 / Question 10 Incorrect 80% Incorrect 90% Correct 10% Partial 10% Figure 9: Question 11 / Question 12 Partial 10% 18 Incorrect 20% Incorrect 50% Partial 30% Correct 20% Partial 80% Figure 10: Question 13 / Question 14 Incorrect 50% Incorrect 80% Partial 20% Partial 50% Figure 11: Question 15 / Question 16 SPRINT REVIEW POLL 19 5 Num of Resp 4 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 12: Question 1 6 Num of Resp 5 4 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 13: Question 2 20 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 14: Question 3 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 15: Question 4 21 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 16: Question 5 5 Num of Resp 4 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 22 Figure 17: Question 6 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 18: Question 7 23 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 19: Question 8 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 20: Question 9 24 5 Num of Resp 4 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 21: Question 10 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 22: Question 11 25 3 Num of Resp 2.5 2 1.5 1 0.5 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 23: Question 12 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 26 Figure 24: Question 13 4 Num of Resp 3 2 1 0 1 2 3 4 5 Strongly Disagree - Strongly Agree Sprint 1 Sprint 2 Sprint 3 Figure 25: Question 14 None of the Comments/Suggestions questions received submissions on any of the three polls. 27 Figure 26: Additions and Deletions from Project as of 4/13/2019 Figure 27: Commit History as of 4/13/2019 28 LIMITATIONS I encountered several issues during the course of this study, imposed by both circumstance and the nature of the situation being studied. The study was initiated very late into the project, due to external circumstances. As a result, the amount of ”live” data that could be collected from my fellow team members was restricted to three sprints’ worth. There are also concerns about the amount of time that has passed since the earlier sprints of this and last semester for comparison purposes, as memories fade. As a result of this, I could not gather more rigorous personal opinion data about prior sprints, and have compensated for this by comparing numbers gleaned from GitHub analysis. Further issues arose from the nature of the team and my role on it. I adopted the role of Scrum Master and Product Owner together, taking elements from both. I, as a member of the Development Team, had no real administrative authority. Therefore, participation during Daily Scrums, Sprint Meetings, Reviews, etc. and in the polls was entirely optional. All of our meetings took place during class time, which is also technically optional. The team has always had attendance issues with some members, and this carried over into the meetings that took place during the study. I had no authority to force attendance, and no authority to force completion of the polls. As a result, some of the meetings, particularly the Review and Retrospective meetings, have had low attendance, and the polls have had less responses than members. This is unavoidable due to the nature of the study, but the response rate has been roughly 70 percent. 29 DISCUSSION The most striking finding with the data was the lack of formal knowledge of what the various Scrum artifacts are. Few of the respondents answered the questions correctly and completely, though all but one in the study had taken a class which taught Scrum. There seems to be a large lack of information about what the Scrum Master role entails, as well as what exactly is to be done during each meeting type. This is reflected in the very casual manner in which sprints were being run earlier in the semester, and indeed in all other classes I have taken in which Scrum development was feasible. The fact that so many of the questions were answered largely incorrectly is another factor in this, as it is very unlikely that a team would practice a formal form of Scrum development without knowing what the constituent components of Scrum are. 40 percent of my team members had never practiced Scrum before, and 40 percent disagreed that they had practiced Scrum formally, with a further 50 percent on the fence, numbers that are similar to Akif, R.and H. Majeed’s findings, that ”... 50 percent [of] team members lack formal training of scrum and are unaware of the Scrum process. The knowledge that they have gained is either because of the other team fellows or from their Scrum masters.”9 . This certainly was the case in our team, and it suggests a lack of professor-guided training in the classes that we have collectively taken. This could be alleviated by a class more focused on the actual scrum process, rather than the project being undertaken, in order to drill the process into student’s heads. There was very little change observed in the amount of work done as revealed by GitHub. The work done during the last three sprints, the ones being observed, largely follows the trend of spikes and dips established during the course of the year’s work. 9 Akif, R., and H. Majeed. “Issues and Challenges in Scrum Implementation.” International Journal of Scientific & Engineering Research, vol. 3, no. 8, Aug. 2012. IJSER, www.ijser.org/researchpaper/Issues-and-Challenges-in-Scrum-Implementation.pdf. 30 The curve does trend upwards, but there is not enough evidence to suggest that this is not simply the result of last minute project work. The Addition and Deletion graph also does not seem to show a significant change from earlier sprints, other than the large spike in Additions near the end of the graph, which seems to be caused by a glitch with GitHub’s tracking, as the dates are beyond the date on which the data was requested. This indicates that implementing the more formal Scrum techniques did not have a substantial effect on the amount of work done, and indeed, in some Sprints, more work was done in a more casual way. Interestingly, the respondents indicated that, during the first formal sprint, that the team completed more work than in any previous sprint, but this rating fell during the next two even as the number of commits generally rose and the number of additions and subtractions held steady. The first sprint was the one in which the formal Product Backlog was introduced, and several team members expressed surprise at the amount of work that was left to do, as well as the amount of work tasked for the sprint. Given the fairly small change in the number of commits etc. in the next few sprints, I believe that this perception of getting more work done is partially due to an increased awareness of the workload, which sprint planning naturally brings about during the construction of backlogs. This positive reaction indicates that the act of organizing and plainly stating the workload, as dictated by Scrum, had a positive effect on the team’s perception of their own efficiency. This effect does seem to be limited however, as the team slowly became less confident of improvement over all prior sprints, but this may also be due to a general decrease in workload on the team as the project winds down and the number of remaining tasks decreases. Due to the limitations listed above, there is not strong enough evidence here to recommend or discourage the formal implementation of Scrum techniques. A longer term study, perhaps performed over the course of multiple years, would likely provide 31 more clear results. Although most of the respondents were positive about the Scrum experience and the project, this coincided with the wind-down and completion of the project, which had been going according to schedule before formal Scrum implementation. This calls the value of these results into question. However, the lack of Scrum knowledge among a group of soon to graduate students is noticeable, especially as the school has made some efforts to teach Scrum, as all respondents confirmed. As Scrum becomes more widely adopted, formal training in the Scrum method continues to yield positive results. The Scrum Alliance’s 2017-2018 State of Scrum report found that, of the 2000 Scrum practitioners studied, ”81 percent agree that certification improves practice, and 91 percent of organizations offer some form of training.” 10 . That formal, rigorous training is well within the power of a college classroom to teach. The time is right for the University to offer that training, to better prepare students for their careers and their futures. 10 “STATE OF SCRUM 2017-2018 - Scaling and Agile Transformation.” Scrum Alliance, 2018, www.scrumalliance.org/ScrumRedesignDEVSite/media/ScrumAllianceMedia/Files%20and%20PDFs/State%20of%2 SoSR-Final-Version-(Pages).pdf. 32 REFERENCES Akif, R., and H. Majeed. “Issues and Challenges in Scrum Implementation.” International Journal of Scientific & Engineering Research, vol. 3, no. 8, Aug. 2012. IJSER, www.ijser.org/researchpaper/Issues-and-Challenges-in-Scrum-Implementation.pdf Beedle, Mike, et al. “Manifesto for Agile Software Development.” Manifesto for Agile Software Development, 2001, agilemanifesto.org/ “Certified Scrum Master Workshop — CSM Training Course.” Certified Scrum Master Workshop — CSM Training Course, ASPE, aspetraining.com/courses/certifiedscrummaster-workshop-csm. Dyba, T., and T. Dingsoyr. ”What Do We Know about Agile Software Development?” IEEE Software 26, no. 5 (September/October 2009): 6-9. doi:10.1109/ms.2009.145 Royce,William W. “Managing The Development Of Large Software Systems” IEEE WESCON, Aug. 1970, pp.328–338, www-scf.usc.edu/ csci201/lectures/Lecture11/royce1970.pdf Schwaber, Ken, and Jeff Sutherland. “The Scrum Guide TM.”Scrum Guides, Nov. 2017, www.scrumguides.org/scrum-guide.html “STATE OF SCRUM 2017-2018 - Scaling and Agile Transformation.” Scrum Alliance, 2018, www.scrumalliance.org/ScrumRedesignDEVSite/media/ScrumAllianceMedia/Files%20and%2 SoSR-Final-Version-(Pages).pdf “Success in Disruptive Times.” PMI, PMI’s Pulse of the Profession, www.pmi.org/- 33 /media/pmi/documents/public/pdf/learning/thought-leadership/pulse/pulse-of-the-profession2018.pdf ”Syllabus”. Syllabus for EAE 4500-001 Fall 2018 Senior Project I. https://canvas.utah.edu/courses/5 Password Protected Takeuchi, Hirotaka, and Ikujiro Nanaka. “The New New Product Development Game.” Harvard Business Review, Jan. 1986, hbr.org/1986/01/the-new-new-productdevelopment-game.