Chapter 5: RetroGuide Evaluation

This chapter presents a RetroGuide (RG) evaluation study which was conducted to assess the flowchart-based modeling approach. This study complements the previously presented case studies in an overall effort to evaluate the RG project. The evaluation was targeted at informatics users with small to moderate analytical experience. Due to RG's key developmental goal to lower the technological barriers for novice users to analyze data stored in an Enterprise Data Warehouse (EDW), this specific target group was deemed most appropriate for the evaluation. A literature review was conducted, looking at how projects similar in nature to RG were evaluated and what study designs and potential measures were used. This review is described in section 5.2. The rest of the chapter describes the design, methodology, results, and discussion of the RG evaluation. This resource structure validation study compared RG with SQL-based tools using a sample of nonexpert users. Using the R G approach, the subjects achieved significantly higher scores in solving analytical tasks, and also scored higher in tasks which required understanding of given analytical solutions. The study demonstrated that most users preferred RG to SQL because RG was easier to learn, it better supported temporal tasks, and it seemed to be a more logical modeling paradigm. Using UTAUT technology acceptance prediction model, the study results suggest that a fully developed, RG-like technology likely would be well accepted by users.