A Forced Choice Method for Evaluating Instruction in Higher Education

Update item information
Publication Type dissertation
School or College College of Education
Department Educational Psychology
Author St. Andre, Mark Edward
Title A Forced Choice Method for Evaluating Instruction in Higher Education
Date 2017
Description Student evaluations of teaching (SET) in higher education are ubiquitous. Many SETs utilize Likert-type scales and data from these evaluations are frequently used as data points in the retention, promotion, and tenure reviews of instructors. This makes them a high impact activity. Research has revealed a number of validity issues with Likert scales that cast doubt on the validity of the results produced by them. The current study proposed a new method for evaluating instruction in higher education that utilizes the forced choice method popularized by personality psychology. This method uses confirmatory factor analytic techniques to avoid the ipsative data problem of conventional forced choice scoring. The current study delivered both a Likert scale (SL) and a forced choice (SFC) version of an instructor evaluation instrument to students. Instructors' self-evaluation and department chairs' evaluation of their instructors served as dependent variables. Results showed that the forced choice method had some indication of better discriminant validity, evidenced by lower correlations between three forced choice factors (ranging from .12 to .63), compared to the Likert version (ranging from .95 to .96). Fewer responses than anticipated resulted in a simplified analysis of convergent validity. Neither of the trait estimates generated by the two student instructor evaluation methods correlated with those generated by the instructor self-evaluations. The Student Likert instrument had low to moderate correlations and the Student Forced Choice instrument had low correlations with the department chair evaluations of instructors. The slightly greater evidence of convergence for the Likert compared to the forced choice may be explained, in part, by a method effect, since the department chair instrument also used a Likert scale. Discussion focused on the surprising finding that students, instructors, and department chairs largely disagreed on the quality of instructors, and on how to improve the instrument and study method for more robust results.
Type Text
Publisher University of Uta
Subject Educational evaluation; Educational psychology; Higher education
Dissertation Name Doctor of Philosophy
Language eng
Rights Management (c) Mark Edward St. André
Format Medium application/pdf
ARK ark:/87278/s6f530xh
Setname ir_etd
Date Created 2020-05-05
Date Modified 2020-05-06
ID 1547093
Reference URL https://collections.lib.utah.edu/ark:/87278/s6f530xh
Back to Search Results