Dependability of college student ratings of teaching and learning quality
Rajat Chadha, Indiana University, United States
Indiana University . Awarded
The U.S. Commission on the Future of Higher Education (2006) has expressed concerns about the quality of higher education. As a response to accountability demands, such as those of the Commission, several institutional-level assessment efforts have been undertaken, such as administration of standardized tests and surveys of student engagement. Nonetheless, universities and colleges have often used student ratings as a means of evaluating courses and instruction—because they are very practical. According to meta-analyses, only a few items on typical course evaluations have been found to be related to student achievement. To address these concerns, Frick, Chadha, Watson, Wang and Green (2008) developed a new course evaluation instrument that consists of nine student rating scales of teaching and learning quality (TALQ). Five TALQ scales measure First Principles of Instruction, which were derived from a synthesis of instructional theories by Merrill (2002). Other TALQ scales measure student perceptions of successful engagement (academic learning time) and learning progress. The present study examines (1) within-class agreement among students on TALQ items that measure instructor-level variables and (2) the dependability of TALQ scale scores, since these psychometric properties have not been previously addressed.
The TALQ was administered near the end of the semester to 464 students in 12 classes taught by 8 professors at a large Midwestern university. Measures of within-class agreement on TALQ items were found to be satisfactory on 27 out of 30 TALQ items in 9 of the 12 classes. Results of two-facet and three-facet generalizability study designs revealed that TALQ scale scores were dependable for student ratings of overall course and instructor quality, satisfaction, learning progress and on 3 First Principles of Instruction (activation, application and integration). Two of the First Principles scale scores were found to be less dependable (authentic problems and instructor demonstration), as well as student academic learning time. Based on these findings, change in 3 items are suggested. Changes in 5 items on the academic learning time scale are also recommended. Future validation studies are recommended that investigate the use of the modified TALQ scales for improving the quality of teaching in postsecondary education.
Chadha, R. Dependability of college student ratings of teaching and learning quality. Ph.D. thesis, Indiana University.
Citation reproduced with permission of ProQuest LLC.
For copies of dissertations and theses: (800) 521-0600/(734) 761-4700 or https://dissexpress.umi.com