 |
 |
|||||||||||
 |
||||||||||||
 |
Improving the Quality of Student Learning The University of Southern Mississippi Based on available data about learning outcomes from the course pilot, what were the impacts of re-design on learning and student development? December 2002 Update: The University of Southern Mississippi has completed pilot (spring 2002) and full implementation (fall 2002) activities designed to transform its World Literature course from a traditional multiple-section format to a single-section, technology-mediated one. In spring 2002 USM offered 5 traditional sections with 175 students and one section of the redesigned course with 128 students. In fall 2002 USM offered only the redesigned course with 406 students. Learning Outcomes
Perhaps the most significant finding was that redesign students who completed the course, regardless of other factors such as gender and race, tended to score below their GPA--that is, for example, a student with a B-level GPA more often than not earned only a C in the course.
Success Rates In spring 2002, taking into account only those students who completed the course, 163 students in traditional sections earned an average score of 74.14 (a mid-C letter grade on a scale where 90-100 = A; 80-89.99 = B; etc.), while 81 students in the redesign section earned an average score of 70.39, a low-C grade. Letter grades in traditional sections are not fully in accord with numeric scores because of what might be called "instructor generosity factor"--that is, the tendency for a student with, say, a high B numeric score to be assigned a letter grade of A for intangible reasons. If one adjusts the dozen or so grades affected by the generosity factor, the percentages of As, Bs, and Cs in traditional sections come more into line with the percentages in the redesign section, where numeric scores translate directly and ungenerously to letter grades. That adjustment is not made here, though it would have the effect of greatly reducing the percentage of traditional section As and shifting the majority of Bs to a majority of Cs--just as in the redesign section. In fall 2002, taking into account only those students who completed the course, 314 redesign students earned an average score of 74.76, a mid-C letter grade, higher than that found in both traditional and redesign groups from the spring 2002 term. Retention
The retention rate was much lower in the redesigned course (63.3%) than the traditional one (93.1%). (For further comparison, the retention rate prior to the introduction of the redesigned course averaged about 85%, with some isolated sections as low as 75%.) Of the 175 students in the 5 traditional sections, 163 completed the course (defined as students who took the final exam) with a grade of A-F. An additional 12 students did not complete the course; instead, they formally withdrew, receiving a grade of WP (withdraw pass) or WF (withdraw fail), or simply stopped attending, thus receiving a grade of F. Of the 128 students in the redesigned section, 81 students completed the course (defined as students who took the final exam) with a grade of A-F. An additional 47 students formally withdrew (10 WPs/WFs), stopped turning in assignments sometime before the final exam (24 Fs), or never started the course but nonetheless remained on the roster (13 Fs). Students in the last two categories will, upon request, have their Fs changed to WPs or WFs depending on how much of the course they completed and how well they were performing when they became inactive; consequently, their Fs are effectively improper withdrawals rather than failures. Some of the students enrolled in the redesign course--for example, the 13 (10.2%) who never once logged into WebCT to begin the course--apparently never intended to be in the class, but likely registered for it solely to boost the number of hours for which they were registered that term. The 24 students (18.8%) who started the course and later stopped submitting assignments are another matter. 7 of these students (5.5%) disappeared in the first quarter of the term; 7 more (5.5%) in the second; 3 (2.3%) in the third; and 7 again (5.5%), probably realizing that it had become mathematically impossible to pass the course, in the final quarter. To these figures one must also add the 10 students (7.8%) who formally withdrew from the course. At best, even discounting those students who were padding their schedules to maintain full-time status, the retention rate in the redesigned section was as low as the lowest traditional sections in fall 2001 and before, and much lower than the traditional sections in spring 2002. One traditional section instructor accounts the unusually high spring traditional student retention rate to fears about the redesigned course. Aware that beginning in fall 2002 the only option would be the redesigned course, conservative students were eager to complete their World Literature requirement in the "normal" way before that option disappeared. Plans to address the retention problem, based on information gleaned from surveys, focus groups, and informal reports, will be in place by fall 2002. It is also important to note that many of the students who failed to complete the redesigned course or who did complete it but with a grade of F were already in one of the university's three academic probation or suspension classifications at the time of the course. In fact 49 of the 128 students enrolled had GPAs below 2.00, and thus had already been flagged as at-risk. Although these 49 sub-2.00 students are not precisely conterminous with the 47 students who received WPs, WFs, and Fs described above, it is clear that the majority of students who did not complete the course or failed it were concurrently failing other courses, were already on academic probation/suspension or about to be placed there because of a continuing pattern of poor academic performance, and finally performed as poorly in World Literature as they did in many other courses.
The retention rate improved in fall 2002. 406 students began and 314 completed the course, a retention rate of 77.3%. Students who did not complete the course fell into several categories: (1) 26 students who formally withdrew from either the course or the university; (2) 17 students who never started the course but nonetheless remained on the roster; and (3) 49 students who stopped turning in assignments sometime before the final exam. Some of the students enrolled in the redesign--for example, the 17 (4.2%) who never once logged into WebCT to begin the course--apparently never intended to be in the class, but likely registered for it solely to boost the number of hours for which they were registered. The 49 students (12.1%) who started the course and later stopped submitting assignments are another matter. 5 of these students (1.0%) disappeared in the first quarter of the term; 6 more (1.5%) in the second; 10 (2.5%) in the third; and 28 (6.9%), probably realizing that it had become mathematically impossible to pass the course, in the final quarter. To account for the full complement of students who did not complete the course, to these figures one must also add the 26 students (6.4%) who formally withdrew from the course or the university. It is also important to note that many of the students who failed to complete the redesigned course or who did complete it but with a grade of F were already in one of the university's three academic probation or suspension classifications at the time of the course. In fact 58 of the 406 students enrolled (14.3%) had GPAs below 2.00, and therefore had already been flagged as at-risk. Such students are difficult to retain under any conditions, and thus it must be remarked that many students who did not complete the redesigned course or failed it were concurrently failing other courses, were already on academic probation/suspension or about to be placed there because of a continuing pattern of poor academic performance, and finally performed as poorly in World Literature as they did in many other courses. Student Satisfaction
Comparing fall 2001 traditional sections to the spring 2002 redesign, the redesign comes out ahead in three of the four target areas, in addition to the overall instructor ranking. Comparing spring traditional sections to the redesign, the redesign comes out ahead in two of four categories, in addition to the overall instructor ranking. We expect improvement in all categories in fall 2002 as the instructors of the redesign section refine their methods in their second pass through the course--just as, it might be noted, the fall 2001 traditional section instructors improved in most categories as they had their second opportunity to teach the materials in spring 2002. One additional measure of satisfaction can be found in answers to a question that asked students to compare the redesign course to all other college courses they have taken and then rank it against the others. Students (N=65) ranked the redesign "outstanding" 10.8% of the time, "above average" 32.3% of the time, "average" 41.5% of the time, and "below average" 15.4% of the time. Given the newness of the redesign approach on our campus, the numbers seem quite good, and we expect these numbers will rise as well as the course becomes more polished in 2002-2003.
Student satisfaction numbers are lower than they were during the spring 2002 pilot by about half a point on a 4-point scale. Although the numbers are still respectable insofar as the numbers in three of the four target areas remain above the "average" or "satisfactory" level, students nonetheless were on the whole often dissatisfied with the redesigned course in the fall. The principle source of that dissatisfaction arose from a fundamental misunderstanding of the hybrid nature of the course, which allowed students the choice of attending class presentations or viewing them online. Many students chose the online option, and then later determined that the option was not the one best suited to their learning styles. However, rather than then making the choice to come to class--an option available to fully three-quarters of the students, who were not concurrently registered for another class at the same time as the presentations--students instead began to complain that a "fully online" course had been foisted upon them. Meetings with students, student government groups, and administrators to remind all that students could make their own choices about how best to engage the course went some distance toward correcting misunderstandings, but, purely in terms of student dissatisfaction, the damage had been done. Several plans, mostly on the public relations front, were put into play immediately to prevent fall's misunderstandings from recurring in spring 2003. Also, at the suggestion of the faculty, a tool that allows students to assess their learning styles before beginning the course has been added so that students can make informed choices about how best to engage the course.
|
 |
Program in Course Redesign Quick Links: Program In Course Redesign Main Page... Lessons Learned: Savings: Project Descriptions: |
 |
||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|