 |
 |
|||||||||||
 |
||||||||||||
 |
Improving the Quality of Student Learning Florida Gulf Coast University Based on available data about learning outcomes from the course pilot, what were the impacts of re-design on learning and student development? In the spring 2002 semester, we offered one section of the redesigned course (30 self-selected students) along with seven sections of the traditional course (200 students). The desired learning outcomes, syllabi, teaching materials, and exams were standardized across all sections so that we were able to gauge which students learned specific material at higher levels. The desired learning outcomes fall into three general areas:
We assessed the development of the content knowledge primarily through three multiple-choice exams, but also through the application of the content knowledge in short essays analyzing artwork. We assessed the development of analytical skills through two sets of essays: a short essay that is included on the exams and a longer essay written out of class (because the longer essay did not have a set, standardized prompt, we did not use it to measure the difference in learning between the redesigned and the traditional courses). Finally, we assessed the development of attitudes through pre- and post-tests and journals. In the area of content knowledge, students demonstrated a markedly enhanced level of learning in the redesigned course. The average score on the standardized exams in the traditional courses was a 70%; in the redesigned course it was a 78%. The number of A-B's increased dramatically, from 31% in the traditional course to 54% in the redesigned course. The number of D-F's dropped from 45% in the traditional course to 21% in the redesigned course. The reason for the increase in performance on the exams is linked to two items. First, students are learning the material from reading the text and taking the practice exams in the redesigned course rather than relying on lectures. Second, students in the redesigned course are learning from taking the practice exams repeatedly; those students who took the practice exams three or more times regularly scored A's on the exams. A more complete analysis broken down by specific questions is underway. The more surprising improvement was in the area of skill development. Students are asked to write a short essay as part of the exam, analyzing a specific work of art using the terms and concepts studied in the text. The same prompt was given to all the sections, and all of the instructors had access to the same teaching materials that were available in the redesigned course. All of the essays were holistically scored together, without the graders knowing which essays came from which classes. Essays were scored on a 4-point scale (4 = A, 3 = B, 2 = C, 1 = D/F). Again, the students did remarkably better in the redesigned course. 6% of the students in the traditional classes received a 4, as compared to 8% in the redesigned course; 25% of the students in the traditional class received a 3, as compared to 36% in the redesigned; 46% of students in the traditional course received a 2, as compared to 56% in the redesigned; and 24% of students in the traditional class received a 1, as compared to 0% in the redesigned. Analysis of the two sets of essays (from the traditional and the redesigned) suggests a clear reason for the difference. The redesigned course included an on-line, peer discussion of sample essays (one strong, one weak), which provided the students with guidance on how to respond to the prompts and how to analyze artworks. While these sample essays were provided to the instructors of the traditional classes, they either did not use them or did not emphasize them in their classes. While we have not completed a full analysis of the pre- and post-tests and the journals, a quick read of these materials from both classes suggest that they are both successful in increasing student attitude towards the arts. In many cases, students enter the class believing that they already have the knowledge and skills that are the focus of the class; invariably, when they leave, they admit that their knowledge has increased and with it a willingness to attend arts activities. A more complete analysis is under way. December 2002 Update: In the fall 2002 semester, we fully implemented the redesigned course. All 387 students taking Understanding Visual and Performing Arts registered in one section, taught as a fully online course via WebCT. The course had a single instructor who taught the course as one course in his normal teaching load, a full-time course coordinator, and eight preceptors. The increase in content knowledge that we saw in the spring 2002 pilot of the course was repeated at a higher rate in the fall 2002 semester. The average score on the standardized exams in the traditional course was 70%; in the pilot, 78%; in the fully implemented course 85%. The reason for the continued increase is clear: in the pilot, as in the fully implemented course, students were allowed to take the practice tests leading up the exam as often as they wanted, but in the pilot they only counted as part of their participation grade. Because of this, students only needed to take the practice tests once – regardless of how they scored – to get full credit. In the full implementation, the practice tests received their own grade, and the grade that was recorded was the highest score on the test. We found that students took the tests as many as 25 times. Further increases are documented in the grade distribution. The percentage of A-Bs on the standardized exams went from 31% in the traditional course to 54% in the pilot to 75% in the fully implemented. The percentage of D-Fs went from 45% in the traditional to 21% in the pilot to 11% in the fully implemented. The increase in skill development assessed through comparison of scores on the short essays was also repeated in the fully implemented course. Essays were scored on a four-point scale (4=A, 1=F).
We are currently working to analyze the pre- and posts-tests as well as the student evaluations of the course in order to assess the change in attitudes. A cursory analysis indicates a clear increase in students' positive attitudes toward the visual and performing arts in the redesigned course. Two other issues in relation to student learning arose this semester. First, students who are not self-motivated and disciplined did not do well in the course, very often falling behind in assignments and then either withdrawing or failing. Second, students assumed that the technology would always work efficiently and effectively, often waiting until the last minute to take their tests or exams and then finding the server overloaded. In the spring 2003 semester, we will be making three slight changes to see if these two issues will be addressed. First, we are training all of the staff in the Writing Center to help students complete all assignments in the course, providing an alternative classroom for those students who need regular contact (several of the preceptors for the course also work in the Writing Center). Second, we are including a new learning outcome – technological literacy – in the course so that the students will be more self-conscious users of technology. Third, we will be offering voluntary first day class meetings so that the students can meet the instructor and so that issues such as motivation and wise use of technology can be addressed.
|
 |
Program in Course Redesign Quick Links: Program In Course Redesign Main Page... Lessons Learned: Savings: Project Descriptions: |
 |
||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|