 |
 |
|||||||||||
 |
||||||||||||
 |
Implementation Issues Florida Gulf Coast University Looking back on the course pilot itself, what worked best?
Students did extremely well on the short essays for the exams, scoring on average one full letter grade higher than in the face-to-face classes taught by adjuncts. The reason for this is clear—the students were able to see and analyze a model essay before they wrote their own. The problems that we saw in the face-to-face classes were gone. Students knew the task, stayed on the task, and completed the task. We will need to develop sample essays for the Critical Analysis Essay Assignment. We ran the first set of essays that we received through the IEA, and scored them ourselves as well. The correlation between the IEA grading and human grading was 72%; we can get that correlation even higher if we feed more than 100 exams into the software to program it. We will be exploring options for this over the summer.
The students who took the practice exams repeatedly did the best on the module exams; we must build on this strength. We will weight each practice exam and keep them open for a longer period of time. The students' score will be the highest score that they get on each practice exam, giving them incentive to take the exam several times in preparation for the Module Exam. In the focus groups at the end of the semester, students explained that they couldn't use the text during the exam because they are timed, which is what we wanted. There was a direct correlation between how many times a student took the practice exams and how well they did on the Module Exam; we will build on this in the future.
Students were surprised, at the end of the semester, at how much they had learned. Based on the pre- and post-tests, the journals, and the focus groups, we were able to gather assessment data that shows that students learned more than they expected. Many came in with what they considered to be a good background in the arts, thinking that they weren't going to learn much. Even those who said they had a strong background came away very pleased with the course, claiming that they had learned a great deal and that their engagement with the arts was enriched. The course has two primary components—learning the content information that provides a foundation for each of the different performing and visual arts and developing the skills needed to thoughtfully analyze art works. These two primary course goals are met extremely well in the course. The content information is learned through reading and studying the text and taking the Practice Exams. This information is then applied in the short essays and the Critical Analysis Essays so it isn't just rote learning. The skills are learned through studying the sample Short Essays, critiquing them in peer learning teams, and then writing their own essays. Students score on average one full letter grade higher on the Exams in the redesigned course over the face-to-face sections taught by adjuncts, and a full letter grade higher on the short essays. We will build on our strengths. December 2002 Update: The three most prominent and important strengths for the full implementation include the alternative staffing model; the use of technology in appropriate ways to reduce cost and increase the quality of the learning experience; and the focus in the course on learning outcomes (specific activities geared towards developing the content knowledge and the skills that are at the heart of the course). What worked least well?
We promised to offer the students a wide variety of learning experiences to meet their different learning styles—textual based material, on-line material, practice exams, lectures, labs, etc. We planned on linking each of these experiences to the students' different learning styles. We discovered two things: the students did not attend any of the alternative learning experiences, sticking with the text and on line materials in WebCT. And, they did very well—better than the students in the face-to-face courses who attended lectures on the material. As a result, we plan on eliminating the buffet model and building on the strengths of the on-line materials.
The Practice Exams counted only as part of the students Class Participation Grade, which means that some of them took the exams only to get them out of the way. We discovered that those students who took the exams repeatedly scored the highest on the Module Exams. In the future, the practice exams will count as an individual part of the student's grade; the score they get for each practice exam will be the highest score that they get on the exam as they will be able to take the exams repeatedly. We also need to keep all practice exams for each module open the entire length of the module.
Currently, the IEA software that we use to grade the short essays is not integrated with WebCT through which we run the course. We must download the student essays, format them, and send them to the IEA for scoring. We are working with WebCT to see if they will allow Knowledge Analysis Technologies (owners of IEA) to go in to the WebCT program to create a way to integrate the two; if they agree, this will be a big benefit to WebCT, but we will have to buy a new server for this. December 2002 Update: The fall 2002 full implementation has had few areas of serious weakness, though we have identified three key issues that need improvement:
However, because technological literacy is a key learning outcome at FGCU, we will not be steering away from this delivery mode. Instead, we will work to make the students more self-conscious about their learning in relation to the use of technology, adding technological literacy as a key learning outcome to the course. Students need to be flexible in their abilities to learn, and this course can be one where they will gain a new learning strategy. What are the biggest challenges you face in moving from the course pilot to the project's next phase? A discussion of the changing goals and the biggest challenges occurs throughout this narrative – the elimination of the buffet model and the emphasis on learning styles; the refocus on the strengths of the course (practice exams, short essays, etc.); the programming of the IEA; and the integration of Banner and WebCT and of WebCT and IEA. Fortunately, all of these issues have resolutions, though some of them will take time to unfold. For instance, the cost of WebCT is on the rise and we are investigating the possibility of other course platforms. However, with the resolution of these issues, we will have developed a course design with several innovative features that reduce labor and cost and increase the quality of student learning -- features that will be easily transferable to other courses and other institutions.
|
 |
Program in Course Redesign Quick Links: Program In Course Redesign Main Page... Lessons Learned: Savings: Project Descriptions: |
 |
||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|