 |
 |
|||||||||||
 |
||||||||||||
 |
Lessons Learned University of Wisconsin–Madison Pedagogical Improvement Techniques What techniques contributed most to improving the quality of student learning? Online homework. The chemistry team has developed 417 homework question sets containing 3,556 questions. A question set consists of between 5 and 20 closely related questions that test the same concept. To generate an online homework assignment, the delivery system chooses one question from each set. Each homework question includes diagnostic feedback. Many of the questions are multiple-choice format, which permits specific feedback pointing out why each incorrect response is not appropriate. When properly delivered, online homework works extremely well. Students find that it structures their studying, and they find the feedback particularly useful. An evaluation of the homework system showed that the online homework was equal to the traditional procedure. The evaluation study of online homework indicates that students with better logical-thinking skills found the online homework less useful, but experience using the materials in General and Analytical Chemistry, a more-selective course with students whose average logical-thinking skill level is higher than that of General Chemistry students, reveals that these students still found the online homework very useful. Online tutorials. To date, the team has developed 37 instructional modules for Web-based delivery. Each tutorial module leads a student through a topic in 6 to 10 interactive pages. When the student has completed the tutorial, a debriefing section presents a series of questions that test whether the student has mastered the content of that module. The team also arranged to use a pre-publication version of Fundamentals of Chemistry, an excellent Web-based chemistry tutorial created by Stanley Smith at the University of Illinois. Both sets of tutorials help structure the discussion sections by having students come to class prepared to ask questions. Students have found these online tutorials very helpful, reporting that they are able to learn effectively from these materials. They particularly like the ability to link directly from a problem they have difficulty with to a tutorial that will help them learn the concepts needed to solve the problem. Many students reported that they found the online material much more accessible than the textbook. Pre-lab tutorials and quizzes. The pre-lab tutorials and quizzes prepare students for laboratory techniques, ensuring that students come to the labs prepared. Better preparation results in better lab reports and notebook writing. Cost Savings Techniques What techniques contributed most to reducing costs? Online homework. Homework assignments are generated automatically. Neither faculty nor TAs need to generate or grade homework assignments, and record-keeping has been automated as well. Because each homework question includes diagnostic feedback, TA time previously spent repeatedly correcting students’ errors is saved. Because the homework structures students’ preparation for discussion sections, less preparation is required of TAs. Online quizzes. During summer 2000 and the 2000–2001 academic year, the chemistry team developed 156 quiz question sets containing 780 questions. These were used in a 250-student lecture section in the pilot program during the fall and spring semesters of 2000–2001. During the summer and fall of 2001, the project developed an additional 123 sets of questions containing a total of 594 questions. The grand total of quiz questions developed is thus 1,374, with the questions grouped within 279 sets. The quizzes are administered by WebCT and require no preparation time or grading time from TAs. To adjust the quizzes appropriately, a short, weekly interaction between faculty and the staff person who prepares the quizzes is required, but this is considerably less time than what faculty normally spend preparing quizzes. Online tutorials. Because students come to class prepared to ask questions after completing the tutorials and because they help structure the discussion sections, less preparation time is required from the TAs. Tutorials also provide an effective substitute for faculty time spent preparing and delivering lectures. When the team did less lecturing and counted on the tutorials to provide a major fraction of the instruction, students were not at a disadvantage. Pre-lab tutorials and quizzes. Because students come to labs prepared, TA time spent teaching laboratory techniques is reduced. Better preparation results in better lab reports and notebook writing, saving TAs grading time. Pre-lab quizzes have also replaced quizzes that TAs formerly had to grade. Implementation Issues What implementation issues were most important? Course-management system. The chemistry team experienced major problems with upgrades to the course-management system. WebCT was upgraded twice during the project. As is common with software changes, each upgrade required the course-redesign team to recheck all of the online homework and quiz questions. In some cases, the online tutorials required revision. Because of the amount of information delivered online within WebCT and the complexity of the use (e.g., Javascript that varies with browsers and more than 1,000 questions in the question database), this proved to be a major effort. After the first upgrade in summer 2000, tutorials that had worked well in the previous version of the software simply stopped working. Online quizzing was hampered by extremely slow response time; faculty, project staff, and TAs found that they were sometimes forced to wait for minutes for a screen update. These problems required tremendous effort on the part of the chemistry team in order to keep the pilot course running. Because of the problems, it would have been disastrous to try to have all faculty members implement the redesign, and so the team continued with only a single lecture section. After another update of WebCT, many homework and quiz questions no longer generated the appropriate information for students. Problems in handling numeric values also resulted in widespread student complaints that had to be dealt with by project staff. To avoid such problems, the course-redesign the team had to retest, sometimes for the second time, all homework and quiz questions that were to be used in the new course. Consequently much of the team’s time was spent redoing tutorials, homework, and quizzes that had been developed and tested earlier. Much less time was available for evaluation than anticipated. Many of the problems the team experienced are a reflection of the state of today’s course-management system software industry. Upgrades can bring problems, and those problems are more likely to occur in situations where the software is being stretched, such as occurred in this chemistry project. Until course-management software can be improved, a project of this nature will continue to face ongoing implementation problems.
|
 |
Program in Course Redesign Quick Links: Program In Course Redesign Main Page... Lessons Learned: Savings: Project Descriptions: |
 |
||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|