 |
 |
|||||||||||
 |
||||||||||||
 |
Course Development Issues Carnegie Mellon University To what extent have you been able to use previously developed materials in your re-design instead of creating new materials? We anticipated using materials previously developed by our own team members (members of the Statistics Department and previous instructors of this course, Joel Greenhouse & Oded Meyer). These materials are the laboratory exercises and other data-analysis assignments used in previous versions of the course. Not surprisingly, they were quite suitable and easy to carryover to the StatTutor format. Changing to the StatTutor format requires putting the materials into a format that can be delivered through the StatTutor Java applet that provides the "front end" that the students see. Throughout the past year, we have been developing professional/research relationships with other Statistics Departments interested in StatTutor, and through these interactions have found access to other course materials that we believe will be equally easy to incorporate into our project. A prime example of this is finding other data-analysis exercises that provide nice coverage of the targeted skills and are of interest to students. What kinds of activities took up the most time in your course development effort? We spent more time than expected refining the StatTutor system features based on student feedback. However, making the system more useable to students seems to be well worth the effort. Other time was spent mostly as estimated: course-proofing the StatTutor software, preparing TAs for new format, developing assessments. Coordinating and analyzing the data is a task that always takes a long time, so the time spent in this activity over the summer was expected. All of these technical problems were eventually overcome with good cooperation from the campus organization that supports the labs. But they were not anticipated and therefore got the project off to a shaky start. Since there were over 250 students taking the course in the new format (about half the total course population for the fall), the large scale of this trial implementation created a certain tension because the students were frustrated by the technology. December 2001 Update: Simply getting the necessary production rule programming done has taken up a larger amount of time than expected. Have students been directly involved in the re-design process? Yes, student responses to surveys about usability of StatTutor and their preferences for problem solving have been considered throughout this project. Their insights into usability (font size, use of color to distinguish different sections) have been quite helpful in most cases. One drawback is that students seem to be heavily influenced by the amount of time (relative to their expectations) the StatTutor lab sessions take. In the past academic year, students completed lab exercises sometimes with and sometimes without the StatTutor system. By chance, the StatTutor exercises tended to be the longer exercises. Some student surveys of StatTutor were less informative in that they simply said: “took too long”. This is not only uninformative but suggests the students were not ideally motivated when they were working on lab exercises (either with or without StatTutor), a problem we must keep in mind. December 2001 Update: Student feedback has been collected using comment forms. These data say something both about usability (and is being used by our HCI expert for interface redesign) and attitudes towards the use of StatTutor. At this stage in the project, disentangling reactions to the content from reactions to the SmartLabs remains difficult. For example, the large reaction that the labs were "too long" may well have more to do with subject matter than the StatTutor version itself. We will know the answer to that question when the early, shorter single variable labs are delivered in StatTutor format this spring. Our efforts to make StatTutor a true network application capable of gathering the data necessary for analysis and iterative improvement presented the standard challenges to users of new software. There were bugs in the labs in which the full version with the new TDK connections were used. Until we get the TDK-based/networked version of StatTutor bug-free, this will be a confounding variable in student attitudes toward the intelligent tutor as a central feature of labs. What kinds of support for your project have you been receiving from your department or the institution more broadly? We are very lucky to have support from the Department of Statistics (Rob Kass, Statistics Department Head), the College of Humanities and Social Sciences (Joel Greenhouse, Associate Dean), and the University’s Office of Technology for Education (Joel Smith, Director). These key players have been immensely helpful in recruiting help (either professional programmers or graduate student) for various activities and for smoothing the entire process. Most importantly, albeit intangibly, the University has expressed sincere intellectual support of our project and its mission through the words and deeds of the above mentioned people. December 2001 Update: Ken Koedinger, one of the principle researchers involved with Cognitive Tutor development (including their commercial use in the form of Carnegie Learning's Algebra Tutors) made a special effort to assist this project during the later summer and throughout the fall of 2001. He actually is continuing to work with Ross Strader in the further development of StatTutor software. This is additional institutional support that has proven critical to the success of the project. It is indeed an example of how the institution has to have depth beyond the principle investigators and a commitment to use that depth for such projects to succeed.
|
 |
Program in Course Redesign Quick Links: Program In Course Redesign Main Page... Lessons Learned: Savings: Project Descriptions: |
 |
||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|