 |
 |
|||||||||||
 |
||||||||||||
 |
The Roadmap to Redesign (R2R)
The University of North Carolina at Chapel Hill Course Title: Precalculus Mathematics Project Abstract
The University of North Carolina at Chapel Hill (UNC-CH) will redesign its Precalculus Mathematics course, which enrolls about 600-700 students annually in approximately twenty sections. The course is currently taught in the traditional lecture/office hour format with optional use of a Mathematics Department Help Center. Students who do not qualify to place into Calculus according to their SAT-IIC scores are placed into the course. The course redesign will implement the following academic goals:
The Math Department plans to reconfigure its multiple sections using the Emporium Model. In any given semester, there will be a single large section of Precalculus Math. Students will learn from a textbook, an online software package ( iLrn ) and instructional personnel. The instructional software will generate problems, weekly low-stakes quizzes and other examinations. A large emporium-style learning center with extended hours of operation and staffing will give students ample and flexible time to schedule periods of contact instruction, thereby providing increased access to instructional resources when students need it as well as more effective monitoring of their progress. Providing ample opportunities to "test out" of modules in a course will help students accelerate their progress. The redesign will greatly improve the delivery of instruction by offering more active learning opportunities, immediate feedback and personalized instruction. Three objectively scored common exams developed by the math faculty and validated by the assessment team will be administered to compare the progress of students in the traditional and redesigned sections. Identical pre-tests and post-tests, objectively scored, will be administered to students in the traditional and redesigned sections during the pilot and again during the full implementation phase. Using retrospective surveys and focus group sessions, the team will also gather data on learning process components like time to learn, the application of concepts to new situations, and efforts to make course content more meaningful and enjoyable for students Fewer personnel will be required in the redesigned course. Replacing traditional-size sections with a single large section will spread individual instructor and teaching assistant contact across a whole course rather than just a single class of 30-35 students. The redesign will, therefore, reduce the overall cost of the course from $95,310 to $74,990, a reduction in the cost-per-student from $162 to $127. By improving learning outcomes and increasing student satisfaction, additional savings are anticipated as student failure and recidivism are reduced. Savings will be used to reduce the workload of graduate teaching assistants who are overloaded and to redirect some of the graduate students currently serving as instructors in traditionally taught sections of these courses to the Math Help Center, thus making expanded tutoring and assessment resources available for other mathematics courses. Interim Progress Report (as of 4/15/06) UNC-CH piloted the redesigned Precalculus Mathematics course during fall 2005. The redesign model, used for three sections, featured the use of online software ( iLrn ), a staffed math help facility and a weekly fifty-minute recitation session. A hybrid redesign model, used for four sections, emphasized sample problem-solving over lecture during three weekly class meetings. Eight traditional lecture-based sections served as the control. Total enrollment for all sections was nearly 300 students. During the pilot, students expressed enthusiasm for the course structure and materials and the ability to see their test scores immediately. The course was structured in terms of mandatory time on task. In addition, the course was offered in space on campus where students typically attend classes. Initial comparison of the pre-test/post-test data indicates improved learning outcomes in both the fully redesigned and hybrid redesigned sections of math. The findings are consistent with the results of a common set of mid-term questions administered earlier in the semester. The redesigned Precalculus Mathematics course began its first semester of full implementation in spring 2006. Modifications included larger recitation sections and closer integration of the online math system and sample problem sets used during the recitations. In the redesign, did students learn more, less or the same compared to the traditional format? Improved Learning The fall 2005 pilot for the precalculus mathematics course compared student performance across three different formats: the traditional course format, a hybrid format and the redesigned format. The primary variable among the three different formats was the amount of face-to-face, instructor-led time provided each week. Students improved test performance with all three teaching modes. The gains in test performance were slightly greater with the redesigned and hybrid sections (average gain of 50.4% ± 1.4 vs. 47.6% ± 2.0, p=0.28). However, none of the differences was statistically significant. The learning outcomes for the spring 2006 full implementation of the redesign continued to show higher gains than the traditional sections, but were slightly lower than the redesigned sections for the pilot. Again, none of these differences was significant. The results track consistently with historical data for the course that show slightly lower performance during the spring semesters. Improved Retention There were no significant differences among the DFW rates for the traditional, hybrid or redesigned sections. Were costs reduced as planned? UNC-CH followed its planned redesign with some modifications. As planned, UNC-CH reduced the number of teaching assistants from 19 to 14. However, they decided to retain two faculty members, which increased the cost-per-student, slightly off-setting the benefits of reducing the number of TAs. In addition, the plan included significant hours (378) for TAs to prepare materials and presentations. In the final implementation, these hours were eliminated since TAs did not need to spend time doing these preparations. The anticipated training time for TAs was also greatly reduced from a planned 910 hours to an actual 84. Finally, although the redesign was planned to accommodate 700 students, only 570 enrolled during full implementation. With these modifications, the cost-per-student declined from $153 in the traditional course to $124 in the redesigned course, a decrease of 19% rather than the planned 22% reduction. Pedagogical Improvement Techniques What techniques contributed most to improving the quality of student learning? One-on-one assistance. In redesigning this course, the math department at UNC-Chapel Hill adopted the Emporium Model pioneered at Virginia Tech. A single facility called the Mathport was staffed by instructors and teaching assistants and open to students 4-8 hours every weekday (36 hours per week) during the semester. In contrast to the very limited number of regular office hours available in traditional sections of the course, students had more flexibility about when they could meet with instructors for one-to-one assistance. Students who used the Mathport were generally pleased with the quality of support they received there, and only a very small percentage of students in the fully implemented redesign complained about access to individualized help. Immediate feedback on quizzes and exams. The course used an online program (ThomsonNOW/iLrn) to facilitate the management of test questions and the creation of student assessments. Multiple-choice questions used for quizzes and tests throughout the semester were graded instantaneously by ThomsonNOW/iLrn, providing students with immediate feedback on their understanding of key course concepts. Students took 22 quizzes and four tests during the semester via the ThomsonNOW/iLrn software. Only the final exam and portions of the four tests were hand-graded. Based on feedback gathered during post-semester focus groups, the immediate feedback on test scores was perhaps most universally appreciated by students. Instructors also appreciated the automated grading capabilities of the software. In fact, they thought it was the most beneficial feature of the ThomsonNOW/iLrn software. They did not view ThomsonNOW/iLrn as a particularly effective tool for introducing students to key course concepts. Consistency across sections. Standardization of student assessments and syllabi across sections allowed the math department to run all student questions about grading through a single course coordinator, who then worked with individual instructors as appropriate. This helped to ensure that students received equitable consideration on grading, regardless of their section or instructor. Accommodation of various student learning preferences. The redesigned format provided students with multiple options for approaching the course material. For example, a number of students indicated that they appreciated the ability to work ahead at their own pace. The flexibility gave them more time to identify and rectify problem areas in their understanding of course concepts. Students who needed one-to-one assistance availed themselves of the graduate assistants and instructors staffing the Mathport. Rich assortment of learning materials. Students had access to a variety of learning materials to support their independent study time. The textbook for the course and the ThomsonNOW/iLrn software were closely integrated. Instructors’ notes were also incorporated into ThomsonNOW/iLrn. The textbook came with a series of tutorials on CD, which provided step-by-step problem solving. VHS video tapes of the textbook author working through representative problems were available for viewing in the Mathport. Finally, Solution Manuals providing more in-depth explanations for the assigned textbook problem solutions could also be checked out at the Mathport. Some students said they used the full range of educational resources at some point in the semester, although most seemed to have clear preferences for their primary and secondary options. Cost Savings Techniques What techniques contributed most to reducing costs? Consolidation of sections. Replacing traditional-size sections with a single large section will spread individual instructor and teaching assistant contact across a whole course rather than just a single class of 30-35 students. As a result, fewer personnel were required in the redesigned course. Standardization and re-use of course syllabi and learning materials. Roughly 95% of the course materials created during the first two semesters of the redesign will be reused in subsequent semesters. Those materials include syllabi and problem sets for quizzes, tests, focus groups sessions and additional help sessions. Consolidation of student/instructor face-to-face interaction into one location. Using a single independent facility to accommodate student/instructor interaction in the course freed up roughly 900 annual classroom hours for the department. The post-design decision to implement the weekly focus group sessions did require use of some of those classroom cycles. However, the increase in enrollment for each of the remaining sections under the redesign allowed the department to conduct the focus group sessions in a larger lab setting. Implementation Issues What implementation issues were most important? Proper balance between course structure and self-paced learning. The ability to work ahead independently, one of the primary benefits of the redesign for some students, may have been a learning impediment for others. Some students said they had trouble learning key course concepts and managing time on task effectively outside the familiar classroom format. In part to help address this issue, the department experimented with the number of face-to-face, instructor-led sessions offered each week. The traditional model used three 50-minute lecture sessions per week. The redesign offered one focus group session, which was required only for students with a course average of less than an A or B. A hybrid model offered two focus group sessions per week. Students enrolled in the hybrid course performed slightly better and expressed a higher level of satisfaction with the course than students in the other two formats, though none of the differences was statistically significant. The full implementation of the redesign used only one weekly focus group session. Orientation of students to course format. Most students enrolled in the course had little or no experience taking courses in a non-traditional format. The assessment team and the department agree that some of the problems experienced by students taking the redesigned course might be mitigated with a stronger introduction to course resources and a clear set of guidelines for success in the redesigned course format. The orientation can be provided in part during the first day of class and reinforced through subsequent reminders during the semester. Mathport location. The data gathered by the assessment team makes it difficult to draw any conclusions about the correlation between the Mathport location and student use of the facility. The Mathport is located in a residence hall on south campus not near to the math department and other buildings where most regular classes take place. Based on the qualitative data gathered during focus groups, location was an impediment for some students (and instructors), although not for a majority. In fact, a number of students who said they did not make use of the Mathport lived in close proximity to it. Still, the math department has decided to open a second facility on central campus beginning this fall. Will the redesign be sustained now that the R2R program has concluded? The math department is satisfied with the overall results of the initial implementation and appears likely to continue it, with some additional modifications to be considered. Student learning did not improve dramatically, but the department can now ensure greater consistency across sections. Projected cost savings were greater than anticipated, and increasing the availability of classroom space has been a benefit at a time where campus construction has limited options for many departments. Several issues concerning this course are likely to be considered by the department in the coming months. Was the course too dependent on the faculty members who pioneered the redesign? How sustainable would the redesign be if they were no longer involved? Given the results of the pilot, there is some support for moving the course toward the hybrid model, but the classroom space necessary to support extra weekly focus group sessions must be identified. Questions have also been raised about whether offering the focus group sessions merely prolong student preference for the traditional classroom model. Finally, there are some lingering concerns about the impact of the course redesign on the teaching credit opportunities required for doctoral students. The university’s participation in the R2R program has helped underscore several readiness factors that may need to be more carefully considered when similar redesign efforts are undertaken on campus in the future. For example, changes in leadership within the math department since the original proposal was submitted have left many unsure about the extent of departmental support for the project. Initial buy-in for the effort may not have been as collective as desired. Will you apply the redesign methodology to other courses and programs on campus? Some aspects of NCAT’s methodology have already been applied to other redesign projects on campus. For example, the romance languages department has adopted key components of the assessment methodology for an introductory Spanish course redesign pilot.
|
 |
Quick Links: |
 |
||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|