Improving the Quality of Student Learning

Northern Arizona University

Based on available data about learning outcomes from the course pilot, what were the impacts of re-design on learning and student development?

In the spring 2002 semester, Northern Arizona University offered six sections of its redesigned college algebra course, using the ALEKS software package, and six sections in the traditional, lecture-based format. The learning outcomes for this pilot semester were very discouraging. First-day enrollment was 208 students in the redesigned sections and 215 in the traditional sections. End-of-semester enrollment was 125 students in the redesigned sections and 191 in the traditional sections. Thirteen percent of the students in the redesigned sections dropped before the 21st day (0% in the traditional sections) and 31% of the remaining students withdrew after the drop deadline (11% in the traditional sections). This amounted to combined drop and withdrawal rates of 40% in the redesigned sections and 11% in the traditional sections.

Of the 125 students remaining in the redesigned sections, only 70 took the final exam. Fifty percent of the students taking the final exam passed the course with a grade of C or better. Nearly 100% of the students who spent the required time on-line and who regularly attended class passed the course. Conversely, nearly 100% of the students who did not spend the required time on-line and who did not regularly attend the class did not pass.

We believe that the spring 2002 semester was not a fair test of the redesigned course and no relevant long-term interpretation should be given to that semester's results. The results were most likely affected by technical and logistical difficulties that occurred at the beginning of the semester. One major problem was that there were an insufficient number of textbooks available. Since access to the ALEKS Web site was bundled with the textbook, many students could not start the course until several days after the semester commenced. This situation obviously put these students at a disadvantage, and some never recovered.

The other significant problem we encountered was the method by which the ALEKS program organized the course. Specifically, (1) there were prerequisite constraints in ALEKS that prevented students from accessing material covered on scheduled course examinations. Access to subsequent topics was either prevented entirely or delayed until such a time that students show mastery of more basic material. Some students did not get to actual course examination items until very late, if at all. The end result was that students did not have the opportunity to learn needed material in a timely fashion. (2) Frequent, mandated re-assessments of students' progress actually inhibited progress and in some cases caused students to lose ground in ALEKS so that they were prevented from accessing more advanced topics in the course. (3) Built-in "four correct in a row before moving on" requirements were onerous and not generally an effective use of student's time. (4) The ALEKS built-in prerequisite system required that students master many topics that were not on the syllabus and prevented them from accessing others topics that were on our syllabus. The instructor did not have proper control over topics students may work on, the scheduling of assessments, etc. The ALEKS staff was very helpful and responsive to our needs, and made many changes in the program’s structure to accommodate us. However, many of the changes came too late in the semester for most students to salvage their grade.

December 2002 Update: Due to difficulties encountered in the spring 2002 semester, Northern Arizona University only offered two sections of its redesigned college algebra course in the fall 2002 semester along with ten college algebra sections in the traditional lecture-based format. Students in the redesigned sections used an Addison Wesley computer product, MyMathLab, as the primary source of content delivery. Students in both the traditional and redesigned sections used the same textbook, completed a common pretest, and took a common final exam.

The first-day enrollments were 69 students in the redesigned sections and 367 students in the traditional sections. The enrollments after the drop deadline were 67 students in the redesigned sections and 356 in the traditional sections, an approximate 3% decline in both types of courses. The course grades were distributed across the two types of sections as follows:

Thus, the DFW rates were 47% for the redesigned sections and 49% for the traditional sections. There appeared to be no significant differences between students in the traditional and redesigned sections when their performance on the final exam was studied.

Back

Program in Course Redesign Quick Links:

Program In Course Redesign Main Page...

Lessons Learned:
Round 1...
Round II...
Round III...

Savings:
Round I...
Round II...
Round III...

Project Descriptions:
Sorted by Discipline...
Sorted by Model...
Sorted by Success...
Sorted by Grant Rounds...