Improving the Quality of Student Learning

Penn State University

Based on available data about learning outcomes from the course pilot, what were the impacts of re-design on learning and student development?

• The pilot group outperformed the traditional group on overall posttest performance (66% correct in the pilot class, 60% correct in the traditional class). This difference was found to be statistically significant.
• Our primary goal was to prepare students for analyzing data on their own through repetition of activities designed to help them identify the appropriate statistical technique to use in data analysis. We think this is our number one success!
• Based on our interpretation of test scores on Readiness Assessment Tests (RATs), it appears that the design works better for the less mathematical topics.
• In our initial implementation of the new course, we have not made as much progress as we wish to make on our goal for students to become more "statistically literate" for their future roles as citizens in our society.
• We did not anticipate that students would be able to learn the concepts nearly as well as they demonstrated. Their performance matched or exceeded what we have observed on in-class exams based on lectures. The idea that the only way students learn is through lectures has been thoroughly demolished!

In order to assess student learning in Elementary Statistics, a content knowledge test was administered at the beginning and end of spring 2000. Students in three sections of Elementary Statistics took the pretest and posttest: two sections were taught in the traditional three lecture, two recitation format (n=340) and the other section was taught in the pilot revised Elementary Statistics of one lecture and two laboratory format (n=140).

On the pretest both traditional and pilot students answered about 50% of the questions correctly. The performance of students in both classes on the pretest measure was equivalent, F (1,444) = 0.067, p > .05. From this result it follows that students in the two classes had equal levels of preexisting statistical knowledge.

The pilot group outperformed the traditional group on overall posttest performance (66% correct in the pilot class, 60% correct in the traditional class). This difference in posttest performance between the pilot and traditional group was found to be statistically significant, F (1, 478) = 13.56, p< .001.

A multivariate analysis of variance was used to analyze differences between the two sections on specific questions in the posttest measure. Five questions have been identified where the pilot section outperformed the traditional section with a difference of 5-15%, although these differences were not statistically significant. The pilot group demonstrated greater mastery and understanding of the concept of median, indicated by a large proportion of students who answered this question correctly. Students in the pilot group also demonstrated superior ability in reading statistical tables, as well as comprehension and application of concepts such as confidence intervals, margin of error, range, standard deviation and percentiles as well as problems addressing the calculation of means, medians and trimmed means. The traditional section had a higher percent correct than the pilot section on one question about reading tables, and this difference was found to be statistically significant.

Three areas where the pilot group showed a statistically significant difference in performance compared to the traditional group (between 15-20% difference) included questions assessing students' abilities to: recognize and identify population versus sample data; chose and differentiate between acceptable and unacceptable measures of variability; and interpret information from a histogram. In all three cases, there was a statistically significant difference between the two sections.

Even though the pilot class performed better then the traditional class on a number of questions, there are some areas where student performance and content mastery was lower then predicted. Apparently these areas seem to be difficult for the majority of learners, regardless of the class format. These areas include questions assessing learner's ability to read and interpret information from tables, reason and solve problems involving the normal distribution, apply appropriate tests of significance as well as reason about probabilities and proportions.

Back

Program in Course Redesign Quick Links:

Program In Course Redesign Main Page...

Lessons Learned:
Round 1...
Round II...
Round III...

Savings:
Round I...
Round II...
Round III...

Project Descriptions:
Sorted by Discipline...
Sorted by Model...
Sorted by Success...
Sorted by Grant Rounds...