Program in Course Redesign

Carnegie Mellon University

The Traditional Course

Introduction to Statistical Reasoning, taught every semester, is a required core course for all students in the College of Humanities and Social Sciences as well as for many other majors. Two large sections are offered each year serving approximately 400 – 500 students (35% of the freshman class) annually. In 1991, the course was revised to include computer labs for data analysis using a statistical package. This initial redesign included heavy reliance on graduate teaching assistants (GTAs) and some undergraduate assistants. Although the 1991 revision reduced the cost-per-student by 14%, finding and training effective GTAs has been an ongoing, significant challenge.

The learning goals of the course require students to:

  • apply techniques of exploratory data analysis for data reduction and summary;
  • understand the concept of sampling variability;
  • understand and critically evaluate the effectiveness of different approaches for producing data; and
  • understand the use and interpretation of data analysis techniques without learning all the probabilistic or mathematical underpinnings.

The overall goal is for students to develop interest in and skills for solving statistical problems before learning the quantitative aspects of statistical reasoning.

To achieve these goals, faculty introduce new ideas, concepts, and tools in very large lecture classes. Students then work in pairs in the lab, applying statistical concepts to hands-on data analysis and problem-solving. Students use a commercially-available statistics package to work through exercises, guided by detailed instructions for the analysis and interpretation of results. The traditional design requires a heavy investment in teaching assistants (TAs): three TAs (two graduate - GTAs; one undergraduate - UGTA) for each lab, for a total of 15 TAs per semester. GTAs check students’ progress and answer questions during labs, grade exercises, and hold office hours. UGTAs assist GTAs in labs.

The most significant academic problems in the traditional course center on student learning and on the ability of TAs to facilitate that learning. While students are learning many skills (e.g., interpretation of statistical displays, evaluation of statistical evidence), there are still some critical content areas that remain difficult (e.g., planning an appropriate analysis, evaluating the validity of statistical inferences). Students need to be able to more effectively transfer the skills they are learning to downstream classes and real-world situations. The traditional lab exercises provide too much explicit direction in an attempt to reduce student time on unproductive paths, and thus are more prescriptive and less exploratory than they should be for optimum learning.

The institution has had difficulty in finding TAs to staff labs, and TAs have varied teaching skills and levels of experience. TAs make the rounds to answer questions and check progress, which means that students often have to wait to have their questions answered. TAs spend a majority of their time dealing with detailed questions about computational techniques instead of deeper conceptual issues. There is insufficient tutoring time during labs and insufficient access to tutoring during homework hours to help students stay on track, making the tutor experience labor-intensive. Added to this problem is the institution’s real need to contain staffing costs to avoid further tuition increases. The goal of the redesign is to further use technology to better serve the same learning goals.

The Redesigned Course

The new redesign will build on the earlier model and use technology to increase student learning and decrease costs by instituting a capital-for-labor substitution. Introduction to Statistical Reasoning will be redesigned using an intelligent tutoring system called StatTutor for the lab exercises that builds on the foundation of computer-based exercises already in the course. StatTutor uses a "scaffolding" approach that provides immediate feedback to students as they navigate the learning environment step by step, keeping students on course to avoid wasting time on bad problem-solving strategies, while allowing for exploratory, active learning. StatTutor is expected to: 1) improve the amount and immediacy of feedback on student exercises, 2) reduce the number of GTAs required to supervise the computer labs, and 3) improve the quality and uniformity of feedback and supervision for student practice.

The goals for the redesigned course include the following:

  • Give students more opportunities to practice solving data-analysis problems, plan the appropriate analyses, and make critical problem-solving choices on their own (with support as needed)
  • Provide exercises that require the application of common data analysis techniques to real-world problems from different domains, encouraging students to learn statistical reasoning skills in a general, transferable way
  • Provide students with tailored, individualized feedback as they solve both lab and homework problems
  • Monitor and assess student progress at a micro-level by logging and archiving their steps in problem-solving

In order to achieve these goals, lab exercises will be significantly redesigned from their prescriptive form in the traditional model to more open-ended instructions, made possible through StatTutor. Lab exercises will be transferred to an online format using StatTutor to offer a dynamic model of problem-solving. StatTutor will ask students to choose relevant variables, categorize variables, and choose statistical package tools. As students work through the lab exercises, StatTutor will provide immediate feedback on their progress using hints and, if a student fails to master particular concepts in the lab, will direct the student to additional exercises and review material, tailoring the instruction to individual student needs. StatTutor will also generate and evaluate homework assignments so that homework and labs reinforce the problem-solving techniques of the course. In effect, each student will have an individual tutor.

By adding StatTutor, the Statistics Department will reduce the number of GTAs from two to one in each lab as StatTutor takes on the roles of supervision, tutoring, and grading. The introduction of StatTutor will mean that five to six fewer GTAs will be needed per year, reducing the cost of instruction in keeping with the university’s attempt to contain tuition costs.

Traditional Course Structure

  • 15-week term
  • 1 lecture section of 210 students each per term
  • 5 labs of 40 students each per term
  • 3 contact hours per week: 2 (1-hour) lectures and 1 (1-hour) lab
  • One faculty member teaches each section. He or she prepares and delivers lectures, supervises TAs, and designs learning experiences and tests.
  • Ten GTAs monitor labs and proctor and evaluate tests.
  • Five UGTAs assist GTAs in the labs.

Redesigned Course Structure

  • 15-week term
  • 1 lecture section of 210 students each per term
  • 5 labs of 40 students each per term
  • 3 contact hours per week: 2 (1-hour) lectures and 1 (1-hour) lab
  • One faculty member teaches each section. He or she prepares and delivers lectures, supervises TAs, and designs learning experiences and tests.
  • Five GTAs monitor labs, and proctor and evaluate tests.
  • Five UGTAs assist GTAs in the labs.

Summary

In summary, the redesigned course will implement the following changes:

  • Use an intelligent tutoring system called StatTutor to introduce, supervise, and provide individualized feedback on student lab and homework exercises
  • Rely on StatTutor to provide immediate feedback to students as they progress
  • Use StatTutor to provide a consistency and abundance of feedback;
  • Eliminate five to six GTAs
  • Reduce the labor-intensive nature of the remaining TA support

Back

 

Program in Course Redesign Quick Links:

Program In Course Redesign Main Page...

Lessons Learned:
Round 1...
Round II...
Round III...

Savings:
Round I...
Round II...
Round III...

Project Descriptions:
Sorted by Discipline...
Sorted by Model...
Sorted by Success...
Sorted by Grant Rounds...