Northern Arizona University
Course Title: Introduction to Psychology
Contact: Michelle Miller
Project Abstract
Final Report (as of 6/1/09)
- Impact on Students
- Impact on Cost Savings
- Lessons Learned
- Pedagogical Improvement Techniques
- Cost Savings Techniques
- Implementation Issues
- Sustainability
Northern Arizona University plans to redesign Introduction to Psychology, a course that serves ~1925 students annually, making it the fifth-largest course taught on campus. It fulfills campus-wide liberal studies requirements and serves a variety of majors and minors, giving it unusually far-reaching impact on the educational experience of students at NAU. The traditional course is primarily lecture-based with 175 students per section, a number that is increasing each year.
Faculty have raised a number of concerns about the quality of the educational experience in Introduction to Psychology. Most instructors perceive that the large class size makes it impractical to deviate from a lecture-based format with assessments limited to two to four multiple-choice exams. Faculty also feel forced to use a one-size-fits-all approach to teaching, despite the enormous diversity in students' backgrounds and preparedness for the course. The number of failing/withdrawing students is the second highest of all courses on campus, and 25% of students report missing more than six lectures or 2-3 weeks during the semester.
The redesign, using the Supplemental Model, is based on the NCAT's five principles of successful course redesign. An active, learner-centered approach will incorporate technology to facilitate a more individualized course experience while simultaneously reducing costs. A student response system will be incorporated into the classroom. Required web activities and practice quizzes will complement course lectures. An early intervention system will target students who are struggling as indicated by attendance, in-class responses, web activities and online practice quizzes. Finally, a team-teaching model will be implemented.
Course quality and learning outcomes will be enhanced in a number of ways. The in-class student response system will promote engagement during class and allow instructors to monitor attendance effectively. These data will, in turn, be used to intervene with students who consistently miss class. They also provide instructors with real-time feedback concerning students' understanding and identify gaps that can be addressed. The required out-of-class web activities and practice quizzes will serve to further engage students with course material, previously limited primarily to reading the textbook. Students also will receive automated feedback to gauge their own progress and achievements. The early intervention system provides individual assistance to students who are struggling. These enhancements will particularly benefit students who are underserved by the traditional format. Team teaching will enhance course quality by giving students the opportunity to learn from faculty with the greatest expertise in a given topic area.
The team plans to assess the impact of the course redesign in several ways. A pre- and post-test knowledge survey has been identified and is administered at the start and end of each class. It measures students' grasp of basic psychology principles across sections and across the semester. Course completion and the percentage of students failing or withdrawing will be compared. In-class and out-of-class assessments also will serve as an indicator of students' learning.
The operational cost of the course will be reduced by increasing enrollment by 4% from 1925 students to 2000, reducing the number of sections from 11 to six, increasing section size, and reducing the number of people teaching the course from seven to three. Implementing a team-teaching model enables a section size increase from 175 to two sections of 400 and one section of 200. Each of the 400 student sections will be taught by two full-time faculty, and one full-time faculty member will teach the 200-student section. A full-time faculty member will function as a course coordinator to facilitate student research participation and consistency across sections. The cumulative impact of these efficiencies will be to reduce the cost-per-student from $62 to $48, a 23% reduction. The cost savings will be used to expand the department of psychology's course offerings to include an honors section of Introduction to Psychology, special topics courses, and individual undergraduate research experiences.
Improved Learning
Very strong results were obtained for the pre-test/post-test comparison of psychology knowledge. In the redesigned section, performance improved from a mean of 31.2% correct on the pre-test to 40.2% on the post-test. This represents an improvement in .72 of one standard deviation, which is the second best ever obtained for a face-to-face section of this course since the department began using the knowledge assessment in 2005.
One of the traditional sections in fall 2006 used as a comparison also showed a large increase across the pre- and post-test from 32.02% to 47.76%. There are two important things to consider regarding this comparatively large improvement, however. First, this traditional section had a high drop rate (12.24%) compared to that in the redesigned section (4.06%) such that some of the improvement across pre- and post-tests could be an artifact of attrition of the weaker students. Second, the variability in performance was greater for the traditional sections (SD = 15.54, compared to 12.50 in the redesigned section). Thus, when expressed in terms of proportion of one standard deviation, the improvement in the redesigned section is closer to that in the traditional section than it would appear based on the mean scores alone.
Improved Retention
The prevalence of D and F grades was approximately midway between those for the two traditional sections used as a comparison. Similarly, drop rates in the redesigned course were neither dramatically higher nor dramatically lower than those observed in the traditional section.
However, it is important to consider these findings in light of the substantial increase in the amount and challenge level of required course work in the redesign compared to the traditional mode of delivery. Traditionally, students completed few or no assignments outside of the standard four exams, and class participation was not required or assessed. In the redesigned course, students completed four required web assignments, daily class participation questions, 14 required online quizzes and a required research survey.
In light of these increased demands, the similarity in grade distribution is acceptable. In other words, the redesigned course asks much more of students, and yet did not cause them to substantially lose ground in terms of grades. Furthermore, detailed investigation of student performance in the redesigned course revealed that every one of the failing students failed to complete one or more major course requirements (e.g., missing most practice quizzes or all web assignments, or never registering a clicker to track attendance.) Of the students who earned D’s, only two completed all course requirements. The team thinks that these results support the project’s goals. Students who make a good faith effort to complete all required work will succeed in the course.
The cost savings plan changed in two major ways.
First, the team realized that the university needed to offer fully online sections of the course throughout the year for both distance education students and on-campus students who have time conflicts with the face-to-face sections. Previously, web offerings during the academic year were not cost effective when taught by tenure-track faculty due to the desire to keep online sections smaller than face-to-face sections. In addition, the low salary paid to adjunct faculty for developing such sections raised concerns about quality, and it was difficult to recruit instructors. The team solved these problems by having a tenure-track faculty member, in collaboration with an experienced adjunct instructor, develop a standardized course shell using features that have been tested and assessed as effective in the redesign. Since the online course is now taught by an adjunct faculty member under the supervision of the psychology coordinator, the quality and consistency of the class is assured as is its cost effectiveness.
Second, due to classroom scheduling conflicts, the department was not able to offer two 400-student sections in fall 2008 as they anticipated. As a result for AY 2008-09, the department had three (400 students) team-taught sections, one (200 students) team-taught section, two (200 students) solo-taught sections and two (90 students) online sections.
The cumulative impact of these changes reduced the cost-per-student from $62 to $42, a 30% reduction. (The team anticipated that the reduced cost-per-student would be $48, a 23% reduction.) The cost savings were used to address budget cuts and to expand the department of psychology's course offerings to include an honors section of Introduction to Psychology, special topics courses and individual undergraduate research experiences.
Pedagogical Improvement Techniques
- Student response system (clickers). Clickers were used to promote participation and regular attendance. Ten percent of the course grade was based on class participation, calculated as the number of times a student clicked in, out of the total number of opportunities to do so. Instructors incorporated three to five clicker questions into each day’s PowerPoint slides. These questions were created to be similar in style and challenge level to the exam questions. Students viewed the clickers favorably, with a majority of respondents agreeing “somewhat” or “strongly” that clickers were useful, promoted understanding of course material, allowed them to connect with the instructor and allowed them to connect with course material. The team saw considerable improvement in students’ perceptions of clickers between the pilot and full implementation of the course redesign as they incorporated feedback obtained from student focus groups. These focus groups revealed that clickers were most effective when they were used frequently to solicit student feedback to challenging questions in class.
- Web activities within Blackboard. Four web-based activities were spaced throughout the semester. Each activity related to one major course topic: biopsychology, cognition, motivation/emotion and psychological disorders. Each activity comprised two separate parts. During the first part, students visited a web site chosen by the instructor for its interactivity and quality of information presented. For example, the biopsychology assignment incorporated the Harvard Brain Atlas, an online database of images of healthy and diseased brains generated using MRI technology. Students explored and interacted with the web site in ways specified by the assignment (for example, comparing specific structures in the brain across the healthy and diseased individuals.) Students then returned to the Blackboard site to complete a short essay assessment designed to gauge whether they engaged with the assigned web site in the desired way.
- Required repeatable quizzes. Using materials supplied by the textbook publisher (Worth), the team generated a set of 14 timed, multiple-choice online quizzes within Blackboard. Each quiz focused on textbook material for one assigned chapter, with questions randomly generated from a large test bank. Practice quizzes, due the weekend before the material would be covered in lecture, comprised 15% of the course grade. Using a procedure modeled after other successful NCAT projects (e.g., University of Southern Maine, University of New Mexico), students were allowed to re-take the quiz as many times as they wanted prior to the deadline, with only the best score counting toward the course grade. In student opinion surveys, online quizzes were perceived very favorably, as indicated by the percentage of students who agreed “somewhat” or “strongly” that quizzes were useful (60.4%), promoted understanding of class material (61.3%), helped in exam preparation (64.8%), and encouraged textbook reading (71.8%).
- Early intervention specialist. The team assigned the most senior and skilled graduate teaching assistant ( GTA ) to serve as the early intervention specialist (EIS). The EIS monitored the performance of students throughout the semester and alerted those with low scores to resources available for extra course help. The EIS served as a personal contact for students having difficulty in the course. The EIS contacted low-scoring students by email after each exam, encouraging them to visit during office hours in order to review exam questions and to learn study skill strategies. In addition, the EIS worked with NAU study skills specialists to develop and hold workshops throughout the semester based on pilot results showing that lack of such skills was a major barrier to success in Introduction to Psychology. The workshop instructed students on the topics of test-taking, lecture styles, effective note-taking and text reading. The workshop was well-attended, with approximately 50 students participating in the first workshop and 25 students participating in the second workshop. The success of the workshops has led to their continued inclusion in the course, with workshops already booked for the current semester.
Cost Savings Techniques
- Consolidating sections. The traditional course offered up to 11 separate face-to-face sections of introductory psychology per year. The redesign offered eight sections per year, including two web-only sections.
- Increasing section size. In the traditional course, face-to-face sections ranged from 100-200 students each and online sections were limited to about 30 students. In the redesigned course, face-to-face sections range from 200-400 students each and web sections enroll 100 students.
- Reducing the number of people teaching the course. The number of instructors teaching the course was reduced from seven to four.
Implementation Issues
- Incremental, multi-method approach. Key to the success of this project was that the team implemented incremental changes rather than one high-stakes innovation. This multi-method approach protected the project from failure in case one particular change did not work. It also allowed for flexibility and the ability to fine-tune the course based on feedback from the pilot phase.
- Importance of GTA input. Several important fine-tuning suggestions came from the GTAs, including strategies for speeding feedback to students and routing email. As the “eyes and ears on the ground,” GTAs were in a unique position to spot and help solve potential problems in a large, complex course such as this one.
- Communication/presentation of class requirements to students. The team underestimated how complicated and intimidating some students found the Blackboard-based course features. Some students missed assignments, complained to instructors and generally had a great deal of difficulty managing these new online course activities. To address this issue, the team sought and received advice from the instructional design staff at NAU’s E-Learning Center on how to improve (without totally overhauling) the course web site. A system of weekly announcements summarizing assignments and important deadlines was also set up within the site. These changes reduced student confusion and dissatisfaction with the online component.
- Managing student email. Students originally used the Blackboard mail function as the primary means to ask questions and communicate with course staff. However, students routinely sent their emails simultaneously to instructors, all four GTAs and the course coordinator with a duplicate email to instructors’ non-Blackboard email accounts. This proliferation of non-directed mail quickly became impossible to manage in a timely fashion. Project staff therefore set up a single email address outside Blackboard where students send all course correspondence. A GTA read each email and routed it to the appropriate staffer for reply, resulting in a dramatic reduction in duplicate emails as well as a dramatic improvement in response time to student email.
- Clickers. Clickers continue to present technical difficulties, particularly at the beginning of the semester when clicker registration is an issue.
- Early intervention. The Early Intervention Specialist (EIS) role evolved somewhat from the original conception. The team found that the EIS got a great deal of student interest in office hours and the study skills workshop without having to pursue students with low grades or poor attendance. Much of the EIS’s efforts are now directed toward reviewing exams and assignments with individual students in keeping with the goal of promoting individual contact in a large-class setting. The EIS succeeded in recruiting over 100 students, mostly first-year students, to attend workshops led by an NAU study skills specialist. In this way, the EIS is promoting student success and engagement, not just in the Introduction to Psychology course but in students’ coursework at large.
- Facilities. NAU has only one large auditorium that can accommodate 400-students. Competition for this space is considerable and access is not guaranteed. With so much at stake, department and project staff now reserve space for this course at least nine months in advance.
- Team teaching. Team teaching has proven to be a major incentive for full-time faculty to teach the course, contributing to project sustainability. Faculty have heartily endorsed the idea that team-teaching reduces the time needed to successfully teach the course.
- Wider effects of the redesign. Going forward, full-time faculty will do 90% of the teaching of the course without its dominating resource use in the department. The redesign also reduced the risk of inconsistent student experiences across different sections. By participating in an enhanced and expanded set of course requirements, students learn that in psychology courses, active, demanding learning exercises will be the norm.
The team is confident that the redesign will be sustained over time. In fact, the department will have full-time faculty teaching all face-to-face sections of the course while maintaining a $42 cost-per-student.Serving 2,000 students in AY 2009-10, the department will have four (400 students) full-time faculty team-taught sections, one (200 students) full-time faculty solo-taught section, and two (100 students) online sections taught by an adjunct faculty member. A full-time faculty member will facilitate student research participation and consistency across sections. One of the co-instructors will facilitate smooth communication among GTAs, other instructors and the students. The transition of a new faculty member into the teaching team highlights the flexibility of the redesign.
Additionally, the impressive learning outcomes and cost savings of the redesign has resulted in the placement of one of the team members at the university level to serve as a campus redesign scholar. Her role will be to continue the university's efforts to improve student learning and foster a campus community of individuals interested in active-learning approaches, especially in large classrooms. Thus, the impact of this redesign project extends to the wider university beyond the psychology department.
Quick Links:
