SUNY at Potsdam

Course Title: History (Europe 1500-1815), History (Europe 1815-Present), History (US to 1877) and History (US 1877-Present)
Contact: Krista Medo

Project Abstract
Final Report (as of 3/15/10)

• Impact on Students
• Impact on Cost Savings
• Lessons Learned
  • Pedagogical Improvement Techniques
  • Cost Reduction Techniques
  • Implementation Issues
• Sustainability

Project Abstract

SUNY Potsdam plans to redesign four history courses. All four courses are currently offered each semester in multiple sections meeting three hours per week, capped at 40 students. In AY 2007-2008, 586 students enrolled in 15 sections of European history and 875 students enrolled in 23 sections of US history. The redesign plan will combine four courses into two. History (Europe 1500-1815) and History (Europe 1815-Present) will be combined into a single European History course. History (US to 1877) and History (US 1877-Present) will be combined into a single American History course.

These history courses face four distinct academic and resource problems. 1) The current courses serve three primary audiences (general education students, teacher education majors and history majors), each with different learning objectives. 2) First-year general education students have a significantly lower successful completion rate (65%) than advanced students (77%) or history majors (83%). The redesign will target the general education students (~1100) who comprise 70% to 85% of the enrollment in the four courses. The two new courses will better serve this audience while allowing the department to tailor other introductory courses to education and history majors. 3) The department frequently cannot meet the current demand for these courses. 4) The department wants to move toward the college’s goal of reducing faculty teaching loads from 24 to 18 hours per year.

The college’s redesign plan, using the Replacement Model, collapses the 27 sections of European and US History historically needed to serve General Education students into a single European history section and a single US History section, each serving 300 students per semester. In-class time will be reduced from three hours to one hour per week. Students will spend a minimum of two hours per week in a computerized History Learning Center dedicated to the two courses. At least one hour will be spent completing online publisher provided quizzes, map exercises, and chronology worksheets. One hour each week will be spent in online discussion groups of 20 students each, moderated by virtual preceptors. Students will be able to test their own skill at historical argument and interpretation.

The redesign will improve the quality of student learning in several ways. First, interactive learning will be substantially increased with online assignments, weekly work in the dedicated History Learning Center and increased participation in the discussion groups. Second, students will receive immediate feedback on their online work and individual assistance from the virtual preceptors and the instructor as appropriate. Staffing the computer center with undergraduate learning assistants (ULAs) for 60 hours per week will also increase on-demand availability of assistance. Third, the redesign will eliminate variation in content and assessment, bringing greater consistency to the student learning experience.

Strategies for assessing learning outcomes will be twofold. The courses are, and will continue to be, regularly assessed by the college’s general education committee using standards that have been in place since 2001 for American History and Western Civilization. In addition to the committee’s assessment, student learning will be assessed by comparing common content items selected from exams, students’ essays using common grading rubrics, and final grades using common criteria.

Cost savings will be achieved by reducing the number of sections serving general education students from 27 to 4 and increasing section size from 40 to 300. Full-time faculty will be reduced from nine to four and part-time faculty from two to none. Ten ULAs and six virtual preceptors will be added for each course. These actions will decrease the cost-per-student in US History from $168 to $128, a 24% reduction. The cost-per-student in European History will decrease from $170 to $118, a 31% reduction. The savings will be used to improve teaching quality by reducing teaching loads, funding undergraduate research, library acquisitions and student travel.

Final Report (as of 3/15/10)

Impact on Students

In the redesign, did students learn more, less or the same compared to the traditional format?

Improved Learning

Assessment data indicates considerable improvement in student learning in the redesigned courses.

Learning outcomes in the American History course were compared using common multiple choice questions and essay exams with common rubrics. In the American History course, average scores on comparable essay questions, graded by the same rubric, improved from 2.22 in the traditional course to 2.58 in the redesigned course. Improvement was greatest in learning objectives related to understanding common institutions in American society (from 2.00 to 2.64) and least in learning objectives related to understanding American history in world context (from 2.49 to 2.56). 

Measured by responses to common multiple-choice questions, similar improvements in student learning are apparent. Overall, correct responses increased from 55% to 76%. Improvements were greatest in response to questions about the world context of American history (from 43% correct to 74% correct) and least (but still significant) in response to questions about common institutions in American society (from 64% correct to 76% correct).

Learning outcomes in the European History course were compared using common multiple choice questions and essay exams with common rubrics.  Averaging factual (multiple choice) learning with interpretive and analytical (essay) learning yields a measure of overall learning that compares very favorably with the traditional course.  Scores at the top of the distribution more than doubled (from 5% to 11%), while those in the “B” range rose significantly (from 39% to 46%).  “C”-range performances diminished substantially (from 27% to 18%) while “D” scores plummeted (from 23% to 4%). 

Clearly, in the redesigned European history class marginal students cannot coast to a passing grade, for the depletion in “D” scores is made up for by the much greater number of outright failures (from 6% to 24%).  The team interprets this bi-modal distribution as an indication that students who put forth even moderate effort in the course enjoy much better learning outcomes, whereas students who do not work effectively cannot squeak by.  Taken as a whole, the redesigned European course has fulfilled its promise to improve student learning.

Improved Completion

Student success rates (grades of C or better) declined in both the American History and European History. In American History, 73% of traditional students received a grade of C or better compared with 61% of redesign students. In European History, 75% of traditional students received a grade of C or better compared with 63% of redesign students.

Retention is defined as students who earned a grade of “D” or better and who remained enrolled to the end of the semester. By this measure, both the American and European versions of the course redesign experienced decreased retention rates.  The withdrawal and failure rates for the European course increased from 14% to 24% in the baseline averages over the period 2005-2007. Corresponding withdrawal and failure rates for the American course rose from 16% to 19%. In each case, decreased retention was due to rising failure rates as well as to stable or increased rates of withdrawals. Withdrawals jumped from 5% to 10% in the European course but held steady at 5% in the American course. Failures rose from 9% to 14% in the European course but less dramatically from 11% to 14% in the American course.

The team cannot yet attribute a definite cause to this drop in retention. Comments made by students on course evaluations in both courses were too various to single out clearly discernable factors. However, the evaluations indicated that a sub-population of the students did not like the reduced professorial contact or the removal of the personal guidance that contact represented for them. While some students reveled in the flexibility and freedom these courses offered, others found it difficult to assume more independence and to take greater responsibility for completing their assignments. 

It may be that decreased retention is the paradoxical result of the course redesign’s pursuit of other goals to make the department’s instruction in history surveys more uniform and to foster greater student engagement. Since generally less demanding adjunct faculty have been eliminated from the American and European survey courses and grading has become more uniform, it may be that past grades were higher because the grading was easier. Because the redesigned courses required a greater degree of weekly engagement by every student than traditional course formats, it may be that less motivated students washed out more readily.

Other Impacts on Students

While student evaluations of the redesigned courses were so varied that they defied easy summary, it is clear that these hybrid classes enabled more students to take a history survey than would be the case were they offered in a traditional format.  Students could fulfill many course requirements at a distance, enabling those with families, jobs, or living far from campus to enroll in the redesigned classes. This had particular merit given Potsdam’s geographical situation in a large, sparsely populated, and poor region of New York State.

Impact on Cost Savings

Were costs reduced as planned?

The redesign plan projected a cost savings of between 20% and 25% for the new courses. Although the cost savings plan was followed as proposed, when measured by cost-per-student, the redesigned courses did not realize their potential to reduce instructional costs during the 2009-2010 academic year.

The fundamental reason why savings were not achieved as projected is because enrollments in the redesigned courses ran well below capacity. The team had projected enrollments of 300 for each of the two courses each semester. Actual enrollments were considerably lower as, in the entire academic year, only 384 students enrolled in the American history course and only 228 students enrolled in the European history courses. Judging from anecdotal evidence and from responses to the comprehensive course evaluations, many students exhibited considerable anxiety about enrolling in a hybrid course that was different from anything they had experienced before.  As both redesigned courses are stand alone general education courses, the first- and second-year students for whom they were designed could put off taking them for several semesters. 

Enrollment trends suggest that student hesitancy to take the redesigned courses will be quickly overcome. Between fall and spring semesters, enrollments in the European course increased by nearly 60%. In the American course, they increased by over 10%. The department expects enrollments much nearer to capacity in the 2010-11 academic year, which will yield real savings.

In addition, the history department was able to develop and offer two freshman writing sections and one freshman speaking section in support of the general education program. Both sorts of offerings allowed it to better serve both its majors and teacher education students. The department plans to offer nearly 20 sections of writing and speaking courses during the 2010-11 academic year. These courses will allow students to complete the four-credit speaking or writing requirement and the three-credit American History or Western Civilization requirement in a single four-hour course. 

The redesign has also allowed the department to make real progress on a college-wide goal of reducing faculty teaching loads from 24 to 21 hours per year. 

Lessons Learned

Pedagogical Improvement Techniques

What techniques have contributed the most to improving the student experience?

• Online discussion groups. Required weekly discussion posts demanded engagement with primary source readings that was both broader and deeper than in traditional offerings of the American and European history courses. Students were required to make a minimum of three discussion posts each week in response to questions and comments pertaining to assigned primary source readings. This meant that each and every student had to “speak up” every week and offer a set of coherent thoughts in a virtual discussion group. This represented a great improvement over a traditional classroom format in which a minority of students engaged in discussion. The moderation of discussion groups by VPs and the instructors of record enhanced the quantity and quality of instructor feedback. In addition, transforming the colloquial English of oral interventions in the classroom into standard written English improved the quality of student discussion, sharpened writing skills and increased the amount of written work students submitted over the course of a semester compared to traditional courses. Because discussion groups were focused on the analysis of primary sources and the integration of those interpretations into textbook and lecture material, students were exposed to a more sophisticated style of learning.

• Weekly online quizzes. Weekly reading quizzes held students to a much higher level of accountability in the redesigned courses than in the traditional courses. Student engagement with the historical narrative was intensified and “extensified” by taking on-line multiple-choice quizzes on each textbook chapter prior to each week’s lecture.  Because students were allowed to take each quiz up to three times (10 or 15 questions were randomly selected from a pool of some 50 questions in each iteration) and record the highest score, an incentive was created to spend more time studying, taking quizzes, studying again, and retesting knowledge. The pedagogical burden of conveying the basic historical narrative was almost entirely transferred to an automated system on Blackboard, thereby freeing instructional resources to the instruction of document analysis and historical interpretation.

• Thematic, creative lectures. Because lectures no longer had to carry the burden of communicating the historical narrative or of explicating primary source readings, lectures could address less conventional topics. Use of smart classrooms allowed for pictures, videos, and music to accompany and illustrate points being made in lecture, enriching the presentation and enhancing student interest. In the fall 2009 semester, the team introduced the use of iClickers to promote student attendance, attention and engagement in lecture, suppressing the distractions of facebook and texting sweethearts. The team is now considering the advantages of extending the present 50-minute lecture to 75 minutes in order to provide time for more creative interaction through the use of iClickers, for in-class group exercises, or to allow more latitude for instructors to respond to student questions.

• Flexibility for students. Students were required to attend one lecture per week. All other course work could be completed at times most convenient for each student. This was especially valuable for financially-pressed students who hold down jobs while attending college and for non-traditional students who may commute long distances to school and pursue their studies amidst the pressures of jobs and families. The History Learning Lab was open and staffed seven days a week for a total 60 hours during afternoons and evenings, greatly facilitating quiz-taking and other course work. Primary source readings and discussions groups could be accessed online from any location at any time, thus providing maximum flexibility to students.

Cost Reduction Techniques

What techniques contributed most to reducing costs?

• Reducing the number of sections from 27 to four. Collapsing multiple sections into one section each of American and European history each semester reduced duplication and the demand for adjunct faculty, while allowing full-time tenure/tenure track faculty to develop introductory courses designed for specific constituencies.

• Employment of low-cost virtual preceptors. Recent recipients of the BA in history at SUNY Potsdam who are now enrolled in graduate programs at Potsdam and elsewhere were employed to moderate online discussions and grade essay exams.

• Appointment of low-cost ULAs. ULAs staffed the computer laboratory for 60 hours a week to provide immediate assistance, both virtual and face-to-face, to students seeking help in the course. The ULAs also assisted in record keeping—attendance, discussion participation, and quiz taking—and initiated email reminders to be sent to students who missed a class, an assignment, or a quiz.

• Machine-graded reading quizzes. Weekly machine-graded reading quizzes were used to insure that students had the factual background, and some understanding of the basic historical narrative, in preparation for the higher order historical analysis presented in weekly lectures and primary source readings.

Implementation Issues

What implementation issues were most important?

• Existing materials. Publisher-provided resources—primary source readings and test banks—required considerable tweaking to make them appropriate for the redesigned courses.  Publisher web sites, which the team had imagined as integral to the courses, proved to be of little use and seldom visited by students. This required more course development than originally envisioned.

• Student buy-in. While a majority of students (75% or so in most categories) responded positively to course evaluations, about a quarter articulated significant complaints. These were of two sorts. Some students found the large weekly lectures, the clicker technology, the computer lab and the online discussion, dehumanizing, and complained that it ran counter to what they expected of a smallish public arts and sciences college. Others complained that the large number of low stakes assignments—weekly reading quizzes and required discussion posts—provided them with too many opportunities to fail. 

• Scheduling. Reducing the number of sections from 27 to four per year provoked unforeseen scheduling conflicts. Students previously had been able to fit their American History and Western Civilization requirements in their schedules around the requirements of their various major programs. The redesigned courses, although they only met for one required hour per week, allowed less flexibility.

• Technology. Admittedly, history is a low-tech discipline, and historians are generally low-tech faculty. But off-the-shelf software, including Blackboard, simply did not meet the needs of the course and its students as well as might reasonably be expected. The testing features, particularly for essay exams, proved so problematic that students wrote their exams in MS Word, printed a copy that served as a “receipt,” and cut and pasted the text of their Word document into Blackboard.

• The teaching team. Incorporating the history department’s outstanding undergraduates and recent graduates into the team that taught the courses was an unexpected benefit of course redesign. The virtual preceptors (VPs) and undergraduate learning assistants (ULAs) worked on the details of implementation from the beginning. They took their responsibilities for the courses with absolute seriousness and were, and remain, integral to the details of implementation.

• Student skills. The redesigned courses clearly were more effective for some students than for others. Responses from student evaluations indicated that organized, attentive, students who were willing to seek help from the ULAs, VPs, and the instructor, who consulted the Blackboard web site, and budgeted their study time responsibly, did well.  While students who fell behind in one or more of these college-level skills suffered, mature and motivated students responded very positively to the new format and valued the flexibility and variety of course work.

Sustainability

Will the redesign be sustained now that the grant period is over?

The redesigned history courses enjoy departmental as well as college-wide support and the commitment of relevant faculty to teach these courses in the foreseeable future.  Additional faculty members are preparing to assume responsibility for the redesigned courses, thus providing a deeper bench of instructors. The department has also assembled an extremely strong team of virtual preceptors (VPs) and undergraduate learning assistants (ULAs), many of whom are now quite experienced in the various facets of course delivery and whose continued service guarantees effective course management and delivery in the future.

In addition, the team’s hybrid course design fueled interest beyond the department.  SUNY Potsdam’s School of Education has solicited the history department to prepare courses redesigned on the same hybrid model, in American and European history that would be specially tailored to New York State teacher certification standards and so directly meet the needs of its students. The department will develop these courses over the summer, and deliver them in the 2010-11 academic year. With their launch the college will realize substantial economies of scale since the History Learning Lab and ULAs will already be in place and can be shared among the (now) four redesigned courses.  The team therefore faces the pleasant prospect of catering more particularly to the curricular needs of various student sub-populations while doing so at even lower costs.

There are, however, some dark clouds on the horizon that may auger less sunny weather ahead. The SUNY budget crisis has prompted Potsdam’s administration to reduce expenditures by severely cutting the temporary services budget, from which the VPs and ULAs are paid. While the team’s redesigned courses have been funded for the fall semester 2010, there is no guarantee that the VPs and ULAs that are essential to the course redesign will be funded thereafter.

The ongoing delivery of the redesigned history surveys is also complicated by SUNY’s recent decision to require that campuses require only seven of the ten categories of the common general education curriculum.  Prior to the 2000-01 revisions of general education, Potsdam had a single history requirement.  Some sentiment exists for folding the current American History and Western Civilization requirements back into a single history course.  Without the demand generated by current requirements, there would be little reason to continue to offer large-enrollment, cost-efficient courses. 

Quick Links:

State University of New York Main Page...