The Learning MarketSpace, July 2010
A quarterly electronic newsletter of the National Center for Academic Transformation highlighting ongoing examples of redesigned learning environments using technology and examining issues related to their development and implementation.
TABLE OF CONTENTS:
Offering perspectives on issues and developments at the nexus of higher education and information
The Retention Fallacy
In partnership with colleges and universities throughout the United States, NCAT has now successfully completed 116 redesign projects that improve student learning while reducing instructional costs. The actual number of courses that have been redesigned is considerably higher than 116 since many projects redesigned multiple courses, either under our auspices or subsequent to their initial redesign. While a substantial majority of these projects have improved student learning outcomes and/or student completion rates, 100% of them have reduced instructional costs.
What accounts for this remarkable success rate in reducing instructional costs? The primary reason is that we require all projects to develop a cost reduction plan that can be sustained with no diminution in academic quality before they embark on their redesign.
Unlike attempts to improve student learning, which may or may not be successful depending on both how good the redesign plan is and how well it is executed, sound cost reduction plans take the “hope” out of implementation.
We are beginning to accumulate enough experience in improving student learning to be able to say that, if you follow our redesign principles extracted from many successful projects, you will be able to improve student learning. We are testing that idea in our newest program, Changing the Equation. But from the time of our first redesign projects in 1999, we have been able to say that we know how to reduce instructional costs and we can guarantee that any redesign project can do so if it has a sound cost reduction plan.
In our original ideas about what constitutes a sound cost reduction plan, we had a few basic ideas. The first was that we needed to move away from what we then called our “credit-for-contact” mode of instruction. The second was that we needed to shift faculty time-on-task to the technology, lessening the labor-intensive quality of instruction. The third was we needed to transfer the locus of activity from the faculty to the student, putting the focus on student problem-solving and projects rather than on presentation of materials. These ideas have proven to be sound over time.
Other ideas about cost reduction have not proven to be sound. The application guidelines for the seminal Program in Course Redesign said, “There are, of course, a variety of ways to redesign courses to reduce costs. One approach is where student enrollments stay the same but the instructional resources devoted to the course (course expenditures) are reduced. Another approach is to increase enrollments with little or no change in expenditures. A third way is to reduce the number of course repetitions required to pass a particular course. In many community colleges, for example, it takes an average of 2.5 enrollments to pass introductory mathematics courses. This means that the institution and the student must spend 2.5 times what it would cost to pass the course on the first try. In each case, a translation of the savings to cost-per-student can be used for comparative purposes.”
The first two approaches have been proven over the course of a decade and more than 100 projects to be valid. Indeed, they have been codified in our Cost Reduction Strategies document included in our planning resources. That’s what accounts for the 100% success rate. But the third, a reliance on increased course completion--or “retention” as it is commonly expressed—as a cost reduction approach has proven to be completely erroneous. Indeed, we now say that retention may represent a further opportunity for cost savings after several terms of fully implementing a sound cost reduction strategy. Instead, we now require (and have done so since 2006) that plans for cost reduction include one of the two strategies described above, both of which will result in immediate savings during the first term of full implementation.
We now do not believe that increased retention is a sound cost reduction strategy. It is a “hope” rather than a plan—and a “hope” that is more often than not unrealized.
Let’s consider why we have arrived at this conclusion.
The first reason for our conclusion is that despite the generally expressed desire to improve completion rates through course redesign, in many cases, completion rates do not improve or improve only modestly (less than 10%). There are several reasons for this.
Pass rates in the course are already high.
While there are certainly many introductory courses that suffer from poor pass rates, there are also many that have high pass rates. We have worked with many projects whose pass rates are in the 90% range. It is unlikely that pass rates will increase substantially in these situations. Rather, these institutions have wanted to improve the depth of student learning and/or reduce the cost of instruction. For example:
Quality improves but the completion rate goes down.
Sometimes completion rates will go down for good reasons. We have frequently experienced the phenomenon of improved student learning outcomes supported by clear assessment data coupled with decreased completion rates. This phenomenon is typically due to prior grade inflation.
Improvements in completion rates are frequently modest (less than 10%).
While many course redesigns have produced significant gains in completion rates, especially in mathematics, many redesigns produce relatively modest gains. Here are some examples of the percentage point gains from a variety of disciplines and a variety of institutions:
Even if the course enrollment is large, often you can’t reduce sections (i.e., reduce costs) because the retention improvement is not sufficient to eliminate a section. To illustrate this point:
Each of these projects actually reduced costs by following one of the first two strategies described above.
The second reason for our conclusion is that even when completion rates improve more than 10%, you must analyze the actual circumstances of the way the course is offered (how many sections, size of sections, etc.) and take steps to reduce the number of sections offered. In other words, you need to have the ability to actually reduce sections. Sometimes the numbers just won’t work.
We have seen “course redesign” projects not conducted by NCAT where the cost reduction strategy was based on increased retention, but the whole course was not being redesigned so there were insufficient numbers affected to actually reduce costs.
A community college biology redesign project planned to reduce costs by increasing retention by 10% in two introductory courses.
There is a relationship among section size, pass rate improvements and possible cost reduction. The larger the section size, the higher the pass rate improvement must be in order to reduce costs. The smaller the section size, the lower the pass rate improvement must be in order to reduce costs. However, reductions in smaller section sizes result in lower amounts of the cost reduction.
The third reason for our conclusion is that even if the number of students enrolled in the course is large and you take the necessary steps to reduce the number of sections offered, the impact on cost reduction may be quite small.
Assessing the actual impact of improved retention on the specific course.
We first considered the idea of recognizing increased retention as a cost reduction strategy in the Program in Course Redesign. The University of Central Florida (UCF) redesigned its American Government course, enrolling 2,200 students in 25-30 sections of 80-100 students. At the time, the total cost of the traditional course was $247,170. UCF data collected in earlier redesigned courses showed a 7% increase in retention. Applying this increased retention rate to 25 redesigned sections resulted in a one-section reduction (7% of 2200 students = 154 students), amounting to a cost savings of just $8,239 (the total cost of a traditional section including personnel and classroom space rental), a 3% reduction.
Clarifying how many students are really impacted by increased retention.
The common wisdom is that if students pass at a higher rate and/or move through a program more quickly, the institution should have to offer fewer sections. While that is true, the actual numbers may be relatively small.
Take a typical community college developmental math course that enrolls 1200 students annually, 50% of whom pass the course. That would seem to suggest that in any given year, 50% of the students in the course are repeats. But we know that isn’t true since many students who fail developmental math drop out of the college and do not return. So the first question one must answer is, how many of the 50% who fail actually re-enroll?
Let’s assume that only half of the students re-enroll in the developmental math course. This means that in any given year, 900 students are new and 300 are repeats.
If you improve course completion rates by 20 percentage points, that’s 20% of 300 repeats or 60 students who would not have to be in the course at any given time. So this says that even if you improve completion rates by 20% and you teach students in sections of 20, you will only save three out of 60 sections, for a cost savings of 5%.
If you are serious about reducing costs, why would you want to produce a savings of 3% or 5% when course redesign produces an average cost reduction of about 35% and frequently allows you to double the number of students served on the same resource base?
If, despite the numbers that we have shown, you still want to consider increased retention as a cost reduction strategy, you must crunch the numbers. It is not enough to merely assert that increased retention equals reduced costs as we did in 1999. You must analyze each particular situation to determine the actual impact and whether it is even possible to reduce costs by increasing retention.
We have shown that retention is a highly questionable cost reduction strategy when redesigning a single course. Some may counter that improved retention in a single course affects the institution’s overall retention rate and that the cost of recruiting a new student is far greater than the cost of retaining an existing student. That may be true for certain kinds of institutions, but it certainly isn’t true for others. To support this argument, you must assess the relationship of an increased completion rate in a single course to overall college retention and cost it out.
What we now know for sure is that two strategies will reduce the cost of instruction: 1) keep enrollments the same and reduce course expenditures or 2) increase student enrollments with little or no change in expenditures. By eliminating retention as a cost reduction strategy, we can take the “hope” out of planning and realize the important goal of producing more learning for less cost.
--Carol A. Twigg
Featuring updates and announcements from the Center.
All three were selected as one of the “100 Most Creative People in Business 2010” by Fast Company magazine. While Lady Gaga was #1, Carol was a very respectable #47. As Fast Company puts it, “This year's 100 Most Creative People offers our own, idiosyncratic perspective on business. The selections reflect the breadth of new ideas and new pursuits at play in our business landscape. From interface designer Yugo Nakamura to HBO Documentary Films president Sheila Nevins to futurist Ray Kurzweil, we can attest that creativity is alive and well in 2010.” NCAT’s innovative projects related to increasing productivity in higher education were highlighted on Fast Company’s web site and in the May/June 2010 issue. To read more and see who else is on the list, visit http://www.fastcompany.com/100/2010/47/carol-twigg. Kudos, Carol!
A May 2010 article in National CrossTalk, a publication of the National Center for Public Policy and Higher Education, entitled “Redesigning the Basics” features the redesign of developmental education at the Tennessee Board of Regents (TBR) institutions. The article by Kay Mills highlights the highly successful outcomes achieved in partnership with NCAT. The TBR institutions are now working on systemwide policy revisions building on the experiences gained in the initial redesigns. The TBR is “expanding its redesign for developmental courses, and by 2013 all its community colleges must have in place programs that have technology as an integral part and must focus on helping students master the subjects at their own pace,” said Paula Short, vice chancellor for academic affairs. “Redesign may appear piecemeal because we’re not finished. It will be systemwide.”
The National Center for Public Policy and Higher Education is an independent, nonpartisan, non-profit organization that promotes public policies that enhance opportunities for quality education and training beyond high school. To read the full article, see http://www.highereducation.org/crosstalk/ct0510/news0510-tenn.shtm.
A new Education Sector report, The Course of Innovation: Using Technology to Transform Higher Education, focuses on NCAT’s evolution over the past ten years and highlights the ways that colleges and universities are using technology to simultaneously improve student learning and reduce skyrocketing higher education costs. Policy Analyst Ben Miller notes that NCAT "has amassed a growing and increasingly complex portfolio of transformation-related projects." Clearly, conditions are right for reform. Yet Miller finds that "colleges have yet to decide, en masse, that adopting a proven method to produce better student learning outcomes for less money is the kind of thing they should naturally do on their own." He cites a number of barriers hindering innovation and presents a series of well-considered recommendations that will encourage and support colleges and universities seeking to transform learning for students.
Education Sector is an independent think tank that challenges conventional thinking in education policy. It is a nonprofit, nonpartisan organization committed to achieving measurable impact in education, both by improving existing reform initiatives and by developing new, innovative solutions to our nation’s most pressing education problems.
In the coming months, NCAT will provide its own analysis of why we believe course redesign has not been more widely adopted despite its well documented successes.
On June 3, 2010, the American Enterprise Institute (AEI) convened a group of higher education experts to discuss the state of innovation in higher education. As described in the June 4, 2010 issue of Inside Higher Ed, various panelists provided a litany of familiar reasons why colleges fall short of graduating students in sufficient numbers and often lack a real plan for dealing with diminishing resources. Participants collectively lamented the barriers to real innovation at colleges and universities, while acknowledging that precious few agreed-upon strategies for transformational change have gained any real foothold. Some panelists argued that there’s simply a dearth of shared ideas that a critical mass of institutions can rally around. The article continues, “For those hoping to control costs and improve learning outcomes, a persistent fan base exists for the National Center for Academic Transformation, or NCAT.” Suzanne Walsh, now a program officer at the Gates Foundation, lamented that NCAT remains among the few go-to examples in any conversation about changing how colleges deliver course content. “While NCAT is a fabulous, fabulous example, is it the only innovation we have in higher ed? Where else can we find pockets of innovation that can help us?” said Walsh. Indeed, the numerous presentations in a full day of sessions left the impression that problems in higher education remain far more readily identified than the solutions. To read the entire article, see http://www.insidehighered.com/news/2010/06/04/aei.
As noted in the October 2009 issue of The Learning MarketSpace, we intend to provide a series of updates on redesign projects that have been underway for an extended period of time and have essentially become the “traditional” way that the course is offered on campus. NCAT’s recently concluded Colleagues Committed to Redesign (C2R) program required institutions to complete a pilot of their redesign, yet the majority of the participants have gone on to fully implement their redesign plans. Here are a few examples:
The team at Harry S. Truman College in Chicago, IL continues to report success in their redesign of College Algebra. From fall 2005 through spring 2007, the average success rate (grade of C or better) in the traditional format of the course was 47%. From fall 2005 through spring 2009, the success rate in the redesigned format was 65%. The retention statistics are equally compelling. The traditional retention rate was 63% compared with 76% for the redesign. Course enrollment in the traditional format was 361 students whereas enrollment in the redesign format has grown to 474, yet Truman still offers the same four sections with the same resources. To learn more, contract Angelito Garcia at firstname.lastname@example.org or see http://www.thencat.org/RedesignAlliance/C2R/R1/Abstracts/HST_Abstract.htm.
In fall 2007, DePaul University in Chicago, IL piloted a redesign of College Algebra. In fall 2008, students in three traditional sections had an average score of 69.0% on a common final examination whereas students in 10 redesigned sections had an average score of 74.8%. In winter 2009, students in six redesigned sections had an average score on the common final of 73.7%. DePaul has also gone on to redesign sections of Precalculus. Here the data are also compelling. In fall 2008, students in four redesigned sections had an average final exam score of 77.5% whereas the two traditional sections had an average score of 57.1%. In winter 2009, four redesign sections and three traditional sections were taught. The redesigned sections had an average score on a common final of 80.4% compared with an average score of 77.5% in the traditional sections. To learn more, contact Jeffrey Bergen at email@example.com or see http://www.thencat.org/RedesignAlliance/C2R/R1/Abstracts/DPU_Abstract.htm.
At the University of Central Florida (UCF) in Orlando, FL, more than 4,000 students enroll in College Algebra each year. UCF piloted their redesign in fall 2008. During fall 2006, which provided the baseline, the success rate of 2250 students in the traditional course was 65%. In fall 2008, the success rate of 941 students in the redesign pilot was 74%. In fall 2009, the success rate increased to 78%. The redesign is approaching full implementation. By fall 2010, the UCF MALL (Math Assistance and Learning Lab) will have over 300 computers. Fall 2010 will also see the beginning of the redesign of Intermediate Algebra, and spring 2011 will mark the onset of the redesign of Precalculus. To learn more, contact Tammy Muhs at firstname.lastname@example.org or see http://www.thencat.org/RedesignAlliance/C2R/R2/UCF_Abstract.htm.
Engaging the nation’s community colleges in a successful redesign of their developmental math sequences.
On May 21, 2010, redesign teams of faculty and administrators from 53 semi-finalist community colleges participated in a planning workshop in Dallas, TX, the next step in the Changing the Equation application process. The purpose of the planning workshop was for the 200 attendees to learn more about developing successful plans for redesigning their developmental math sequences and to have an opportunity to get their questions answered by NCAT staff and Redesign Scholars. Carol Twigg reviewed the responses to the Readiness Criteria the semi-finalists had submitted and provided an overview of the requirements for the final proposals. The rest of the day was organized around three topics: How to Organize a Math Emporium, How to Modularize the Developmental Math Curriculum and How to Engage Students. The NCAT Redesign Scholars shared their experiences and expertise on each topic, and participants met in smaller groups with the Redesign Scholars to discuss each topic in more depth. The workshop was highly successful. As one participant commented, “It’s so good to know that we are not alone in trying to improve our students’ success, that we are part of a movement working on a common goal.”
Final proposals from the 53 semi-finalist teams are due on August 1, 2010. At least 25 institutions will be selected to receive a grant of $40,000 to redesign their developmental math course sequences using NCAT’s Emporium Model. Grant awards will be announced on August 15, 2010. Pilots of the redesigns will occur in spring 2011 and full implementation is scheduled for fall, 2011. Changing the Equation is supported by a grant from the Bill & Melinda Gates Foundation. To learn more about this program, see http://www.thencat.org/Mathematics/CTE/CTE.htm.
On June 6 - 9, 2010, Carol Twigg and Carolyn Jarmon joined 250 representatives of other projects funded by the Bill & Melinda Gates Foundation in Seattle, WA. During the meeting, Gates Foundation staff shared their perspective on the challenges faced by higher education and outlined their plans for future project development. Carol joined four other project leaders in a panel discussion entitled “Reconceptualizing Developmental Math.” The Gates Foundation has made significant investments in the five projects represented on the panel: $1.7 million to NCAT’s Changing the Equation program, $5 million to the Monterey Institute for Technology and Education to produce developmental math software, $3.6 million to the League for Innovation to develop “faculty-driven scalable pedagogy and curriculum,” $5.3 million to the Washington State Board for Community and Technical Colleges to “impact student success and higher community college completion” and a significant investment in the multi-million dollar Carnegie Statway project. To date, the Gates Foundation has invested over $100 million in reforming developmental education. Based on the panel presentations, it is unclear whether all of these projects will be able to demonstrate improvements in student success supported by data, to impact large numbers of students, and to do so within the next several years. Vicky Phillips, Director, Education, United States Program at the Foundation has stressed, “We are going to be relentless about acquiring quality data and the need to focus on evidence–based decision making. And we will go where the evidence takes us.” These projects represent five different approaches to tackling the developmental math problem. We look forward to learning about which ones have the most promise.
Featuring initiatives to scale course redesign through state- and system-wide redesign programs.
Eight project teams have fully implemented their course redesign plans as part of the SUNY Course Redesign Initiative. Four of the eight projects demonstrated improved student learning as measured by direct comparisons of content mastery. Five of the eight projects improved course completion rates (as measured by a final grade of C or better). Seven of the eight projects reduced their instructional costs—on average by 35%—and two projects saved more than originally projected. The annual savings for the eight projects was approximately $534,655 per year. Altogether these eight redesigns impact about 5,000 students, and all will definitely be sustained in the future. Brief summaries of the outcomes of each project following the fall 2009 full implementation are presented below.
Buffalo State College fully implemented its redesign of The Economic System, which annually enrolls about 500 students, using the Replacement Model. Student learning outcomes were compared using 25 common final exam items. Students in the fall 2009 redesigned course had significantly higher percentage scores (M=66.28, SD = 18.36, n = 209) than students in the spring 2009 traditional sections (M=60.64, SD = 18.53, n = 157), p < .004. The redesign also showed higher success rates (grades of C or better). In fall 2008, 67% of students in the traditional course achieved a C or better compared with 79% of students in the fall 2009 full implementation of the redesign. The redesign reduced the cost of instruction by reducing the number of sections from two per semester to one, and increasing section size from ~120 to ~240 students, reducing the number of full-time faculty teaching the course from two to one each semester. As planned, these actions decreased the cost-per-student from $94 to $51, a savings of 46%. The redesign has allowed the economics department to add graduate sections to respond to the growth of its graduate program. To learn more, contact Karen O’Quin at OQUINK@BuffaloState.edu.
SUNY Canton’s redesign project combined two biology courses, which annually enroll about 390 students, into one using the Replacement Model. To compare learning outcomes, the team administered a pre- and post-test to students in both the traditional and redesigned courses. Students in the fall 2009 redesigned course (N = 244) gained 18 points on pre/post-test performance compared with students in the spring 2009 traditional course (N = 78) who gained 12 points. Course completion rates (a final grade of C or better) also improved. During the fall 2009 full implementation, the percentage of students earning a final grade of C or better was 50% compared with 46% in the fall 2008 traditional course. SUNY Canton planned to reduce costs by increasing section size from 60 to 80 and reducing the number of sections offered from eight to five. This plan was carried out. However, one of the primary cost saving measures--hiring a course coordinator to provide support for the primary instructor--was not implemented. A state-imposed hiring freeze prevented the team from filling the position, and the tasks of the course coordinator were assumed by the lead instructor. While this change increased the planned workload of the lead instructor, his load was equivalent to that of teaching the traditional course. To learn more, contact Ronald Tavernier at Tavernier@canton.edu.
The redesign of First-Year Spanish, which annually enrolls about 380 students, at SUNY Fredonia involved combining a two-semester course sequence into a one-semester course. This enabled students who have taken high school Spanish to earn five credits in one term instead of six credits in two terms. The percentage of students receiving a grade of C or better in the traditional course was 84%; the percentage of students receiving a grade of C or better in the online portion of the redesigned course--which focused on practice of grammar, vocabulary, listening, reading and writing--was 83%. The percentage of students receiving a grade of C or better on the in-class portion of the redesigned course--which focused on communication and conversation--was 93%. In addition, 95% of students in the redesign received a cumulative grade of C or better on four oral examinations (77% received an A, 13% received a B, and 5% received a C.) The projected cost savings of more than 40% was achieved as planned. One intensive semester fulfilled the requirement usually completed in two semesters, enabling all of the resources (instructors and classrooms) to serve an entirely new group of students in the second semester. To learn more, contact Juan DeUrda at Juan.DeUrda@fredonia.edu.
The team at Niagara County Community College redesigned the introductory statistics course, which annually enrolls more than 200 students, using the Replacement Model. Student learning outcomes in the redesigned course were compared to those of the traditional course using scores on common, comprehensive final examinations. The final exam average increased from 64.9% for the traditional course to 68.7% in the full implementation of the redesigned course. The percentage of students scoring a C or better on the final exam increased from 41% for the traditional to 53% for the redesign. This difference was statistically significant at the 95% confidence level. However, the redesigned course experienced a decrease in course completion rates (grades of C or better) from 62.4% to 58.3%. A contributing factor was that a portion of redesign students who passed the final exam did not complete the required assignments. Such students would have been successful in the traditional course where homework completion did not contribute to their final grade. The redesigned course reduced instructional costs from $339 per student to $162 per student, a 52% savings. Cost reduction strategies included reducing the number of sections covered by full-time faculty and increasing the number of sections covered by adjunct faculty. Each full-time faculty member was partnered with an adjunct faculty member to team-teach two sections simultaneously. To learn more, contact Daniel Miller at email@example.com.
Using the Replacement Model, SUNY College at Old Westbury redesigned College Algebra, which has an annual enrollment of ~600. To assess learning outcomes, the team compared common final exam scores. During the three semesters (fall 2008, spring 2009 and fall 2009) when both traditional (N = 525 students) and redesigned sections (N = 516 students) were offered, there was no significant difference between the traditional and redesigned sections in final exam performance. The redesigned sections, however, had a significantly higher number of A’s on the final exam. The 70.4% success rate (grade of C or better) for the fall 2009 redesigned sections was significantly higher (99% confidence) compared with the average fall semester passing rate for 2003 – 2007, which was 62%. The cost savings plan, which projected a reduction in the cost-per-student from $176 to $155, was carried out as planned. Savings were slightly increased because in the lab, two courses used the facilities and tutoring services rather than only the College Algebra students. To learn more, contact Jim Llana at firstname.lastname@example.org.
The SUNY at Oswego team fully implemented its redesign of College Algebra using the Emporium Model. To compare student learning outcomes, the team administered a locally written, 28-item test at both the start and end of the semester. Although the comparison showed a slight decrease in the points gained from 38 in the fall 2008 traditional format to 36 in the fall 2009 full implementation, the 2009 cohort began the course with a weaker background and the difference was not statistically significant. In addition, the percentage of students earning a grade of C or better increased from 42% in the traditional course to 52% in the redesign. The redesign achieved greater savings than anticipated. The cost of the traditional course was $77,400; the cost of the redesigned course was $37,400. This represents a 52% savings rather than the planned 42% savings. These savings were achieved by changing the mix of personnel and reducing the number of instructors from seven to two instead of seven to three as planned. Traditionally, nine sections of ~25 students (N = ~200 students) were offered each year. The original redesign plan envisioned six sections of 50 to reduce the number of sections and accommodate enrollment growth. Since this growth did not occur during the full implementation, the team restructured the course to include three sections of ~75 annually. In addition, beginning in spring 2010, the number of required lab meetings was reduced from three to two per week, resulting in an additional cost savings from undergraduate learning assistant salaries. To learn more, contact Patricia Pacitti at email@example.com.
Using the Replacement Model, the SUNY at Potsdam team redesigned four history courses: European History 1500-1815, European History 1815-Present, US History to 1877 and US History 1877-Present. Historically, 27 sections of European and US History were needed to serve over 1400 general education students annually. These were collapsed into a single European history section and a single US history section, each serving 300 students per semester.
In the American History course, average scores on comparable essay questions, graded by the same rubric, improved from 2.22 in the traditional course to 2.58 in the redesigned course. Correct responses to common multiple-choice questions increased from 55% to 76%. Averaging factual (multiple choice) scores with interpretive and analytical (essay) scores in European History yielded a measure of overall learning that compared very favorably with the traditional course. Scores at the top of the distribution more than doubled (from 5% to 11%), while those in the “B” range rose significantly (from 39% to 46%). “C”-range performances diminished substantially (from 27% to 18%) while “D” scores plummeted (from 23% to 4%).
The redesign plan projected a cost savings of between 20% and 25% for the new courses. The cost savings plan was followed as proposed. The redesigned courses did not realize their potential to reduce instructional costs during the 2009-2010 academic year because enrollments ran well below capacity. Enrollment trends suggest that this problem will be quickly overcome. Between fall 2009 and spring 2010 semesters, enrollments in the European course increased by nearly 60%. In the American course, they increased by over 10%. The department expects enrollments much nearer to capacity in the 2010-11 academic year, which will yield real savings. To learn more, contact Krista Medo at firstname.lastname@example.org.
At Stony Brook University, the redesign of Physics for Life Sciences, with an annual enrollment of ~1370, replaced lecture time with three weekly interactive workshops. Pre/post-tests were used to measure changes in student learning outcomes. While students in the redesigned course showed a gain of 25.6 points compared with 17.7 points in the traditional course, the post–test execution was not consistent between the two formats of the course, making the results inconclusive. The percentage of students completing the course successfully (a grade C or better), however, was 83% in the traditional and 86% in the redesign. The team implemented the original cost savings plan, reducing the overall cost-per-student from $463 to $236, a saving of 49%. The elimination of faculty-taught recitations produced the bulk of the savings and liberated faculty for an expansion of help rooms for more courses. The new format of the lab allowed the graduate teaching assistants (GTAs) to spend less time on introducing material and more time working with the students in executing experiments. This, in turn, enabled the GTAs to grade the individual student’s contribution to the execution and analysis of the experiments in situ without having to grade a report and made possible a 20% increase in the number of students per GTA. To learn more, contact Rod Engelmann at Engelmann@sbhep.physics.sunysb.edu.
A summary of the results and comparisons of the planned and actual cost savings are found at http://www.thencat.org/States/NY/SUNY%20Outcomes.html. Final reports for each project, available at http://www.thencat.org/States/NY/SUNY%20Project%20Descriptions.htm, include learning outcome data, course completion data, cost reduction data, a discussion of the most important pedagogical techniques that led to increased learning, a discussion of the most important cost reduction techniques that led to reduced costs, a discussion of the most important implementation issues encountered during the redesign process and a discussion of future sustainability of the redesign. To learn more about the SUNY initiative, contact Sharon Gallagher at Sharon.Gallagher@suny.edu or see http://www.thencat.org/States/SUNY.htm.
On April 20-21, 2010, 15 redesign teams from the Mississippi Institutions of Higher Learning Course Redesign Initiative met in Jackson, MS to share the results of the full implementation of their redesigns in fall 2009. Full final reports will be available at http://www.thencat.org/States/MS/MS%20Project%20Descriptions.htm by the end of summer. Brief summaries of the outcomes for each project are provided below.
Projects at Mississippi State University (MSU)
MSU redesigned two biology courses, Plants and Humans and Animal Biology, using the Replacement Model. Prior to the redesign, enrollments in each course were about 750 annually. Pre- and post-tests were used in both courses to assess student learning outcomes. In Animal Biology, students in the traditional course showed a loss of .1 point; students in the redesigned course showed a gain of .8 point. In the Plants and Humans course, students in the traditional course showed a gain of 2.7 points; students in the redesigned course showed a gain of 2.6 points. The results for both courses were not significant. The percentage of students receiving a grade of C or better in Plants and Humans were similar in both course formats (89% and 88%). In Animal Biology, success rates decreased from 87% for the traditional to 81% for the redesigned course. As planned, the team was able to create additional sections of both courses without adding faculty, resulting in a reduction in the cost-per-student from $127 to $99 in Plants and Humans (a 22% savings) and $164 to $100 in Animal Biology (a 39% savings). To learn more, contact Nancy Reichert at NReichert@biology.msstate.edu.
The redesign of Survey of Chemistry I used the Replacement Model. The team assessed student learning by comparing performance on common final exams. Traditional students scored a mean of 59, and redesign students scored a mean of 56. This difference was not significant. Similarly, the percentage of students earning a grade of C or better was not significantly different between the two formats. As planned, MSU reduced the cost of instruction by doubling the annual enrollment from 500 to 1000 students on the same resource base. Section size increased from 150 to 450, and the number of sections was reduced from four to three, which decreased the cost-per-student by 62%, from $133 to $51. To learn more, contact Svein Saebo at email@example.com.
The redesign of Introduction to Statistics used the Replacement Model. To measure learning outcomes, performance on common final exams, similar in content and difficulty, in two redesigned sections taught during fall 2009 was compared to performance of students taught from 2002 to 2007. A slight improvement was seen in the redesigned sections; the mean exam score in the redesigned course was 75% compared with the five-year average of 72.7% in the traditional course. This difference was not statistically significant. In the redesigned course, 65.4% of the students completed the course with a C or better; the five-year average for students in the traditional course was 67.12%. MSU’s plan to reduce the cost of instruction by increasing the number of students served annually from 790 in 22 small sections to 980 in seven large sections was implemented successfully, but enrollment was less than expected. In future semesters, the team expects the projected 17% savings to be realized. To learn more, contact Mohsen Razzaghi at firstname.lastname@example.org.
Using the Emporium Model, MSU redesigned Statics, an engineering mechanics course enrolling ~350 students annually. During full implementation, students performed significantly better on assignments (average score of 90 versus 73) and in-class tests (average score of 79 versus 66) than their peers in the spring 2009 traditional group, at 95% confidence level. Average final exam scores of the two student groups were the same (64.7 and 64.1). Final grades from students in the traditional course during AYs 2001 – 2005 were compared to final grades in the fall 2009 redesigned course. There was no statistically significant difference between success rates (grades C or better). MSU achieved greater cost savings than originally anticipated. In fall 2009, seven fully redesigned sections of Statics were taught with one full-time instructor (course coordinator) assisted by 11 undergraduate learning assistants (ULAs) and one graduate teaching assistant (GTA). The traditional course had been taught in six sections by six adjunct and tenure-track instructors assisted by undergraduate graders. The cost-per-student was reduced from $299 to $252, a ~16% decrease versus the planned 8% decrease. Since student enrollment in the course is gradually increasing, the cost savings is expected to be sustained at the current level or show an upward trend. To learn more, contact Masoud Rais-Rohani at email@example.com.
Projects at the University of Southern Mississippi (USM)
The redesign of Introduction to Computing, a course enrolling about 700 students annually, used the Fully Online Model. To measure learning outcomes, performance on common final exams from 270 redesign students taught during full implementation was compared to performance on the same final exam for 395 traditional students taught during fall 2007. The percentage of redesign students scoring a C or better on the exam was 69% compared with 61% of traditional students. Student success rates (final grades of C or better in the course) were 72.8% in the traditional course and 68.1% in the redesign. These results occurred even though the assignments in the redesigned course were more complex than the ones previously assigned in the traditional course. The team’s cost savings plan was successfully implemented; significant savings were achieved by reducing the number of instructional staff, reducing instructional staff time required for materials development and lecture preparation and eliminating duplication of effort for tasks required of each member of the instructional staff. The consolidation of efforts resulted in a cost-per-student reduction from $249 to $109, a savings of 56%. To learn more, contact Nancy Howell at firstname.lastname@example.org.
Nutrition and Food Systems at USM is offered in two formats: traditional face-to-face sections and fully online with a total annual enrollment of about 900. In the face-to-face formats, the mean score on common final exams was 78 for the spring 2009 traditional sections and 84 for the summer and fall 2009 redesigned sections. In the online formats, the mean score on common final exams was 67 for the traditional sections and 78 for the redesigned sections. These improvements were statistically significant. Overall 78% of students in the redesigned formats achieved a C or better in the course compared with 77% of students from fall 2005 through spring 2007. The team’s original cost savings plan intended to reduce the total number of course sections to one online and one face-to-face each term and to increase the number of students per section for both formats, yielding a weighted average cost reduction for the whole course from $81 for the traditional to $31 for the redesign, a 62% savings. During the course of the redesign implementation, the team revised this plan, resulting in a cost-per-student reduction from $81 to $44, a 46% savings. To learn more, contact Denise Brown at Denise.M.Brown@usm.edu.
USM’s redesign of First-Year Spanish using the Replacement Model involved two courses. In both courses, student learning outcomes were compared using common final examinations. In Spanish 101, the mean final exam score for the traditional students was 74.82 and the mean score for the fall 2009 redesign full implementation was 73.5. In Spanish 102, the mean final exam score for the traditional students was 77.12 and the mean score for the spring 2010 redesign full implementation was 72.53. In Spanish 101, the percentage of students earning a C or better in the spring 2009 redesign pilot implementation was 74% compared with 76% in the spring 2009 traditional sections. The percentage of students earning a C or better in the fall 2009 redesign full implementation was 69%. In Spanish 102, the percentage of students earning a C or better in the fall 2009 pilot redesign implementation was 86% compared with 75% in the fall 2009 traditional sections. The percentage of students earning a C or better in the spring 2010 redesign implementation was 59%. USM carried out its proposed cost reduction plan. The number of sections was reduced, and section size doubled. The same number of students was taught with fewer faculty. Graduate teaching assistants, who used to cover two sections each, were assigned three sections each. These actions reduced the cost-per-student from $152 to $110, a 28% savings. To learn more, contact Leah Fonder-Solano at Leah.Fonder-Solano@usm.edu.
Enrolling about 850 students per year, Technical Writing at USM was redesigned using the Replacement Model. Students in the redesigned course performed significantly better on a benchmark writing assignment compared with students in the traditional course: 78% of redesign students scored "OK" or better compared with 67% of traditional students. Improvement was especially impressive at the top end of the scale where 38% of redesign students scored "Excellent" compared with just 13% of traditional students. Student performance on the lower end of the scale improved as well, if less dramatically: only 22% of redesign students scored "Poor" or "Very Poor" on the assignment compared with 33% of traditional students. Although USM planned to reduce the cost-per-student from $188 to $50, a 73% savings, the team actually reduced the cost-per-student from $188 to $62, a 67% savings. Savings were slightly less than anticipated due to two factors: 1) the need to continue offering two sections of the traditional online version of the course in order to support fully online programs that require the course for the degree, and 2) an increase from 3 to 3.5 GTAs necessary to staff lab/studio meetings and drop-in hours. To learn more, contact Michael Mays at Michael.Mays@usm.edu.
Projects in Mathematics
Six projects were conducted in mathematics with very different outcomes. One advantage of having had six redesign projects in mathematics is that it is possible to learn as much from the successes as from those which were not as successful.
One project improved both learning outcomes and student completion rates:
Two projects improved learning outcomes but student completion rates declined:
Where there is a contradiction between increased student learning outcomes and decreased completion rates, we believe, as do the project leaders, that this is a result of prior grade inflation and/or inconsistency in one case (Alcorn) and a failure to enforce appropriate deadlines in the other (USM). In both cases, these issues will be corrected in subsequent implementations.
Two projects failed to make any impact on student learning outcomes or on student completion rates, and one project negatively impacted both learning outcomes and completion rates. NCAT believes that this lack of success is directly attributable to each project’s use of the Replacement Model rather than the Emporium Model in its redesign.
Linking content and software providers with leading edge institutions.
Building on highly successful prior workshops, Pearson Education will hold its annual course redesign workshop on September 24 - 25, 2010 at the Hotel Del Coronado in San Diego, CA. Participants will learn more about how to get started on course redesign from NCAT’s Carolyn Jarmon and from large- and small-group interaction with experienced educators in accounting, biology, chemistry, developmental reading and writing, economics, English composition, math and world languages. The workshop will highlight successful course redesigns that take advantage of technology to improve student learning and efficiency in large-enrollment, introductory courses. Participants will learn how to implement course redesign principles in both quantitative and qualitative disciplines. Faculty will also have the opportunity to learn more about Pearson's leading technology products. To learn more and to register, go to http://www.pearsoncourseredesign.com/events.html or contact Karen Mullane at Karen.Mullane@pearson.com.
McGraw-Hill plans to hold a series of course redesign workshops to help instructors maximize their use of the integrated solution of McGraw-Hill Connect™ and Blackboard Learn™, which was announced in late July. McGraw-Hill Higher Education and Blackboard are partnering to introduce a new, best-of-class learning management offering that joins McGraw-Hill’s Connect™ with the latest capabilities of Blackboard’s Web-based teaching and learning platform, Blackboard Learn™. These workshops will be focused on improving student learning while reducing instructional costs using this solution. Details will be announced in upcoming issues of The Learning MarketSpace. To learn more, see McGraw-Hill Connect + Blackboard Learn or contact Torie Anderson at Victoria_anderson@mcgraw-hill.com.
At NCAT, we believe that strong partnerships between educators, publishers and software developers will accelerate our goals of improving quality and reducing costs. The ready availability of high-quality, user-tested, affordable content and technology products provides valuable support to course redesign projects, allowing faculty to focus their time and resources on pedagogy and course organization. While NCAT does not endorse any particular products, we do emphasize the value of well-designed and developed learning resources to increase the opportunities for students to be successful.
The corporate members of the Redesign Alliance have committed significant resources to learning about the principles of course redesign and are taking active steps to incorporate those principles in their products and services. Though the NCAT role always remains that of consultant and facilitator, these companies benefit from in-depth information on what worked and what didn’t in actual projects as well as a consistent emphasis on student outcomes and reduced costs in evaluating the effectiveness of their content and technology products.
For potential redesign projects, NCAT’s relationship with its corporate partners provide one source of learning materials with some key advantages for project participants: maintenance and updates are the responsibility of a commercial organization; feedback and product requirements are provided within a common NCAT-centered framework; individual educator/vendor partnerships are governed by a common vision of what’s important and what needs to be achieved. Product improvements and new product development benefit from the fact that corporate members are actively engaged in the challenges of redesign and equally excited when improvement in student outcomes and reductions in instructional costs can be demonstrated. Incorporating commercial organizations into course redesign projects can bring new resources, new talents and new skills to the mix.
For more information about membership in the Redesign Alliance, see http://www.thencat.org/RedesignAlliance/Call.htm or contact Carolyn Jarmon at cjarmon@theNCAT.org.
Reporting on initiatives that share the Center's goals and objectives.
As reported in the last issue of The Learning MarketSpace, Complete College America is building an Alliance of States ready to take bold actions to significantly increase the number of students successfully completing college and achieving degrees and credentials with value in the labor market and close attainment gaps for traditionally underrepresented populations. Thus far, 23 states have joined the Alliance. On June 3-4, 2010, representatives gathered in Nashville, TN to launch the work needed to increase college completion. Included in this initiative is the establishment of completion goals, the collection and reporting of progress and the development of action plans and policies to increase the cadre of college-educated citizens. All 23 states have committed to adopting Common Completion Metrics.
Established in 2009, Complete College America is a national nonprofit working to significantly increase the number of Americans with a college degree or credential of value and to close attainment gaps for traditionally underrepresented populations. To learn more about this effort, see http://www.completecollege.org/.
The new chair of the National Governors Association (NGA), West Virginia Governor Joe Manchin, announced his chair's initiative, Complete to Compete, which will focus on increasing the number of students in the United States who complete college degrees and certificates and improving the productivity of the country's higher education institutions. "The nation has fallen from first to twelfth in the world in the number of students who complete degrees. Now, we're faced with a generation of students that is projected to have lower educational attainment than their parents," said Governor Manchin. This initiative aims to create a set of common higher education completion and productivity measures that governors and higher education leaders can utilize to monitor state progress and compare performance to other states and between institutions. Governor Manchin believes states must take concrete steps to foster this agenda. “From transforming first-year coursework to implementing performance funding, it is up to states and institutions to create policies that can improve degree attainment and more efficiently use the dollars invested by states and students.” To read more about this initiative, see http://www.subnet.nga.org/ci/1011/.
Will standards alignment work this time? Kati Haycock’s article in the July/August 2010 issue of Change Magazine, “Building Common College Ready Standards,” offers cautious optimism. Haycock provides a brief history of the standards movement and where it is today. If successful, this initiative could ensure that more graduating seniors are immediately ready for college and fewer need significant remediation on admission. Haycock also describes the role higher education can play in bringing into alignment what is taught in high school and what is needed to be ready to engage in college-level learning. According to Haycock, “Higher education faculty can play an important role. They can host conversations about the new standards with teachers and principals in their local school districts and explore what it means for a student to meet each standard, what evidence would demonstrate mastery, and how such evidence should be weighed.”
Kati Haycock is president of the Washington D.C.-based Education Trust; the Trust works with educators and policymakers to improve achievement, especially among low income students and students of color. To read the full article, see http://www.changemag.org/Archives/Back%20Issues/July-August%202010/building-college-readiness-full.html.
The National Center for Academic Transformation serves as a source of expertise and support for those in
Copyright 2010, The National Center for Academic Transformation