HOW TO ORGANIZE A CAMPUS-WIDE COURSE REDESIGN PROGRAM USING NCAT'S METHODOLOGY
VI. Assessing Course Readiness
The purpose of this stage of the program is to ensure that course redesign teams are created that are clear about what they are trying to accomplish and how they intend to achieve it. This stage, which will take about three months to complete, comprises three parts:
- Require applicants to establish course redesign teams
- Require applicants to complete the readiness instrument as a team
- Review responses to the readiness instrument
Require Applicants to Establish Course Redesign Teams
Program leaders should require those interested in participating in the redesign initiative to establish redesign teams and to think carefully about which courses are good candidates for redesign. Successful course redesign is the product of a team effort. It is neither a faculty project nor an administrative project nor a professional staff project. It takes all of those people—because it is a team effort.
Teams that are well organized understand the amount of work needed for the redesign and allocate the redesign work among members. The teams move expeditiously through the planning and development process—spending their time effectively—and achieve both success and sustainability. In evaluating prior redesign programs, we have found that taking a team approach always receives the highest possible rating from participants. We have also found that projects that did not form project teams tended to struggle and be less successful.
Those interested in participating in the program should establish redesign teams that consist of the following.
- Faculty Experts. Course redesign requires that faculty experts explicitly identify a course’s desired learning outcomes and agree on course content. Most courses appropriate for course redesign are typically taught by more than one faculty member. To ensure course consistency, faculty experts must work together on the redesign—resolving any differences in how the course will be offered—and must collaboratively plan the most effective way to accomplish the redesign goals.
- Administrators. Because redesigns affect multiple sections, large numbers of students, and academic policies and practices, it is important that the team involve academic administrators. The level of those administrators will depend on the organization of the institution and the institution’s size. For some, it will be the provost/academic vice president or designee; for others, it will be a dean or department chair. Those team members play important roles when institutional issues arise such as changes in scheduling or the use of classroom space. If unexpected issues arise in the process of redesign implementation, administrators can help the team resolve them quickly and effectively across institutional offices.
- Technology Professionals. These team members provide expertise so that the redesign goals are accomplished in ways that make the technology as easy as possible for students to use. Technology professionals contribute ideas about how to increase interaction with content as well as with other students. They also suggest design approaches that ensure that the technology will not limit students’ learning options.
- Assessment Experts. In Chapter VII of How to Redesign a College Course Using NCAT’s Methodology, NCAT sets forth straightforward methods whereby student learning in the redesigned course can be compared with student learning in the traditional course. It is, however, useful to include on the team a member who is knowledgeable about assessment and research design—especially if the institution seeks to measure additional facets of the redesign such as performance in downstream courses or student satisfaction. Such expertise may be found in a department of psychology or a department of education or in offices of institutional research.
- Instructional Designers. If your campus is fortunate enough to have instructional designers on staff, you may wish to add one to the team. An instructional designer can help guide the re-sequencing of instruction and provide insight into learning theory and modularization. Subject matter experts are not always learning experts, and such guidance can be crucial.
Why: Part of the goal of the redesign process is to teach institutions how to improve learning while reducing cost, which means involving a variety of personnel at all stages of the projects. Teams are key to successful redesign projects, and all players—not just faculty—should be included in early planning because of the multiple dimensions involved in large-scale course redesign.
Require Applicants to Complete the Course Readiness Instrument as a Team
Some courses may be more ready than others to be the focus of a large-scale redesign effort. Because of prior experiences with technology-mediated teaching and learning, and because of numerous attitudinal factors, some faculty members may be more ready to engage in large-scale redesign efforts to achieve the program’s goals.
Those interested in participating in the redesign program should be asked to think carefully about which courses are good candidates for redesign and to respond to the Course Readiness Instrument as a team, which will be the team’s first activity. Completing the readiness instrument enables each team to assess collectively its strengths and weaknesses, thereby gaining an understanding of what it needs to do to close gaps in its preparation early in the process. No team perfectly meets all of the readiness criteria, especially at the beginning of the planning process. Every team will discover things it needs to work on in order to carry out a successful course redesign.
Why: This exercise is designed to establish the importance of a team effort, to help teams select the appropriate course, to analyze their institutional circumstances so they can be sure they are prepared to launch a successful project and to identify “gaps” in readiness that need to be addressed. Reviewing readiness responses enables the program leaders to assess (1) teams’ seriousness of purpose in the way they complete their responses and (2) their readiness to take on a large-scale project as well as to eliminate those teams that have chosen inappropriate courses.
Course Readiness Instrument
The readiness criteria are designed to help program leaders select the courses to be redesigned with the highest chances of success. In some cases, applicants will be asked to read an article, discuss the reading as a team and make a tentative decision, which may change as they learn more about the redesign process. Answering each as honestly as possible—and providing data to support their answers—will lead to the most positive outcome for each potential project.
1. Course Choice
Choosing the right course is the first step in a successful course redesign project. Courses that face academic or resource problems or both are the best targets. What impact will redesigning the course have on the curriculum, on students and on the institution—i.e., why do you want to redesign this course? Please be specific by providing data on pass rates, enrollment numbers, and so on.
Is there an academic problem in this course such as a high failure rate? Does the course face a resource problem such as how to meet increased enrollment demand with no commensurate increase in resources? Is the redesign linked to some larger institutional goal such as a Quality Enhancement Plan (QEP), a campus strategic plan, or a reaccreditation process?
2. Redesign Model
When you develop your redesign plan, you will be asked to select a redesign model. Please read Chapter III of How to Redesign a College Course Using NCAT’s Methodology, which describes six possible models. At this point in the planning process, which redesign model do you think would be most appropriate for your redesign? Why?
When you look at the models chosen by successful redesign projects, you will notice that certain disciplines select particular models—e.g., math uses the emporium model, foreign languages use the replacement model, and so on. What aspects of the model you are thinking about using fit your particular discipline and your particular students? Have other successful course redesign projects in your discipline used this model?
3. Assessment Plan
When you develop your redesign plan, you will be asked to select an assessment model. Please read Chapter VII of How to Redesign a College Course Using NCAT’s Methodology, which describes four possible models. At this point in the planning process, which assessment model do you think would be most appropriate for your redesign? Why?
Successful large-scale redesign efforts begin by identifying the intended learning outcomes and developing alternative methods other than lecture/presentation for achieving them. Have those responsible for the course identified the course’s expected/intended learning outcomes in detail? Do you have baseline data for the course in its traditional format? If so, please describe. If not, how do you plan to collect baseline data and compare it to student learning outcomes after you have redesigned the course?
4. Cost Savings Plan
While developing your redesign plan, you will be asked to select a cost reduction strategy. Please read Chapter V of How to Redesign a College Course Using NCAT’s Methodology, which describes a number of strategies for producing cost savings. At this point in the planning process, which cost savings strategy do you think would be most appropriate for your redesign? Why?
What does cost savings mean in practice? In the past, cost reduction in higher education has meant loss of jobs, but that’s not the NCAT approach. In every NCAT course redesign project, the cost savings achieved through the redesigned courses remained in the department that generated them, and the savings achieved were used for instructional purposes. By reducing the cost of offering the redesigned course, institutions have been able to reallocate and do what they would like to do if they had additional resources.
5. Learning Materials
Successful course redesign that improves student learning while reducing instructional costs is heavily dependent upon high-quality, interactive learning materials. Today’s commercial marketplace offers many reasonably priced materials that meet that requirement. NCAT has worked with close to 200 redesign projects that have considered, used and continue to rely upon such materials. Are the participating faculty members able and willing to incorporate existing curricular materials in order to focus work on redesign issues rather than materials creation? What learning materials are you thinking about using in your redesign?
Ideally, one wants the faculty to have a "head start" in the redesign process if possible. Is the discipline one with a comparatively large existing body of technology-based curricular materials and/or assessment instruments? Are the faculty willing to use these materials if they meet course objectives? Will they employ an appropriate blend of using these materials and created "home-grown" materials in a non-dogmatic fashion? Are they willing to partner with other content providers such as commercial software producers or other universities who have developed technology-based materials?
6. Active Learning
Greater student engagement with course content and with one another, supported by information technology, is essential to achieving student success. Do the course faculty members have an understanding of and some experience with integrating elements of computer-based instruction into existing courses to support active learning?
Sound pedagogy is the key to successful redesign projects. When sound pedagogy leads, technology becomes an enabler for good practice rather than the driver. Some faculty may have a great deal of enthusiasm for large-scale redesign but little prior experience in this area. It is difficult to complete a successful project by starting from scratch. Having some experience helps to prepare for large-scale redesign efforts. Have the faculty systematically thought about and investigated alternative methods for empowering students to learn? What evidence can you provide to demonstrate faculty experience with integrating computing into existing courses in order to support active learning?
7. Collective Commitment
Collective commitment is a key element in the success and sustainability of redesign projects. As part of the planning process, you have been asked to form a course redesign team. Please describe the members of your team, list the skills they bring to the project, and determine what their roles will be in both the planning and implementation phases of the project. Please read Chapter XIV of How to Redesign a College Course Using NCAT’s Methodology, which discusses how to achieve initial and ongoing consensus among faculty, campus offices, and senior administrators.
Are the faculty ready to collaborate? Have they engaged in joint conversations about the need for change? Are decisions about the course made collectively—in other words, beyond the individual faculty member level? Substantive changes cannot rely on faculty initiative alone because they are systemic and involve changes in such areas as policy (class meeting times, contact-hour requirements, governance approvals), budgeting (planning and processes that support innovation), systems (registration systems, classroom assignments), and infrastructure (equipment purchase and deployment). What is the level of support for the project beyond the departmental level?
Teams wishing to participate in the program should send a narrative addressing each of the course readiness criteria (about one page each) as they apply to the selected course, focusing on evidence that demonstrates the way in which they meet each criterion.
Responses to the Course Readiness Instrument should have a deadline date within the timeline and should be submitted electronically.
Teams should be asked to include a cover page with their proposals on which they
- List all team members by name including titles, academic affiliation, phone numbers and email addresses;
- Identify the person who is the primary contact for the team project, with the understanding that the primary contact will share communications appropriately with the rest of your team.
Review Responses to the Readiness Instrument
Program leaders need to review the responses in order to select teams to be invited to Workshop II: Developing the Redesign Proposal described in Chapter VI. NCAT recommends that reviewers use a three-point scale where each response to the Readiness Instrument is read independently by program leaders and rated 1 (strong response; no outstanding issues or only minor ones), 2 (potentially acceptable pending resolution of outstanding issues or confusion), or 3 (weak response or does not meet program guidelines).
Why: Responses to the readiness criteria provide indicators of how well teams understand the program, how they are thinking about possible redesigns, and how much initial preparation the teams have undertaken. The responses generally will not lead to rejection of a team’s ideas this early in the process. Some of the responses, however, will serve as an early alert or warning that some teams or team members have not totally embraced the goals of the program or that some may need additional explanation of exactly how the process will work. The responses enable the program leaders to send feedback to those submitting responses as well as to clarify or emphasize the program’s goals for those who are not clear at this point or who seem to demonstrate some ambivalence about the goals. The responses also help program leaders advise weaker redesign teams about what those teams need to do to become ready.
In most situations, completing the readiness instrument represents a first pass at ensuring that participants understand the program’s goals and expectations. In cases in which a large number of responses to the instrument are received—as was the case in most NCAT national programs—the readiness criteria may enable the program leaders to make screening decisions. The decision to eliminate a particular proposal that received a 3 rating would be based on things like the team’s selecting the wrong course or making statements that clearly indicate the team has no interest in reducing cost. The program leaders can address those issues individually with teams or simply not invite a particular team to Workshop II.
Readiness Review Criteria
Indicators of strong responses to each criterion are listed next.
1. Course Choice
- The enrollment in the course is relatively high and the course is taught in more than one section by more than one instructor.
- The course faces a clear problem that redesign can solve such as high drop/withdrawal/failure rates, inconsistency among sections, and difficulty finding qualified adjuncts.
- The team intends to redesign the entire course: all sections and for all students.
- Faculty plan to coordinate their pedagogical approach rather than leaving it up to individual instructors to make decisions so that all students have a uniform, high-quality learning experience.
2. Redesign Model
- The team has thoughtfully considered the six models and has made its tentative selection of a particular model based on its readings, its examination of NCAT case studies in its disciplines, and the characteristics of its students.
- The institution and the team can support the choice of model (e.g., sufficient lab capacity for an Emporium Model, sufficient skills and support for the relatively complex Buffet Model.)
3. Assessment Plan
- The team has established learning outcomes for the course to be redesigned that have been agreed to by all who teach the course.
- The team has already collected baseline assessment data for the course (e.g., the instructors may have used a common final exam in all sections of the course for the past five years and have scores for students) or has a clear description of how it expects to collect the needed data for the traditional course during the pilot period.
- The team has thought about how to collect comparable data in the redesigned course.
4. Cost Savings Plan
- The team has read Chapter V of How to Redesign a College Course Using NCAT’s Methodology and understands and supports the program’s goal of cost reduction.
- The team has decided on a tentative cost-saving strategy. The more clearly and simply the strategy can be stated, the stronger the response.
5. Learning Materials
- The team is clear about the program’s focus—which is not on software development—and is committed to using existing, commercial products as a centerpiece of the redesign.
- The team has investigated or is willing to investigate the range of existing learning software in the selected academic area and to collaboratively select a product that will foster the goals of the redesign. For example, the team may have already made appointments with publishers or software companies or contacted other institutions to learn what they are using in comparable courses that have been redesigned.
- The team sees that software is an integral focus of the course rather than a supplement or add-on.
- All faculty teaching the course will use the same learning materials, collaboratively selected by the team.
- The team recognizes the need for training and mentoring of all faculty—both full-time and adjuncts—teaching the course in how to use the materials.
- The information technology staff is willing to assist the team in installing and maintaining the software and in training and supporting faculty.
6. Active Learning
- The team understands the need to move beyond lecturing to engage students in active-learning activities. Some lecture may be included, but it represents a minor portion of the course; that is, students will spend the majority of class time working in small groups solving problems or actively engaged in using interactive software.
- The team can supply concrete examples of what it will do to foster active learning by way of such statements as, “Low-stakes quizzes will be required twice each week” or “Students will be expected to work in small groups online and then present the results of their work in class.”
- The selected software will include features that enable students to do such things as practice key content ideas and apply principles.
7. Collective Commitment
- A team has been created that includes faculty as well as administrators, information technology representatives, and, if available, instructional designers.
- The team recognizes the importance of collective commitment and its pivotal role in sustaining the redesign.
- The team has achieved consensus among those teaching the course and in the department about the redesign initiative and its goals, the anticipated changes in the course, and the impact on the department.
- The team has alerted other institutional offices—such as advising, financial aid, and the registrar—that the redesign will require their involvement.
Q: NCAT requires potential program participants to form teams. Do you ever run into problems in this regard?
A: Ideally, teams should be formed immediately after the first workshop—meaning, very early in the process. Responses to the readiness criteria and workshop homework should be completed as a team. In our experience, sometimes individuals rather than teams have completed those assignments. The worst instance of this approach occurs when a team asks the business office to complete the Cost Planning Tool, a sure indicator that the team does not understand the program’s goals. Without the early formation of teams and consistent participation (e.g., lack of correlation between attendees at the first two workshops), a shared learning experience cannot result. The consequences are weak proposals and weak projects.
Q: Who should review the responses to the readiness criteria?
A: The project leaders (the full team) should read the responses to the readiness criteria and rank them individually. The leaders should then meet as a team to discuss the rankings and the responses.
Q: What should the program leaders do if a team chooses the “wrong” course?
A: This can sometimes happen when a particular team becomes enthusiastic about course redesign, but course redesign is not applicable to the team’s course. Typically, this happens because the course is already so inexpensive that there is no easy way to reduce costs. What motivates the team is pedagogical improvement. The program leaders need to remember that a course redesign initiative is not simply a quality improvement program.
Q: Is it possible to choose the wrong redesign model?
A: Generally, any of NCAT’s six models will work with any academic area—with two important exceptions: First, redesigns in mathematics should be required to use the Emporium Model. (See How to Redesign a College-Level or Developmental Math Course by Using the Emporium Model for a full discussion of the reasons.) And second, we do not believe that the Fully Online Model is appropriate for traditional-age freshman students—with the exception of information technology courses—or for particularly disadvantaged students because such students require a great deal of structure in order to succeed. (See The Essential Elements of Course Redesign for an elaborated discussion of this point.) The students in almost all of NCAT’s fully online redesigned courses (see http://www.theNCAT.org/PCR/model_online_all.htm) have been mature adults for whom the fully online environment has proved very effective.
Q: What should the program leaders do if there is dissension in a department and some faculty members don’t think the redesign is a good idea?
A: You need to assess the seriousness of the objections. It may surprise you to know that some faculty are resistant to change. If this is the reason, you need to persuade the resisting faculty to let the course redesign faculty conduct an experiment and judge its merits based on the data. A redesign that improves learning, increases course completion and reduces costs—supported by valid and reliable data—is hard to argue against. But if there is a bitter division within the department or some kind of ideological issue driving the resistance, it would be prudent to not allow the department to move forward in the application process. Eventually, the department’s composition will change, and the newly fashioned department might participate in future rounds of the initiative.
Table of Contents
I. The Critical Components of a Successful Course Redesign Program
II. Assessing Institutional Readiness to Redesign
III. Making Key Decisions before the Program's Launch
IV. Developing a Plan of Work
V. Building Awareness and Capacity
VI. Assessing Course Readiness
VII. Preparing Teams to Submit Strong Proposals
VIII. Selecting Proposals That Will Succeed
IX. Monitoring the Redesign Implementations
X. Maintaining Consensus and Ensuring Sustainability
XI. Building Capacity and Scaling Initial Success
A. Plan of Work
B. Publicity Plan
C. Call to Participate
D. Application Guidelines
E. Workshop I Agenda
F. Workshop II Homework
G. Workshop Logistics
H. Workshop II Agenda
I. Workshop III Invitation
J. Workshop III Agenda
K. Workshop IV Invitation
L. Workshop IV Agenda
M. Final Report Format
N. Program Evaluation