The previous sections of this book covered the process of determining your instructional design needs and then developing your learning resources and activities. Now, we focus on the Implement phase of the ADDIE process. This is our opportunity to test our instructional design efforts with a live audience and get feedback from a student’s point of view.

In many cases, we do not have the opportunity to pilot a selected segment of our online courses with a test audience. Our test audience is a group of actual students in a live offering of our courses. That does not mean that we can skip the process of gathering feedback so that we can improve the course design for future offerings. For our purposes here, we will look at the process of pilot testing the selected learning modules developed as part of the ID projects developed by participants in my instructional design courses.

First things first, some important dos and don’ts for pilot testing:

• Do make sure that your pilot module is published and fully accessible by participants by the designated “go live” date for the pilot testing phase. Just like a real-world course, the start date for pilot testing is the same as the start date for an academic term. You are responsible for ensuring that your course is ready for the first day of class!
• Do NOT make substantive changes to your pilot module during the pilot testing phase. Once students have begun a course, you should NEVER make substantive changes to the overall structure, site design, learner activities, or assessments. This will only lead to confusion and frustration for your students!
• Do make necessary administrative updates, such as fixing typographical errors or broken links that you or your students discover while working through the module.
• Do set up a designated place where students can ask questions or report issues they are experiencing. I like to include a Questions and Help discussion forum in my courses. I ask my students to use this place to ask any course-related questions or report technical issues they encounter. Doing this serves several purposes. First, it gives students a place to check if the question has already been asked and answered or if the issues have already been reported and resolved. Second, it helps me, as an instructor, to manage my time. I set aside a designated time to check in with the Questions and Help forum before looking at other discussion activities. This helps me to avoid the deluge of emails instructors often get from students and keep focused on other tasks throughout the day. Third, the Questions and Help forum is a “one-stop shop” for me to collate and review trouble spots and issues to address once the course has ended. I do not need to search through other forums or my email inbox to find all these issues. Of course, I remind my students that if their question or issue is of a personal or sensitive nature, they should feel free to email me with those queries!
• Do keep an “updates logbook.” Even if the updates you make throughout a course are only administrative, for live courses reaching a certain threshold of updates may trigger a formal course or program review process at an academic institution. Having a record of those changes will facilitate this process and provide a guide for instructional developers who are responsible for maintaining “master copies” of online courses.

Here is an example of an “updates logbook” that I kept for a previous offering of one of my instructional design courses.

Gathering Student Feedback

The use of standardized student feedback forms to make decisions about revising the instructional design of our online courses is covered in the Using Student Feedback and Rubrics to Guide Reflection chapter in the Evaluate section of this eBook. It is important to ensure that student participants in your pilot module have easy access to whatever feedback tool you are using and that they know when and how to submit their feedback.

The Pilot Testing Process

I typically randomly assign participants to a group with three or four classmates for the pilot testing phase in my instructional design courses. I do not let participants pick their own groups for two reasons:

1. You do not get to pick your real students before a live course.
2. I want to ensure that nobody is part of a group who has previously seen the pilot project and completed an “expert” peer review during the design and development process.

We typically set aside one week for pilot testing, where each group member takes a turn as the “instructor” for one day of the week. This means that on the first day, everyone looks at just one pilot module, and the module’s author monitors things and interacts with their group members just as an instructor would in a real course. At the end of the day, everyone completes a student feedback form and forwards it to the module’s author. Then, on subsequent days, group members switch off to look at a different pilot module. At the end of the week, group members gather together all of the student feedback forms they have filled out and submit them to me as one of the peer review assignments. They also gather all of the student feedback forms they have received to prepare for the Evaluate phase of their projects.