18

Note: The subcategories in this section are adapted from those used by Wiggins and McTighe (1998) in Understanding by Design.

Quizzes/Tests

Selected and short response assessments (e.g. tests or quizzes) are useful for assessing students’ abilities to recognize and recall content. They are also fairly easy to grade; and when faced with a large class size, you can make the grading automatic depending on the question type. However, these online tools also arguably provide students with “more ways to be academically dishonest” (Watson and Sottile, 2010).

In the design of effective assessments of learning, Hoffman and Lowe (2011, January) note that the “focus must be on student learning, not student control.” Particularly when dealing with online assessment (e.g., the ubiquitous auto-scored multiple choice quiz tools within learning management systems) it is tempting to design a testing environment in which all variables are controlled and student responses do naught but reveal students’ mastery of course objectives. However, as Dietz-Uhler and Hurn (2011) note, “the evidence, although scant, suggests that academic dishonesty occurs frequently and equally in online and face-to-face courses” (p. 75). It is counter-productive to adopt an adversarial stance as we attempt to fence in students to prevent them from cheating (in any modality). Nevertheless, there are steps we can take to make online testing more effective. Many of these are applicable to face-to-face environments as well.

Creating Effective Online Tests

Hoffman and Lowe (2011, January) identify a number of techniques for creating effective online
assessments. These are grouped into online assessment tool features and assessment design strategies.

Online Assessment Tool Features

Online quizzing tools typically provide some affordance for randomization of test items. Depending upon how the instructor uses the tool, this may range from merely randomizing the order in which the same set of items appears to each student all the way to sophisticated alternative test versions in which test items in various content categories and at different levels of difficulty are dynamically-generated for each student (i.e., each student receives a different test, but each version is equivalent). The instructor may impose assessment time limits such that the test is only available within a certain window of opportunity (e.g., an entire week or just one evening). Additionally, time limits can also be placed on the period between the opening of the quiz and its submission (e.g., a few minutes to multiple hours). Related to this restriction, the instructor can also allow students to see the entire test at once or only one test item at a time. Supported by the online quizzing tool, the instructor may choose to establish rules for assessment completion. For instance, students may be required to complete the quiz in one sitting once the quiz is launched, or they may have the option to start the quiz, log-out, and come back later (within whatever time restrictions have been established). Online assessment tools also support proctoring if the instructor (or institution) chooses to undertake the logistical arrangements involved in vetting proctors. An approved individual receives a password to unlock the quiz and then he remains present while the student takes the test. The proctor may be asked to verify the student’s identity and/or ensure compliance with certain test-taking protocols (e.g., open/closed book, etc.). Commercial
tools for remote proctoring have appeared on the market in recent years. The functionality of the proctoring programs range from taking pictures of students during an exam to a remote individual watching students via live video feed. The conundrum here is whether the institution or student pays the fees to proctor an exam. For more information about proctoring options at the University of Alberta, contact the Centre for Teaching and Learning.

Assessment Design Strategies

Apart from the affordances of the online testing tools, online auto-scored assessments may also benefit from well-designed multiple-choice items with an emphasis on application and higher-level thinking. While many online quizzes (especially many of those available as supplemental instructor resources) focus on low-level factual recall, multiple-choice items may be written at the higher application, analysis, or evaluation levels. Such items often involve some sort of scenario aimed at promoting learning transfer from one context to another. Additional strategies might require students to view a chart/graph and select the most accurate interpretation from among several alternatives or even to collaborate with classmates in selecting the best justification statement for why a given answer is correct prior to individually submitting their quizzes.

For detailed information on the kinds of assessment design strategies contact the Centre for Teaching and Learning and/or enroll in Concepts in Course Design: Mapping Out a Learner Centered Course available in eClass. In particular, if you would like a refresher on writing effective multiple choice items at various cognitive levels, you may wish to review the following:

• Writing Good Multiple Choice Questions
• Using Bloom’s Taxonomy to Create Multiple-Choice Questions
• Effective Assessment Examples
• Question Improvement Suggestions

Many of the above techniques for creating more effective assessments are relevant for online quizzes, traditional face-to-face exams, and online testing implemented in a face-to-face environment; a range of automated assessment options in a blended learning course.

Essays/Academic Prompts

Assessments that require a subjective analysis are often more difficult and time consuming to grade, however this type of assessment is appropriate for gauging how well students are able to apply the concepts learned in class. Within most learning management systems (LMS, e.g. Moodle) there are a variety of tools to facilitate these types of assessments. Such platforms typically include the following tools at a minimum:

1. Discussion forums –often used for generating student-to-student interaction based on an instructor-specified critical thinking challenge.
2. Assessment tool – can be used to construct essay-type questions (which must be manually scored).
3. Assignment tool – can be used to submit papers, essays, or other types of assignments.

Projects/Authentic Tasks

The following section is excerpted from “Assessment and Evaluation” by Dan O’Reilly and Kevin Kelly in the Commonwealth of Learning’s Education for a Digital World in compliance with the Commonwealth of Learning’s legal notice and may not be remixed apart from compliance with their repackaging guidelines.

Authentic student assessment strategies for the online environment

Often when we talk of assessment in an online environment, we think of automated quizzes and grade books. While useful in many circumstances, automated quizzes do not always accurately reflect a student’s abilities, especially when you are asking them to achieve a higher level of difficulty in the cognitive learning domain, to demonstrate a physical skill in the psychomotor learning domain, or to evaluate attitudes in the affective learning domain (see descriptions of learning domains at: Cognitive Domain, Psychomotor Domain, Affective Domain). Authentic assessment—assessing student abilities to apply knowledge, skills, and attitudes to real world problems—is not only possible in an online environment; it is getting more popular.

When you consider what types of online assessment strategies to choose, the list will be very similar to the print-based strategies that you know and already use. However, there are a few additional assessment strategies that the online environment makes possible. The list below is not comprehensive by any means. It also does not show which tools could be used to facilitate the different types of assessment strategies. Some of these activities may require students to have access to equipment or software applications to complete.

Table 14.1. Assessment strategies and disciplines that may commonly use them

Notice that some assessment strategies require participation by someone other than the student. For example, a K–12 master teacher would submit an observation log for a credential student performing his or her student teaching. Similarly, a health clinic supervisor would submit an observation log for a nursing student related to his or her abilities to draw blood for testing. A theatre arts student may need someone to record his or her monologue.

Some assessment strategies allow students to get creative. It is important to make sure that students have access to, or ability to use the technologies required to complete the tasks, but once you do that, you could ask students to create a video advertisement that demonstrates the application of marketing principles, an audio recording that demonstrates mastery of inflection and tone when speaking Mandarin Chinese, or a PowerPoint slide show with audio clips that demonstrates competency with teacher education standards. The age-old practice of storytelling has been “remastered” as digital storytelling through blogs, wikis, podcasts, and more. Students are taking advantage of these new media formats to illustrate that they have met certain requirements. In some cases, each product becomes an “asset” or “artifact” in a larger electronic portfolio that contains items for a single class, an entire program or department, or all curricular and co-curricular work that a student does.

Regardless of what products students provide to show their abilities, you need a way to evaluate their work.

Defining Expectations

After determining how students will show how they can meet the learning objectives, it is time to choose an evaluation method. You can use a number of tools, ranging from a simple checklist of criteria to a rubric that contains the same criteria as well as a range of performance and degrees to which students meet the criteria.

You can use qualitative or quantitative degrees to evaluate criteria (see Table 14.2 for an example of each). Share the checklist or rubric with students before they begin the assignment, so they know what will be expected of them. In some cases, instructors create the entire rubric, or portions of it, with the students.

Table 14.2. Portion of a student presentation assessment rubric

Preparing an Assignment for Assessment

The first step to assessing online work is to prepare each assignment. Since students may not have you
around to ask questions, you need to anticipate the types of information that students need. There are
some standard items to include in your instructions for all types of online assignments:

• Name of the assignment (This should be the same name as listed in the syllabus).
• Learning objective(s) to which this assignment relates.
• When the assignment is due.
• Any resources that you recommend using to complete the assignment.
• Expectations (length, level of effort, number of citations required, etc.).
• Level of group participation (individual assignments, group or team projects, and entire class projects).
• Process (how students turn in the assignment, if they provide peer review, how peers give feedback, how you give feedback).
• Grading criteria (include rubric if you are using one).

By including these items, you give students a better idea of what you want them to do.

ATTRIBUTIONS

This chapter is a remix containing materials licensed under a variety of open licenses including:

• derivative work of content from The BlendKit Reader, edited by Dr. Kelvin Thompson, available under a CC-BY-NC-SA 3.0 license;
• derivative work of content from Assessment and Evaluation byDan O’Reilly and Kevin Kelly. This work was published inEducation for a Digital World for the Commonwealth of Learning and is available under a CC-BY-SA 4.0 license
• derivative work of content from Evaluating and Improving Your Online Teaching Effectiveness by Susan Crichton and Elizabeth Childs. This work was published inEducation for a Digital World for the Commonwealth of Learning and is available under a CC-BY-SA 4.0 license