Which Tool?

Tests are, of course, not the only way to evaluate a learner’s mastery of an outcome. But with all the assessment tools to choose from, how do you match the right tool to the outcome? This page isn’t meant to give you the “right” answers, but rather focus on the sort of things that you need to consider when you are planning your assessments. To begin with, you need to ask yourself:

• Thinking about the student’s future employers or clients, what am I really interested in assessing?
• Is my purpose an initial diagnostic, a formative assessment, or a summative evaluation?
• What assessment technique measures the outcome to the correct “Bloom’s level”?
• What assessment technique is authentic to the industry?
• What planning needs to go into your proposed assessment tool? How will it be implemented, i.e. how will expectations be communicated to students, and how will students access resources needed to complete the assignment? How will it be graded?
• What constraints do you need to consider (e.g. accrediting body requirements, program budget, industry expectations, etc.)

Answer these questions, and then select a good assessment tool.

What the Research Says…

1. For attitude, belief, opinion, and communication related outcomes (affective domain), consider peer-to-peer discussions (Schaber, Wilcox, Whiteside, March, & Brooks, 2010) and action mazes (Calandra & Harmon, 2012).
2. For hands-on and physical skills (psychomotor domain), consider demonstration (face to face) or video (online).
3. For knowledge & thinking outcomes (cognitive domain), consider electronically submitted essays, short answers, multiple choice with specific targeted feedback, portfolio submissions, etc. — check out the University of Connecticut’s list of cognitive domain assessment tools here. More objective items (like multiple choice) are generally better suited to the lower-order cognitive domain; more subjective items (like portfolios) are generally better suited to the higher-order cognitive domain.
4. Always consider carefully whether multiple choice tests are the way to go, and if they are, be sure to use good design practices in creating the questions (see OER: Multiple Choice).
5. Online discussions and other peer-peer interactions are tools for both teaching and assessing soft skills (affective domain: attitudes, beliefs, opinions, and the art of communication) and higher-order cognitive skills. Their utility diminishes when teaching or assessing specific knowledge or physical skills (lower-order cognitive and psychomotor domains). For more information, see PEP Talk: Discussions.
6. Psychomotor (hands-on / physical) skills are ideally evaluated face-to-face through direct observation (using the real tools or realistic simulators). Where this isn’t practical, well-designed and well-produced video (see the PEP Talk: Video) allow student performances to be shot by recorded and uploaded for instructor grading, and self and peer review.

Top 5 Tips

1. Write your own multiple choice questions. Exam banks are designed for norm-based, not criterion-based assessment. Sask Polytech is a criterion (outcome) based school! If you must use a bank, go through question by question and eliminate or reword any question that does not directly evaluate the learning outcome, is not relevant to the lives of your students, uses obfuscating language, or makes reference to anything not explicitly covered in your program.
2. Use online discussions to assess affective domain outcomes, i.e. uncertainties or controversial aspects of your learning outcomes, soft skills, and “grey areas.” Discussions can be used to assess students’ knowledge of their own learning, their respect for diversity of opinion, and their industry-acceptable communication in the context of uncertainty or conflict.
3. Use a skill demonstration to assess criteria in the psychomotor (skills) domain. Be sure that students have a rubric /checklist well in advance, preferably near the start of the course, so that they know exactly upon which skills they will be evaluated. If students are at a distance, be sure to provide a mechanism for them to record their skill demonstration and upload it to you. If you intend to infer a mastered skill from completion of a finished product, ensure you have considered whether all alternative processes for creating a finished product are acceptable.
4. Be careful with tasks involving writing. Not all students have comparable writing skills. Your assessment will be invalid if it advantages some students over others who otherwise have met the outcome, unless you are explicitly evaluating writing skills in your outcome.
5. Use self and peer assessments, particularly for formative assessments. Students should have clear grading criteria when conducting such assessments (like a marking scheme or a rubric). If appropriate, students can help design such grading criteria themselves.

Cautionary Notes

Online Discussions:

Keep discussions as forums for:

• presenting different ideas
• expressing uncertainties
• collaborating on complex tasks, and
• assessing the affective and higher-order cognitive domain criteria that are related to these activities.

Avoid using the discussions as public drop boxes for summative assignments, or as forums that reward students who like to type a lot.

Equal Access:

No matter what assessment you choose, be sure that all your students can access it. You can assess skill demonstrations at a distance, but be sure that your students have access to appropriate A/V facilities, internet bandwidth for uploading videos or other artifacts, instructions on how to do it, and technical support if they run into trouble.

Extend Your Skills

Consider…

• Using response journals, blogs or discussion posts to evaluate cognitive and affective domain tasks at the same time.
• Using a realistic physical structure that needs to be “fixed,” like domestic wiring, an emergency department patient room, a poorly laid-out office design, etc., to evaluate cognitive and psychomotor domain at the same time.
• Using a role-play scenario involving group problem solving, dangerous client behaviour, dangerous worker behaviour, etc. within a physical space to evaluate affective and psychomotor domain at the same time.
• Using a team approach to responding to an emergency, building a structure, developing an app, or any other realistic scenario, to evaluate cognitive, affective and psychomotor tasks at the same time.
• Using “Action Mazes” (“choose your own adventure” style branching narratives, where students experience consequences of decisions).