6
In this section we’ll focus on designing assessments that are appropriately aligned with the learning objectives and instructional activities we reviewed in the previous section.
Quality Matters Assessment and Measurement
In Module 1, we introduced the Quality Matters Standards for online course design. These standards emphasize alignment between course and module learning objectives, assessments, and the learning objectives. The third General Standard in the Quality Matters Rubric focuses on Assessment and Measurement. There are five specific standards in this category.
QM General Standard 3 – Assessment and Measurement
3.1 The assessments measure the stated learning objectives or competencies.
3.2 The course grading policy is stated clearly.
3.3 Specific and descriptive criteria are provided for the evaluation of learners’ work and are tied to the course grading policy.
3.4 The assessment instruments selected are sequenced, varied, and suited to the learner work being assessed.
3.5 The course provides learners with multiple opportunities to track their learning progress.
You should keep these standards in mind as you make decisions about the general types of assessments you will use.
Purposes of Assessment
An assessment will typically fulfill one of two general purposes.
- Formative assessments are designed to help you receive and give feedback that will promote further student learning (e.g., a low-stakes reading quiz that gives students practice in recalling key concepts).
- Summative assessments allow students to demonstrate their achievement of the learning objectives at the end of the learning process (e.g., a high-stakes cumulative final exam overall course content). The following table further highlights the differences between formative and summative assessment:
INSERT TABLE
| Factors for Comparison | Formative Assessment | Summative Assessment |
|---|---|---|
| Purpose | To gauge and improve quality of student learning | To provide evidence for evaluating or grading students |
| Characteristics |
|
|
| Emphasis | Giving students useful feedback for improvement | Grading |
| Examples |
|
|
A Note about Time
Please note that with formative assessments, instructors typically seek to maximize the amount of immediate feedback to students while minimizing the effort involved in grading the assessments. For example, self-checks and selected response quizzes can be automatically graded, and students receive immediate feedback. Also, in the case of peer reviews, students can provide one another with immediate feedback during the instructional activity. However, with summative assessments, instructors will often spend more time grading the assessment and writing delayed feedback that students will receive several days after completing the assessment.
To calculate the amount of time you might spend grading, think about the average number of students you will have in your class. Then think about the average amount of time you want to spend per week (a) reviewing and grading these assessments and (b) providing written or verbal feedback to all of your students. The total amount of time instructors spend per week teaching a course depends upon the instructor’s role within the department and the expectations of the department for the percentage of time the instructor is expected to devote to teaching. However, a general rule of thumb is that for a three-credit, fifteen-week course, an instructor would spend no more than eight hours per week teaching a course.
It is important for you to think about your time constraints now because you may not want to use a particular assessment option if it will take you too much time to effectively carry out the assessment. For example, if you have a class of 25 students, it may be possible for you to use 5-minute oral exams to test the depth of student understanding of course content, as it would take a little over two hours to conduct the assessment. However, if you’ve got 150 students in your course, a 5-minute oral exam for each student would take more than 12 hours. Consequently, the oral exam would not be a regular, weekly form of assessment in your course, as it takes too much time to administer it.
From the students’ perspective, think about the average amount of time per week that you would like students to spend working on both formative assessments (e.g., reading quizzes, homework assignments, etc.) and summative assessments (e.g., cumulative exams, essays, etc.). While the amount of time students spend working on the course is defined by your institution’s policy, it is still up to you to determine how much of that time students will spend working on assessments.
In addition, it should be noted that sometimes a formative assessment may be considered to be a type of learning activity. For example, if you ask students to submit their answers to homework questions, that might be considered to be both an assessment and a learning activity. Likewise, if you grade student participation in group discussion, this could be considered to be both an assessment and a learning activity. For practical purposes, it is often necessary to attach points or some other consequence to a learning activity to increase student motivation to do the work. In such cases, the line between assessment and learning activity becomes blurred. But don’t worry about that. For now, just think about the assessments you want to use to measure student progress.
Types of Assessments
After you determine the purpose of the assessment (formative or summative), you will next want to decide which type of assessment will provide you with the information you want about student achievement. Richard Stiggins (2013) offers a simple framework for thinking about four main assessment options, as outlined in the table below.
INSERT TABLE
We encourage you to think about all of the above options as you design assessments for your learning objectives. As you will see throughout this module, incorporating a variety of assessment methods into your course is a “best practice” for several reasons. First, coming at assessment from multiple perspectives helps to ensure a more accurate, fair sampling of student performance. Second, multiple assessment methods can provide a good mix of immediate and delayed feedback to students every week. Finally, students have the opportunity to repeatedly recall and apply what they have learned in different ways, which promotes long-term retention of knowledge and skills. If you have any questions about which types of assessment will work best for you and your students, feel free to talk with your instructional design consultant.
Tools to Deliver Your Assessments
A wide variety of tools may be used to assess student learning in an online course. The following tools are available as part of the Blackboard learning management system. Other learning management systems such as Canvas and Moodle will have a similar set of tools:
Tests: Question types include multiple-choice, multiple-answer, true-false, matching, short-answer, essay, etc.
Surveys: All question types, including Likert-scale items.
Assignments: Work may be submitted directly in the text box or as an attached document.
Discussion Board: Posts to threads may be graded by the instructor or rated by other students.
Blogs: Blog entries may be graded by the instructor.
Journals: Journal entries may be graded by the instructor.
Wikis: Wiki entries (also known as “saves”) may be graded by the instructor.
Blackboard Collaborate Ultra (a web conferencing tool): Instructors may use this to conduct oral exams or watch students perform particular tasks.
Other tools are readily available outside of the learning management system, and they may be used in whatever way works best for you and the students. These tools include the following:
Google Meet (formerly Google Hangouts): Instructors may use this web conferencing tool to conduct oral exams or watch students perform particular tasks.
Google Docs, Sheets, and Slides: Instructors can see which students have edited a document or submitted comments.
Email: Still one of the easiest ways to have students send you assessment results that you want to check and return.
Audio/Video Recordings: Students may use various audio/video recording programs to record their performance of a task or their response to a test question (Jing, YouTube, etc.).
Publisher-Based Assessments: Many textbook publishers provide online assessments that students may access on the publisher’s own learning management system.
There are other tools that you may employ in your course, but this sampling gives you a good idea of the many options available to you. Your instructional design consultant can work with you to find any additional tools you need.
Issues and Problems With Online Assessments
As you design your assessments, a number of potential issues or problems may arise. Most of these problems exist with face-to-face courses as well, so they are inherent in the assessment process and do not arise solely from the use of online instructional technologies. However, online assessment tools may introduce new aspects of these problems that you have not encountered before. Here are a few of the challenges you may face.
Authentication of Student Identity: It can be difficult to verify that online students are who they say they are.
Cheating: Students can use a variety of technologies to capture and share questions and answers before, during, and after an assessment.
Technical Difficulties: Due to the delivery mode of online learning, occasional technical difficulties can occur for both students and instructors.
Plagiarism: Because it is so easy to copy and paste text, it may be tempting for students to copy the work of others and try to pass it off as their own. Also, there are web services that will write papers for students–for a hefty fee.
Use of Proctoring: Many faculty would like to have students take proctored exams. The Associate Coordinator of Remote/Online Testing at Boise State University offers online proctoring services for all online courses. You can contact the Associate Coordinator of Remote/Online Testing at Testing Center.
Effective Solutions for Online Assessments
The problems listed above are solvable, but they may require you to use new assessment strategies or tools. The following solutions can reduce or eliminate a variety of assessment problems. These solutions are gleaned from the resources listed at the bottom of this page. If you would like to explore these solutions in more detail, feel free to read the original articles. Also, you may talk with your instructional design consultant about any assessment problems related to your course.
Use a Variety of Assessment Types
By using a variety of assessments, you not only get a more holistic picture of student performance, you also discourage cheating and plagiarism. Strategies that students might use to cheat or plagiarize on one type of assessment may not work on other types of assessments.
Use Low-Stakes Assessments to Track Progress Over Time
Require students to complete a number of low-stakes quizzes or assignments before they take a major exam or paper. For example, you may have students complete a series of short practice tests in preparation for a major exam. You may then be able to identify possible instances of cheating and plagiarism by comparing their performance on the low-stakes assessments with their performance on the major exam or paper. For example, if a student bombed all of the practice tests but aced the exam, this may signal a possible cheating problem. For essays, you may require the students to turn in multiple, ungraded drafts of their paper. This not only helps reduce plagiarism, it also helps the students learn to use multiple revisions to improve their writing.
Regularly Change Assessment Questions and Assignment Topics
One common strategy for reducing cheating on multiple-choice tests is to set up the tests so that a small number of questions (e.g., 15-20 questions) are drawn at random from a large question pool (e.g., 100+ questions). This makes it much harder for students to share answers. Also, students will be less likely to plagiarize if you assign new paper topics each semester, or even assign different topics to different students for the same essay assignment.
Give Timed, Open-Book Tests
By assuming that students will have access to the textbook and other information sources during the test, you will design the assessments differently. They will become assessments FOR learning, rather than assessments OF learning. If students have a limited amount of time to take the test, they will be more motivated to do the assigned coursework prior to the test. This is because they simply won’t have enough time to look up all the answers. They’ll need to already know the content well in order to finish the test on time.
Give Proctored Exams
There are several online proctoring services that you may use to have students take a test online. These web-based services use technologies that authenticate student identity and ensure the students don’t have other programs or browser windows open as they take the test. In addition, you may require students to take a test in a local proctored testing center. TheTesting Center at Boise State provides detailed information about proctored exams on the following web site:
You may also talk with your instructional design consultant about the proctoring options that would best serve you and your students.
Use Technologies to Detect Plagiarism
There are a number of technologies that you may use to detect plagiarism. One simple way is to copy a sentence or two from the essay and paste them into a Google search. If the results show web pages with the same sentences word-for-word, you may have a problem with plagiarism. (Of course, if the student wrote these other pages–such as a Wikipedia article–citations are required just as they are with any other source material.) Other technologies have been created to detect possible instances of plagiarism. For example, the Blackboard learning management system used by Boise State allows students to submit a “SafeAssignment” that automatically checks the paper to see if there are any indications of plagiarism. The program compares the student’s paper with the contents of papers that have been submitted by other students. If there are any duplications in the text, the student’s paper will be flagged as a possible instance of plagiarism.
Anticipate Technical Problems in Your Assessment Directions
If you think that technical problems might occur for a particular assessment, let the students know about them ahead of time. Identify possible problems in the assessment directions, and tell the students what steps they may take to deal with them. One helpful tip is to allow students more than one opportunity to complete the assessment, but explain to students they should only take it a second time if they have a technical problem. Require these students to contact the Boise State Help Desk to troubleshoot the problem first before they email you with a description of what the problem was and how they tried to solve it before taking the test the second time. It is also a good idea to set the due date and time of the assessment one hour before the Help Desk closes. That way, the Help Desk can actually be of help to the students, and you won’t get as many emails from panicked, sleep-deprived students at 12:01 AM Sunday morning.
Provide Clear Expectations of Academic Integrity
Boise State Office of the Dean of Students recommends that an academic integrity statement be included on all syllabi, although one is not required. A recommended statement is included in the syllabus template shared with you in this seminar. In addition, we recommend that you provide clear, complete, concise and easy to read directions that outline specific expectations for exams, projects, discussions and writing assignments within the individual activity directions:
- Allotted time frame
- Approved materials students are allowed to use (calculators, books, notes, web sources, formulas, dictionary, etc.) and/or
- Unapproved materials (closed book, no notes, no web sources, four-function calculator only, no cell phones or electronic devices, etc.)
- Academic integrity guidelines for group work, collaboration, or assistance from others
- Assignment-specific citation requirements
- Reusing work previously created for any class
We suggest you write expectations in positive terms, e.g.: “Should you find inspiration in others ideas or writing on this subject, be sure to cite them so I can understand which ideas and writing are your original work and what sources you used.” not, “Failure to cite your sources is considered academic misconduct.”
We also suggest that you write expectations that tell students what they should do and why these are your expectations, rather than what they should not do.
Designing Effective Individual or Group Assessments
A complete listing of all the steps involved in creating every type of assessment is beyond the scope of this seminar. Your instructional design consultant will be able to provide any information or assistance you need to design and develop the different assessments for your course. As you work together on the design of those assessments, you will learn more about how to build effective assessments. However, before you begin building any assessments, there are some general questions that should be considered if you want to ensure that your assessments are effective–that they promote learning and accurately measure student achievement. Rick Stiggins (2013) has identified five keys to the creation of productive assessments in college courses. As you read about these keys and their accompanying questions, think about how you would apply these keys to all of your assessments.
Key #1: Clear Purpose
Both you and your students should know who will use the results and how. The following questions can help clarify the purpose of the assessment.
- Why is this assessment being given?
- Who will use the results?
- How will the results be used?
- How will students use the results?
Key #2: Clear Learning Targets
As previously stated in this module, clearly stated learning objectives are essential to the design of appropriate testing procedures and assessment items. The following questions help to ensure that you have clear learning objectives for the assessment.
- What are the learning objectives?
- Are they clearly stated?
- Are the learning objectives appropriate for this type of assessment?
- Do students understand the learning objectives?
Key #3: Quality Assessment
Once you know why the assessment is being given (Key #1) and what will be tested (Key #2), you can turn to questions about your methods of assessment–the general types of assessments you will use, the tools used to administer the assessment, and the solutions you have chosen to address possible assessment problems. It is also important that you have an adequate sampling of the students learning. You need to have a sufficient number of clearly written and relevant questions to accurately measure the depth and breadth of student knowledge and skill. In addition, the methods should reduce or eliminate possible sources of bias that would decrease confidence in the validity and reliability of the assessment results. Bias can be avoided by building student confidence in test taking and the knowledge being tested (through sufficient practice and feedback), providing clearly worded directions and questions, allowing adequate time for reading, evaluating appropriate criteria, providing one correct or best response, and avoiding grammatical and cultural cues in the question stem or answers.
- What method will be used?
- Are the questions of high quality?
- Is sampling of the students’ knowledge and skills adequate?
- Are sources of bias avoided?
- Will students self-assess?
Key #4: Effective Communication of the Results
In order to fulfill the assessment’s purpose, it is imperative that the results are communicated in a timely manner to the users. Your answers to the following questions will help you form a solid communication plan.
- How will grading take place?
- How will feedback be provided to students?
- How will you use the information to plan the next steps for the students, or to plan revisions to your instructional strategies and materials?
- How will students let you know the results have been successfully communicated?
Key #5: Student-Involved Assessment During Learning
If you want the assessments to impact student learning, you must involve them in assessing their own progress.
- How will you make the learning targets (learning objectives) clear to students?
- How will you involve students in assessment development (as appropriate)?
- How will students set goals for learning and then assess and track their progress?
- How will students communicate with you and with each other about their learning?
If you can answer these questions for each type of assessment in your course, you can be confident that your assessments will be effective. The key in all of this is to make sure you know how both you and your students will use the feedback from the assessments. The feedback should be designed to help the students to know (a) where they are now, (b) where they need to be, and (b) how to close the gap (Hattie and Yates, 2014). After receiving feedback from these assessments, students should have a better idea of the next steps they need to take and how to take them.
Revise the grading schema so that it contains the letter grades and percentages you prefer to use for assigning grades. If you like the grading schema example, feel free to adopt it as is.
- Anything else on point structure?
Next Steps
If you would like to further explore issues related to online assessment, you may select from the resources listed below for more in-depth information.