Goal

To describe recommended practices for designing and administering assessments in courses with remote or flexible delivery. 

Overview

Assessments generally serve a few major purposes in teaching and learning–to provide both students and faculty with feedback on whether learning goals are being met and accountability. They are typically categorized as either formative or summative. Formative assessments are low-stakes and informal, and administered while learning is still occurring. Summative assessments are high-stakes and performed after learning occurs typically for a grade. Within the literature, it is widely established that formative assessments are more impactful for student learning than summative assessments (Black & Wiliam,1998; Black et al., 2004). Regardless of type, all assessments designed for a course should align with learning outcomes and learning activities, following the basic principles of backward design. Students’ specific accommodations should also always be taken into account in the design and administration. The Office of Accessibility Services is a campus resource that can be contacted with any questions regarding student accommodations for assessments. 

For instructors accustomed to teaching face-to-face courses, teaching in online and hybrid-flexible environments can require reimagining how formative and summative assessments are performed. However, following a few key principles, and weighing various options, online assessments can be accurate and reliable measures of student learning. Ultimately, instructors must consider the advantages and disadvantages of each type of assessment to make the best choices for their course. 

Principle #1 – Use more formative, low-stakes assessments than high-stakes assessments in a course. 

As described previously not only have formative assessments been shown to be more effective in helping students learn, they are also less vulnerable to academic integrity issues compared to high-stakes assessments.  Here are several examples of formative assessments shared during a Course Design Institute by instructors preparing to teach face-to-face, fully online, remote or hybrid-flexible courses. These include items such as ungraded Moodle quizzes, polling through platforms like PollEverywhere, and peer feedback on drafts. If an instructor prefers to implement traditional, graded assessments, they can consider administering several smaller quizzes, and a traditional, comprehensive high-stakes test if there is a pedagogical rationale. This will allow the students will complete continuous assessments to support their learning of course material. 

Principle #2 – In designing high-stakes assessments, opt first for authentic assessments. 

Authentic assessments encourage deeper more meaningful learning experiences for students. They often have unique elements that allow students to produce original work, and carry out tasks that have real-world applications. Examples include podcasts, student portfolios, opt-Eds, letters to the editor, blogs, wikis, brochures, infographics, and presentations to stakeholders. Because of their originality authentic assessments are also less vulnerable to academic integrity issues. Further they typically can be submitted or viewed digitally. The Authentic Assessment Toolbox has several additional examples. Such assessments can be graded using analytic or holistic rubrics

Other Assessment Types

Take Home Assessments – If more traditional high-stakes assessments (e.g. graded quizzes, exams) are important for the course and fit better within instructor comfort level, then consider making them open-book, open-notes, open-internet. 

These work better for courses with learning objectives that operate at levels of Bloom’s Taxonomy higher than “remembering” or recalling information (Bengtsson, 2019). A major advantage of take-home assessments is that they can reduce student anxiety over testing. They can also be more accessible to students with accommodations, and assess higher-order thinking skills.  Instructors should encourage students to study for the take-home exam as a typical exam so that they fully prepare. A disadvantage is that there can be concerns around unethical behavior in completing take-home exams. A few recommendations are included below for designing take-home exams as remedies for such concerns (p. 9). 

Recommendations 

  • Design & Implementation: 
    • Include primarily open-ended questions, not multiple choice questions. 
    • Design questions that require students to have a comprehensive understanding of course material.
    • Ensure questions are highly contextualized.
    • Have students apply their knowledge in new contexts.
    • Ask students to justify all of their answers.
    • Have students agree to a statement such as: “I have never given nor received aid on this exam” and include a link to the College Academic Integrity Statement.  
    • Narrow the timeframe to complete the test. 
    • Scramble the order of questions at random.
  • Grading
    • Require answers to provide direct references to course material.
    • Subtract points if students do not include references.
  • Instructions to Students
    • Give explicit instructions about how students should take the exam. Here is an example for a face-to-face course that can be adapted to an online environment and contextualized to Lafayette: Example.

Oral Assessments – If there are concerns about designing take-home exams fairly and student collaboration, and there are a manageable number of students in the course, consider implementing oral exams. 

Oral assessments can provide a good sense of how much students have learned. A caveat is the time investment for conducting oral exams, however grading can be finished as soon as the last student completes their oral exam if the process is carefully designed. 

Recommendations for Implementation:

  • Oral exams can be scheduled online as one-on-one sessions (e.g. 15 – 20 minutes) for students on Zoom. Students can be admitted into the main room from the waiting room during their scheduled time and moved back in when they are finished with their exam. Sessions can be recorded for the faculty member’s usage so that they can be returned to later by the faculty member if needed for review.  Some faculty may choose to perform group exams where each student takes a turn being the primary one to answer various questions. 
  • During oral exams students can be asked questions about particular course concepts and then probed for deeper understanding. Some faculty may choose to provide students with general prompts in advance of the exam and either indicate they will ask all questions or randomly choose a few by rolling a die. 
  • Prior to the exam the faculty member can create an answer key/rubric that can be used to score students during the assessment. 

Unproctored, Traditional Assessments – If neither of the options above seem comfortable or feasible for a course, design an unproctored assessment with a few simple strategies as noted below. 

The advantages of these assessments is that because they are typically traditional in format they may feel similar to an on-ground exam. The disadvantages are that they can sometimes cause students to have more anxiety, and students and faculty may have some concerns around academic integrity. 

Recommendations for Design and Implementation

  • Incorporate more open-ended questions.
  • Use multiple forms of a test by randomizing questions and answers.
  • Administer the exam to the entire class at a scheduled time.
  • Limit the amount of time that students have to complete the exam.   
  • Ensure that questions are presented one at a time. 
  • Have students agree to the statement, “I have neither given nor received aid on this exam” and include a link to the College Academic Integrity Statement

Timed, Proctored, Online Traditional Assessment – If it is most comfortable and feasible to hold a traditional proctored assessment, then this can be carried out following some basic principles. 

These assessments require more effort for instructors and students to perform in an online setting. Students should be encouraged to contact the Help Desk at help@lafayette.edu if they don’t have access to a laptop with a webcam or reliable internet. A few minutes should be devoted prior to class to allow students to practice taking the exam, review procedures, and troubleshoot issues. Instructors should be aware that students may choose to turn off their webcams and may feel using them is an invasion of their privacy. There have been a number of reported concerns on this issue

Paper and Pencil Tests (e.g., tests involving problem solving)

  • Using a Moodle Assignment, students can upload pictures of their answers or they could use a scanning app (e.g., JotNot, Evernote Scannable) on their phone to upload multiple pages as a single PDF. Instructors can configure the assignment to remain hidden until a specific time and day, or unhide it manually. It’s also possible to enable a cut-off time, after which students would be unable to submit anything, though instructors may consider adding extra time for students to complete the upload process. Within the Assignment, an instructor might also consider including the exam as an attachment (PDF, Word, etc.) for students to download.
  • To observe students taking the exam, instructors can schedule a Zoom or Google Meet session during the exam period asking students to arrive a few minutes early. For Zoom, set chat permissions so students can only chat with the host (i.e., the instructor). 

Online Tests 

  • Design the assessment using a Moodle Quiz, making it available to students at a date and time designated by the instructor. Also, Test time can be extended for students with accommodations. Start times in Moodle are based on Eastern time, so if an exam is scheduled to start at 9:00 a.m. EDT, a student on the West Coast would be required to start at 6:00 a.m PDT.
  • To observe students taking the exam, instructors can schedule a Zoom or Google Meet session during the exam period asking students to arrive a few minutes early. For Zoom, set chat permissions so that students can only chat with the host (i.e., the instructor).

References 

Bengtsson, L. (2019). Take-home exams in higher education: A systematic review. Education Sciences, 9, 267.

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the Black Box: Assessment for Learning in the Classroom. Phi Delta Kappan, 86(1), 8–21. https://doi.org/10.1177/003172170408600105