Plan your assessment

Planning Your Online Assessment

When planning an online assessment, several key factors must be considered to ensure it aligns with your educational goals and provides a fair evaluation of student learning.

Purpose and Goals: Clearly define what you want to achieve with the assessment.
Assessment and Question Types: Choose types that align with your goals and are appropriate for the stakes.
Feedback: Decide on the timing and type of feedback to provide.
Time Limits: Set reasonable limits to ensure fairness.
Security: Consider using tools like Questionmark Secure for high-stakes assessments.
Accessibility: Ensure all students can access and complete the assessment.
Legal and Ethical Considerations: Be aware of the legal implications and ensure fairness.

Purpose and Goals

First, determine the purpose of your assessment. Is it a diagnostic test to identify students’ current knowledge and skills, a weekly formative evaluation to monitor ongoing progress, or a summative end-of-term exam to evaluate overall learning? Aligning the purpose with your learning outcomes and course objectives is crucial for creating an effective assessment.

Examples:

Diagnostic Test: Used to identify students’ current knowledge and skills.
Weekly Assessment: Regular checks to monitor ongoing progress.
End of Term Exam: Summative assessment to evaluate overall learning.

Assessment Types

Questionmark Perception (QMP) offers four types of assessments that you can choose from when creating your assessment: quiz, test, exam, and survey.

Understanding Assessment Stakes

Assessments can be categorized by their stakes, which refer to the consequences for the candidate. High-stakes assessments, such as final exams, have significant consequences and require rigorous security measures. Medium-stakes assessments, like mid-term tests, influence course grades but are not final. Low-stakes assessments, such as quizzes and surveys, have minimal consequences and are often used for formative purposes.

Before examining how assessments can most effectively be used in the learning process, it’s also important to understand that the various types of assessments can be categorized in terms of high, medium or low stakes. The level of an assessment’s stakes refers to the consequences to the candidate. For example, an exam normally has a higher consequence, while a survey has low or even no consequence. In low-stakes assessments such as quizzes and surveys, the consequences to the candidate are low, and so the legal liabilities are low.

The table below was adapted from this QMP Whitepaper from 2004 - https://www.cedma-europe.org/newsletter%20articles/misc/qmark.pdf

Category	Low	Medium	High
Consequences	Few	Some	Major
Decisions	Few and easily reversed	Can be reversed	Very difficult to reverse
Options for participant	Refine studies	Pass, fail or work harder	Pass or fail
Motivation to cheat	Low	Medium	High
ID individual	Not important	Potentially important	Always and constant
Proctoring required	No	Sometimes	Always and constant
Development effort	Minor	Medium	Major
Check reliability and validity	Rarely	Subject Matter Expert	Psychometrician / Subject Matter Specialist

Assessment Affordances

The table below includes details about what functionality is available for each assessment type:

Assessment type	Record answers	Provide feedback	Open access	Time limit	Use Questionmark Secure	Anonymous results
Quiz	Optional	Optional	Optional	Optional	No	No
Test	Yes	Optional	Optional	Optional	Optional	No
Exam	Yes	No	No	Optional	Optional	No
Survey	Yes	Optional	Optional	Optional	No	Optional

Quiz Test Exam Survey

Question Types

Choosing the right question types is vital for assessing different cognitive levels. Consider using multiple-choice questions for quick grading, essays for evaluating critical thinking and writing skills, true/false questions for straightforward assessments, matching questions for testing associations, and hotspot questions for interactive elements. Ensure a balance of question types to cover recall, application, and critical thinking skills.

Commonly used question types:

Multiple Choice: Easy to grade, but may not assess deep understanding.
Short Answer: Allows for more detailed responses, but harder to grade.
Essay: Assesses critical thinking and writing skills, but time-consuming to grade.
True/False: Quick to answer, but may encourage guessing.

Full list of question types

Design Feedback and Links to Learning

Decide on the type of feedback you will provide. Immediate feedback can help students learn from their mistakes right away, while delayed feedback allows for reflection before receiving input. You can choose between generic feedback, which applies to all students, or personalized feedback tailored to individual responses. Additionally, link feedback to further study resources such as reading materials, videos, or related modules to enhance learning.

Immediate Feedback: Helps students learn from mistakes right away.
Delayed Feedback: Allows for reflection before receiving feedback.
Links to Resources: Provide additional learning materials for further study.

Determine Time Limits and Conditions

Set appropriate time constraints based on the complexity and length of the assessment. This ensures that students have a fair amount of time to complete the assessment without feeling rushed. Plan for accessibility accommodations to support all students, including those with special needs. Consider implementing conditional assessment pathways, where additional sections are unlocked based on student performance, to provide a more tailored assessment experience.

Time Limits: Ensure assessments are completed within a reasonable timeframe.
Conditional Assessments: Adapt the difficulty based on student performance.
Allow buffer time: Allow additional time after the end to allow people to have a staggered start and finish i.e. open at 0900, end at 1110, but allow 120 minutes for the assessment.
Reasonable adjustments

Security

Ensuring the integrity of high-stakes assessments is critical in maintaining academic standards. The Questionmark Secure browser can help by creating a controlled testing environment where students are unable to access other applications, websites, or resources during testing sessions. This can introduce additional support requirements and is best used for in-person (online) exams with staff present. For additional security, consider randomising questions and answers, as well as incorporating application-based or open-ended questions that are less susceptible to cheating. When practicable, monitor the assessment using proctoring solutions, whether remote or in-person, and establish clear guidelines for students on acceptable behavior during testing to deter misconduct.

Accessibility

Online assessments must be accessible to all students, including those with disabilities, to ensure equal opportunities. Check all images have alt text and are understandable without relying on colour, for example. Check the questions with a screen reader to confirm that the text can be read aloud automatically and understood. Offer practice tests to familiarise students with the platform and ensure they can request accommodations, such as extended time or alternative question formats, in advance of testing.

Legal and Ethical Considerations

Designing and delivering assessments requires adherence to both legal standards and institutional academic integrity policies. Take steps to guarantee fairness, such as providing clear instructions, offering accommodations for disabilities, and avoiding cultural or linguistic biases in questions. Be transparent about how the assessment will be monitored and scored, and address student concerns promptly to maintain trust and uphold ethical standards.