How to Uphold Integrity in Online Assessment Design

Online learning environments are a new realm for many educators–and thus an opportunity for innovation and improvement. That said, anything new can be daunting.

Integrity is not just about preventing plagiarism or academic misconduct but also about upholding best pedagogical practices to enhance learning and teaching efficacy. Nowhere is this more necessary than in assessment, where data exchange between students (feedback to contribute to a growth mindset) and instructors (data insights into student learning) occur.

What do educators need to consider as they begin online assessment design?

According to Inside Higher Education, just like during traditional in-person learning, instructors must consider the purpose of an exam when creating an online assessment. Will it be formative or summative? Will it assess the memorisation of facts, the understanding of concepts, or the demonstration of skills? These questions inform assessments that are equitable and accurate.

Other questions include learning criteria; what is being taught and will your exam evaluate student understanding of what is taught? When learning criteria is clear, it is more straightforward to assess student understanding of concepts, and students, in turn, recognise what it is they need to know.

There are multiple types of assessments to achieve the goals above, according to Thomas R. Guskey who details three categories:

• Product criteria: Summative assessment is useful for gaining insights across a broad range of concepts in a shorter period of time. This kind of assessment may not test higher-order thinking–and the emphasis is on a numerical grade. An example of this is standardised testing or multiple-choice final exams.
• Process criteria: Because product criteria may not provide insights into deep understanding of concepts, this form of formative assessment provides students with opportunities to display how they arrived at their responses. Low-stakes quizzes, assignments, and class participation fall under this category.
• Progress criteria: Another way to gain insights into student learning is to measure what is called “learning gain;” how much has the student learned throughout the course? While other forms of assessment may have firm expectations and criteria, this one is more flexible and individualised (Guskey, 1994).
  
  

None of the above forms are perfect or one-size-fits-all. But they are each useful in measuring different components of learning.

So how do educators uphold integrity when designing online assessments?

Once the purpose of the online assessment has been determined, educators should provide variety in assessment formats to address different learning styles and measure different facets of learning. Providing multiple access points for assessment promotes a sense of belonging and equity for different learning styles–and doing so reduces the risk of academic misconduct. Multiple-choice exams, for example, are a convenient way to test a broad range of facts. When supplemented with short-answer or long-form assessments, students have multiple ways to demonstrate their knowledge and educators have a more complete picture of student learning.

However, providing a variety of assessment formats in online learning environments may feel challenging–how can educators build, deliver, and grade short answer assessments from afar? Feedback and assessment software like Gradescope enables educators to adapt their existing assessment for remote learning in a variety of ways, including delivering fully-online, student-uploaded paper-based, or hybrid assessments.

Accurate assessment and integrity can be upheld by asking students to explain their approach, logic, or thinking. This can involve short-answer items, written work, annotated portfolios or recorded and annotated presentations to ask students to talk through how they came to an answer. By promoting higher-order thinking in assessment, educators can improve student reflection on course content, which reduces opportunities for potential misconduct and supports deeper understanding.
 

Equity and integrity also mean designing assessments without bias. Mitigate misconduct by designing assessments focused on student knowledge (UC Berkeley, 2020) and consider eliminating components that do not inform learning. As stated above, this means designing assessments that measure higher-order thinking. This also means not grading on a curve, which can increase competition between students and a sense of unfairness.

To that end, using a rubric is helpful because it not only clarifies expectations for students, it also increases consistency and decreases time spent grading. Gradescope, too, can help ease the burden on instructors by helping teams of graders build and apply one aligned grading standard for all students.

More frequent, low-stakes assessments uphold integrity by providing more insights into student learning and allowing for instructor intervention if necessary. Consider providing a greater number of quizzes and assignments as well as exams to promote knowledge retention and mitigate the high-stakes, high-stress scenario of one to three exams in a course. This may also ease student anxiety and reduce student temptation to cheat when assessments are high-stakes and few. While it may be daunting to consider more work grading assignments, quizzes, and exams, grading software like Gradescope can streamline workflows and save time while promoting best pedagogical assessment practices.

Finally, item analysis is a significant thoroughfare through which educators can inform future exam design. By conducting item analysis or, in other words, formally examining student responses and patterns, instructors can pinpoint whether or not assessments are accurately assessing student knowledge. Did every student get one particular question wrong? Did every student get one particular question correct? What kinds of answers are your test questions eliciting? Did you get the answers you expected? These data points and more can help educators adjust and fine tune future exam questions. In online learning environments, item analysis is particularly useful because it is a rich data source. Gradescope too, provides item analysis insights for each assessment.

When designing online assessments, consider what may be the most appropriate assessment to use for your course content and the online context that supports student learning and academic integrity. Because ultimately, the goal of assessment with integrity is to support student learning and teaching efficacy.

Jason Gibson and Jamie Whitehead will be presenting Navigating Challenges in Remote Assessment with Gradescope at OEB20 on Tuesday, December 1, 2020 11:30 AM to 12:00 PM