Assessment in a Different Light

How the pandemic forced us to rethink the concept of testing

The pandemic necessitated time- and location-independent testing, which at the same time, brought several challenges to light. Within institution assessment programs, assessment policy and assessment formats had to be adapted to the new situation. In the past year, Dutch higher education institutions have been looking for solutions for remote testing in various ways. In order to tackle the emerging questions, the Working Group Remote Assessment has been set up at national level (see box below). This working group has investigated the most important bottlenecks and developments in the Netherlands and is working on solutions at national level.

Short-term solutions

The pandemic made it particularly difficult for institutions to maintain their regular program of summative assessments. The choice was either to test differently or to administer tests via online proctoring or postponement. But the latter is certainly no longer an option given the long duration of the pandemic. Research by the Remote Testing working group shows that some teachers were reasonably able to cope with converting tests to another format, but a substantial part of the teachers struggled with this question. They realised that there is a lot of knowledge in this field, but that knowledge is not easy to find and apply.

The working group collected the knowledge and developed an online tool that helps teachers to choose alternative forms of testing. The working group is happy to share this public tool during OEB Global.

Redesigning the assessment program

The pandemic created room for the question: is the way in which we set up our testing and assessment programs right? Why do we take so many summative (knowledge) tests? Can’t we come up with much better ways to assess students?

For years now, the Netherlands has had a number of strong advocates for organising assessment differently, such as Cees van der Vleuten and Dominique Sluijsmans.

Cees van der Vleuten has developed the concept of programmatic assessment, in which he argues for more small measurement moments, so that at the end of the day you have built up a very reliable picture of what the student can actually do.

Dominique Sluijsmans emphasizes that we need to move much more towards a system of evaluation, instead of assessing and giving marks. Constructive and timely feedback plays an important role in this evaluation system. This vision calls for a rearrangement of the assessment process, for example the use of (peer) feedback tools, digital portfolios and systems for submitting and assessing papers.

In the autumn, the working group will, together with higher education institutions, make an inventory of the assessment scenarios that Dutch higher education expects in the coming years and what this means for the use of software and tools, among other things.

Item banks

With a different design of the testing program, the wish also emerged to be able to focus more on formative testing, assessment for learning. To make this possible, item banks are needed with a large quantity of high-quality questions. The working group therefore invested in the online manual: to an item bank in 5 steps.

Developing, managing and maintaining item banks requires good project management. With this publication we want to show how to set up a digital system to exchange items with colleagues from inside and outside the institution. In this handbook we bundle the knowledge and expertise that institutions, partnerships of institutions and SURF (the collaborative organisation for ICT in Dutch education and research) have acquired on this subject. It helps institutions that want to take advantage of digital item banks. The manual consists of a concrete step-by-step plan and an in-depth part, with references to theory and literature.

The manual focuses primarily on the development, management and use of item banks within a study program or within one institution. Because there is much to be gained if different institutions within the same knowledge domain start working together on item banks, the book also offers a look at what is needed to start collaboration across institutions or nationally.

Online proctoring

Some of the institutions have opted to use online proctoring: remote monitoring of students via the camera. In principle, institutions were always very reluctant to use online proctoring, especially because it is privacy-invasive. SURF already wrote a white paper on the advantages and disadvantages of online proctoring in 2016, in which the pros and cons of online proctoring were discussed in detail. An update of this white paper was immediately published at the start of the pandemic.

An important change was that, in the context of the pandemic, the choice to use this form of testing could fall under the legal term ‘legitimate interest’. This does not alter the fact that the use of online proctoring in the Netherlands has caused quite a stir. Student unions rose up and questions were raised about the need for online proctoring as far as the Dutch parliament. The current trend is that online proctoring should really only be used in very exceptional cases. And what do those exceptional cases look like? The working group will discuss this with the student unions in the last quarter of this year.

In the meantime, it has become clear that online proctoring is a valuable addition to the range of options for administering tests. Online proctoring offers a solution for specific target groups, such as top athletes, students abroad, sick students or students with a disability. At the same time, there are still many questions within the institutions about how to use online proctoring effectively. That is why the Working group Remote Assessment set up a national user group together with SURF in which 100 members now share knowledge with each other.

Summarising: the pandemic has disrupted entrenched patterns around (digital) testing. It has led the way to giving more attention to other forms of testing, the development of item banks and the (careful) use of online proctoring. The Remote Testing working group is happy to share its knowledge and experiences about the experiences and lessons learned in Dutch HE on OEB!

About the working group Remote Assessment

The COVID-19 crisis showed how resilient Dutch higher education can be in terms of online education, but also where the pain points lie. The pandemic necessitated time- and location-independent assessments, and this brought several challenges to light. For this reason, the working group Remote Assessment was set up at the end of 2020 as part of the National Acceleration Plan Educational Innovation with ICT (a collaboration of Dutch HE organisations and associations). The working group focuses on identifying issues, actively sharing new insights, initiating solutions, developing practical tools and combining the forces of existing networks. We look forward to sharing our experiences at OEB and are curious about experiences in other countries.

This article was written for OEB21 by members of the the working group, which consists of: Marjoleine Dobbelaer, Educational researcher at HAN University of Applied Sciences; Sharon Klinkenberg, Senior Lecturer University of Amsterdam, and Chair National Special interest group on Digital Assessment and; Annette Peet, Project manager at SURF (the collaborative organisation for ICT in Dutch education and research).