Learning analytics offers big opportunities

Carlos Delgado Kloos is Professor of Telematics Engineering at the Universidad Carlos III de Madrid (UC3M) and holder of that university’s UNESCO Chair on “Scalable Digital Education for All”. His work as a professor, researcher and as Vice-President for Strategy and Digital Education have naturally drawn him to the edTech sector, which he considers “a very exciting field”. He also coordinates eMadrid, a network of excellence around educational technology around the Spanish Capital. Four of its members are to speak at the upcoming OEB in a session on analytics, so the News Portal caught up with Professor Delgado Kloos to ask him all about the big trends in education data.

By Alasdair MacKinnon

In traditional education the only data recorded from students was the occasional test or exam score. What new sources of data are available using education technology, and how are they being used?

Indeed, in traditional education there were tests and exams, but the personal interaction in class also served as a source of data. Unfortunately, this personal interaction does not scale well. In a classroom of hundreds of students, it is impossible to interact with all of them and know where their problems lie and how they are advancing in their learning.

However, once the learning interaction goes through the digital channel, we have a new source of data, with the advantage that it can be processed programmatically in a variety of ways. From a low-level point of view, these are mainly just clicks with a destination and a timestamp. From a higher-level point of view, these are posts in forums, viewing and reviewing of videos, responses in quizzes, etc.

What opportunities does the harvesting of large amounts of data offer?

The opportunities lie at different levels of discourse. We can improve an individual’s learning through reacting automatically (adaptive learning) or with a personal or group intervention. There are different possible time frames: we can use the information just for reporting purposes, for programming interventions in real time, to predict future behaviour, or even to find the optimal educational path in advance.

Learning analytics and educational data mining can be used to empower educators, but they can also be useful to identify best practices and help advance research in the learning sciences.

Furthermore, this field can help define policies at different levels (university, regional, national, global). Think about PISA and how it has impacted governments. Well, PISA is based on assessments; now imagine that we could replace data from assessments by data from individual interactions in an organized way.

So, the implications and opportunities are big and can have a huge impact.

What problems does one encounter when trying to harvest these large datasets?

One problem is that there is a long way to go from low-level information, like clicks here and there, to useful knowledge about the learning process that can be used in productive ways.

This can be better illustrated with an example. Below I show the viewing behaviour of a video in edX:

For every second, we see how many learners have viewed it (light blue), but also how many have replayed it (dark blue). So, second 00:45 has been viewed by 7,476 different learners and has been replayed 3,752 times. On average, half of the learners have replayed this particular instance. There is a peak there. This is very relevant information that draws our attention to the explanation happening at this very moment. The problem is one of interpretation. What was the issue here? Was the explanation not good enough? Was it too fast? Is this a conceptually difficult topic? We can now use other techniques to answer these questions. The good news is that we could not get this information automatically from physical interaction. The possibilities arising here open up the discussion at a new level. We can automate the improvement of our teaching in general from a systemic point of view and for each student in particular. However, there is still a long way to go.

Do ethical concerns arise, and how can they be addressed?

One has to be very careful with privacy issues, be respectful to all stakeholders and compliant with all applicable legislation. The medical sector has been using private information for many years for the good of the individual patient and for medical research. It is a good model from which to get inspiration.

At UC3M, we are participating in a European effort, the SHEILA project, coordinated by the University of Edinburgh that addresses these issues and will build a policy-development framework.

Where do you see data analytics in education progressing, and what are its limits?

Like in medicine, analytics is key to knowing what is going on and to being able to take action. But learning analytics technology just gives us information that we have to correctly understand and interpret. This game of numbers can show us a correlation, but this by no means equates with causality. This means that though the information might be very valuable, we have to see it from this limiting perspective.

Speaking at the eMadrid session will be

Pedro Muñoz Merino (UC3M), a member of edX, who will present some examples of how to analyse students’ behaviours on online educational platforms, as well as some visual analytics tools that can help provide understanding of the learning process.

Ruth Cobos (Universidad Autónoma de Madrid), another member of edX, who will go into results obtained from an analysis of MOOCs.

Manuel Freire (Universidad Complutense de Madrid), who wiill offer learning analytics results obtained from the booming field of educational games.

Edmundo Tovar (Universidad Politécnica de Madrid), who will give insight into a MOOC case study related to language learning.

Find out more about eMadrid at OEB16, November 30 – December 2.