How to Judge Effective Use of AI in Education

I am the Product Designer for BigBlueButton, an open-source virtual classroom used by millions of users world-wide in billions of minutes of online classes. I’ve had this role for fourteen years and, during that time, I’ve thought a lot about the following question: How can we help teachers teach and learners learn?

Here I’ve used the term “teachers” to collectively refer to professors in higher-education, instructors in the work-force, and traditional classroom teachers.

Over the years, as technology advances, the complexity to this question has increased, and no more so than in the last twelve months with the emergence of Artificial Intelligence (AI). Enter ChatGPT, a large pre-trained language model that gives everyone with a web browser a familiar chat-based interface to generate answers (right or wrong) to any question based on its training data, which, as of ChatGPT 4.0, was trillions of words (basically, nearly the entire internet as of September 2021).

Let’s deal with the AI elephant in the room first: cheating. It’s clear that students are heavily using ChatGPT to circumvent the effort of doing an assignment. Like Napster, which made it trivial to download music, ChatGPT makes it trivial to type in a question and get an answer (right or wrong) to an assignment question. Not too surprisingly, ChatGPT usage actually dropped in June as students presumably stopped using it for plagiarism. I believe we won’t stop plagiarism; rather, we need to create a framework to answer the question “What is an effective use of AI in education?”.

In the BigBlueButton project, we created a framework for determining whether a given feature will help the teacher teach or the learner learn. Applied to the virtual classroom, this framework asks three fundamental questions:

Can the feature save the instructor time?
Can the feature enable students to apply themselves during the virtual class?
Can the feature increase the amount of feedback students get when struggling?

If the answer is ‘no’ to all, then the feature is not effective and will not help the teacher teach and the learner learn.

This framework is general enough to judge the efficacy of any educational technology. With AI, the framework applies as follows:

Can the AI-based feature save the instructor time?
Can the AI-based feature assist students when applying themselves to solve a problem?
Can the AI-based feature increase the amount of feedback students get when struggling?

We boiled this framework down into the following mantra: maximize time for applied learning and feedback. As you’ll see below, the key word is “learning”. Let’s break down how this framework cuts through some of the AI hype.

Saving Time

The most valuable resource a teacher has is time. If an AI can automate existing manual tasks and save just 10% of a teacher’s time, for example, it’s a HUGE benefit. Here are some good examples:

Generate personalized lesson plans based on an individual student’s interests.
Create quiz questions from course content for formative assessment (and export to common formats, such as Moodle XML, for easy uploading to the LMS).
Assist in grading assignments by making recommendations on incorrect answers or improvements to answers.

The more you save the teachers time with manual tasks, the more time they have to teach. In this regard, an effective use of AI is to assist teachers, not replace them.

We are human, and students learn best from other humans. If AI can free up more time for a teacher to strengthen that human bond, it’s a good use. If you hear that AI is going to replace humans in teaching, based on the above, I don’t believe it’s going to happen.

Applied Learning

How do we actually learn? There are many well established theories of Pedagogy – the science of teaching and learning – that already answer this question. Consider Bloom’s Taxonomy, which states we learn in six stages.

Figure: Bloom’s Taxonomy Six Stages of Learning

I’ve expressed the diagram as a sort of staircase. You can think of reaching mastery as climbing Bloom’s Staircase (and climbing a staircase takes effort).

A fact about learning is that our brains are not computers, they are biological. We can’t watch four hours of basic German and walk out on the street and start speaking German. Our brains are more akin to a muscle: we need to struggle with new concepts to trigger the chemical reactions in our brain that create new neural pathways. Like a workout mantra at the gym that states “no pain, no gain”, learning has a similar pithy mantra “no struggling, no learning”.

The following diagram shows how I believe AI can help a student climb Bloom’s Staircase.

Figure: The three roles of AI for climbing Bloom’s Staircase

In (1), AI is helping the student get the base concepts to ensure they can start applying themselves. An example of a base concept would in accounting and understanding the difference between debits and credits. This stage involves a lot of memorization and creates the scaffolding for applying the knowledge.

In (2), this is where the real learning occurs. My math teacher once told me, “The real learning occurs when you close the textbook and try to solve the problem on your own.” The real learning occurs when you apply and struggle.

Struggle is good and essential. You can’t walk into the gym and watch others lift weights and expect to get stronger, as there is no stress on your muscles. They won’t grow.

You can’t passively watch a live class or recorded class, or type a question into ChatGPT and look at the answer and think “that looks good” and expect to learn. There is no struggle in your brain. There are no chemical changes that create and strengthen new synapses in your brain. With no changes, you won’t climb Bloom’s Staircase.

The real use of AI is to optimize the struggle for maximum learning. Think Lev Vygotsky’s seminal theory of learning and development: The Zone of Proximal Development, which states the most effective learning occurs at the edge of your ability (with good scaffolding) and with help from a skilled teacher (or tutor). Here, an AI based tutor could (a) ensure the scaffolding is solid, (b) assess your skills and challenge you with a task that puts you in the zone, (c) give you hints after you have struggled a bit (like a good tutor), and (d) never, ever simply tell you the answer (like a bad tutor).

If you’re interested in reading more about the role of “AI as a Tutor”, I explored this much further in ChatGTP needs a Tutor Mode (https://medium.com/@ffdixon/chatgpt-needs-a-tutor-mode-2e4fd1e8f423).

In (3), AI assists the student in efficiently applying their new skill. In this regard, as a student masters a new skill, the AI can save the student time and enhance, but not replace, their ability to demonstrate competency in the skill.

You might argue that AI can help students completely skip (2), which is essentially what is happening when students turn in AI-generated solutions to assignments.

I would argue that the disruption coming to schools is not how we learn – just because we have access to AI doesn’t change the way our brains work (again, our brains are not computers) – but a resurgence of having students demonstrate competency without any assistance. This “old school” approach will see students writing essays in person, not submitting them online.

Feedback

Humans best learn from humans. AI is not going to change that, but it can enhance it. Every great athlete has a great coach. Every great student has great teachers.

If AI can assist the teacher in diagnosing the areas that students are struggling, for example, then the teacher could spend more time helping the student. The more feedback students receive, the more they realize “the teacher is aware of me, cares about me, and is taking the time to help me.” This ultimately strengthens the human bond and the student’s ability to learn.

I predict, with the assistance of AI which gives teachers more analytics, their roles will shift a bit more towards that of a coach. Nothing massive, again, maybe 10%. But in this shift, they will have more time to inspire, mentor, and push students to achieve their full potential. The potential benefit for students is huge and life changing.

A Test of the Framework

Let’s take a scenario that you have already heard about how AI will help education and use the framework to test its efficacy.

Imagine you are in a business meeting and AI can summarize the meeting for you (based on text-to-speech transcript and using a large language model to summarize). After an hour long meeting, the language model creates a neatly formatted set of bullet points that summarize the main topics.

Now imagine doing the same for a virtual class. Is this a good use of AI to summarize the class for students to help them learn?

The answer is no. The goal of the virtual class is not to meet, it is to learn. Real learning occurs when students have to struggle (again, we need the chemical changes in our brains to actually learn). A better use of AI would be to have students summarize the key points, have AI assist the teacher in determining the best summaries, enabling the teacher to give real-time feedback to students, praising those that did well and encouraging those that struggled that they are making progress.

Summary

We need a world filled with skeptical, curious, caring, persistent, and skilled life-long learners that are good at working with others to solve big problems. And we need teachers more than ever to cultivate such students.

AI has a potential to incrementally optimize the efforts of both, but only if it plays the correct role in helping teachers teach and learners learn. We’ve used the above framework in building towards the world’s most effective virtual classroom, and we believe this framework gives you a lens to cut through the hype and determine the most effective uses of AI in education.

Written by the Blindside Networks team for OEB Global 2023.