Flashy or Effective? We Need to be Honest when Evaluating New Tools

Think back to your early school days for a moment. Were you overjoyed when your teacher gave you homework? Did you appreciate the instructor that demanded more work or pushed you outside of your comfort zone? How did you feel when you were assigned another group project? At the time, many of us were probably frustrated, but these are exactly the educators that helped us learn better, with a deeper understanding of the material and long-term knowledge retention. When students (appropriately) struggle with an assignment, they are learning. When they are asked to recall information or discuss their ideas and misconceptions with each other, they are learning. They learn, even if they sometimes complain.

True transformation in teaching and learning should be motivated by learning outcomes, not popularity contests

A painful truth in higher education is that, while some students may perceive lectures to be a more effective method of learning (“I’ll absorb the knowledge directly from the expert professor”), they actually learn better when they are active and engaged. Active learning, such as group work and problem-solving, has many advantages, including a focus on higher-order thinking. It also has the welcome side effect of soft skills development. Skills such as verbal and written communication, empathy, and social skills needed for teamwork. In a meta-analysis of 225 papers, data on student outcomes comparing lectures to active learning show that students are 1.5x more likely to fail with traditional lecturing vs. active learning (Freeman et al. 2014).

Another research study published in Proceedings of the National Academy of Sciences, compared two groups of physics students at Harvard University, one learning with lectures, and the other switching to active learning: experimentation and group problem-solving. The findings showed that the active learning cohort’s outcomes in the final exams were 10 points better than their lecture-only counterparts. Ironically, the students still clearly preferred the lower-effort methods of lecturing. In the words of the researchers: “We find that while students in the active classroom learn more, they feel like they learn less. We show that this negative correlation is caused in part by the increased cognitive effort required during active learning.”

In other words, students prefer a more passive consumption of content to active, involved learning methods – but it isn’t what’s best for them. Why is that? Because it’s what they’re used to, it demands very little of them, and they (wrongfully) believe it’s the preferred method of instruction. Simply put, they “like it more,” even though the numbers show that students learn better in active learning situations where likeability is nearly irrelevant.

This trend continues in the digital arena, with teachers and instructors eagerly looking for the “latest and greatest, slick tool” to entice their students. New learning platforms roll out tools and features to capture this expanding interest despite unclear efficacy. Typically, when new learning technologies are introduced, companies and organizations immediately seek user feedback. “Did you like it?” “Was it better than XYZ tool that we are using today?”

But are those the right questions to ask? If not, what are the right questions? Perhaps there is a way to get “learning better” and “liking it more” in one place? I believe this is possible, and it starts with being honest about your goals and asking the right questions about a prospective learning platform.

• What are the skills learners should acquire during their learning path?
• How will you know they learned these skills?
• Were they active and engaged in their learning?
• Did they collaborate with peers and share their perspectives?
• Did they feel seen and heard?
• Were they able to make meaningful connections, both with other humans and between neurons in their brains as part of the learning process
  
  
  

Considering what we know about effective learning, evaluations of new learning technologies should be based on objectives rather than whether it’s “cool”, comfortable, or popular. Assessment of hybrid and online pedagogy are already showing us the answer. The real task before us is understanding how to implement these new technologies in a way that is as effective as it is enjoyable. When we start with the “why,” that is when the true impact is possible.

Written for OEB Global 2022 by Talia Kolodny, Director of Partnerships, EMEA at Engageli