Skip to main content

Outline

Hello March! This week, we’re covering details revolving around the social factors of pedagogical agents. Specifically, we’re asking:

  • Do pedagogical agent facial expressions and gestures impact learning?
  • What factors of social fidelity affect learning from pedagogical agents?

*As a side note, I named this issue from the following book that my infant ADORES:

Facial Expressions & Gestures in Pedagogical Agents

Research on pedagogical agents, those handy virtual characters that guide us through learning situations, has increased quite a bit over the last decade. While some research finds positive learning effects, other studies indicate that including pedagogical agents can actually decrease learning. Why is that? Well, being mindful about our use of pedagogical agents is the key! LSW Issue #26 covers some ways to optimize your use of pedagogical agents.

One of the main reasons that we see positive effects from using pedagogical agents is because we are social beings! The computers-as-social-actors (CASA) paradigm “suggests that subjects interact with computers in the same natural way as they would interact with fellow human beings” (Schneider, Krieglstein, Beege, & Rey, 2022). Thus, we have an additional social drive to learn. This is also supported by the media equation theory, persona effect, and social agency theory (Reeves & Nass, 1996; Woo, 2008; Moreno et al., 2001; Schneider et al., 2022). In support, we also see that pedagogical agents are particularly helpful when they are realistic (Baylor & Kim, 2004). It’s also important to note that a pre-recorded person is also considered a pedagogical agent.

However, to achieve a realistic (i.e., human-like) pedagogical agent, shouldn’t they also move like a human? Past work has predominantly focused on only gestures and facial expressions that were directly related to learning (Schneider et al., 2022). But as people, we often make extraneous movements - I mean, I come from a Big Italian Family and they basically pride themselves on their hand gestures.

(Welcome to the world of Italian hand GIFs!)

With the idea of CASA and cognitive load in mind, the authors of this study wondered whether gestures and facial expressions that are unrelated to the material actually impact learning (Schneider et al., 2022). To answer this question, learners all viewed instructional videos on geyers, but were split into 4 groups based on the gestures/facial expressions provided by the agent: 1. No gestures/no facial expressions, 2. No gestures, with facial expressions, 3. With gestures, no facial expressions, and 4. With gestures, with facial expressions (Schneider et al., 2022).

The results were “in line with the CASA paradigm,” such that learners reported the instructor to be most “human-like” with facial expressions and gestures both (Schneider et al., 2022). Learners also reported the instructor to be more “learning-facilitating” when gestures were performed as opposed to the no-gesture learners. What about actual learning? Learners in the course with gestures and facial expressions scored significantly higher than all other groups for retention (see graph). A similar pattern was observed for transfer, as “both gestures and facial expressions helped to increase transfer knowledge” (Schneider et al., 2022).

(Schneider et al., 2022)

The findings from this study indicate that pedagogical agents with facial expressions and/or gestures can help to promote learning retention and transfer!

Key Takeaway: Learners seem to perform better with pedagogical agents that are more human-like and behave naturally. When implementing a pedagogical agent, including facial expressions and body movements that are not related to learning can lead to increased learning retention and transfer.

Read More ($): Schneider, S., Krieglstein, F., Beege, M., & Rey, G. D. (2022). The impact of video lecturers’ nonverbal communication on learning - An experiment on gestures and facial expressions of pedagogical agents. Computers & Education, 176.

“Learners can reliably assess whether a pedagogical agent behaves naturally. Interestingly, even learning-irrelevant body movements can improve learning and do not increase the perceived cognitive load.- (Schneider et al., 202

Keeping the Social Presence

Since our first article covered gestures, which is a form of social fidelity, it seemed like a recent review of social fidelity in pedagogical agents would be valuable! Looking at the virtual environment, two types of fidelity are identified. The first is functional, which references “how closely the items match the real functions and react as they would in the real world;” the second is physical, which refers to “how close the environment looks to the actual environment” (Sinatra, Pollard, Files, Oiknine, Ericson, & Khooshabeh, 2021). In the current review, the authors apply the types of fidelity of a virtual environment to the social aspect of pedagogical agents. Switching to social fidelity, functional social fidelity is how close the interaction is to real-world interactions regarding content; physical social fidelity references how close the auditory and “communication methods are to a real social interaction” (Sinatra et al., 2021). Then, which facets of social fidelity impact learning outcomes?

In this issue, we’re covering functional social fidelity. The authors identified the following as topics of importance: language use, politeness and enthusiasm, attention/feedback and social memory, personality and interactivity, and gestures. Below, I address the key findings from each topic.

Language Use

  • Pedagogical agents should use colloquial or personalized language, which improves retention and transfer (Sinatra et al., 2021).

Politeness & Enthusiasm

  • Enthusiasm, through tone of speech and facial expressions, can be beneficial for comprehension (keep in mind that some research suggests this is only true when cognitive load is low - see: LSW Issue #57).
  • Polite pedagogical agents lead to improved learner outcomes regarding self-assessments, self-regulated learning, and time spent reviewing information (Sinatra et al., 2021).

Attention, Feedback, & Social Memory

  • Pedagogical agents with the ability to hold social memory, such as remembering the learner’s name and/or course performance, lead to increased levels of learner engagement. Further, agents that provide varied feedback, rather than a “standard” or repetitive feedback phrase, also improved learner engagement (Sinatra et al., 2021).

Personality & Interactivity

  • The personality of the pedagogical agent should remain consistent. When the agent’s characteristics (voice traits, length of speech, introvert/extrovert, etc.) remain the same, as well as when the characteristics match those of the learner, social presence is increased (Sinatra et al., 2021).
  • Some pedagogical agents provide long monologues or lectures, while others are interactive. Studies show increased learning transfer with interactive agents (Sinatra et al., 2021).

Gestures

  • As with Schneider et al. (2022), gestures have been found to be quite beneficial. Specifically, deictic gestures show consistent benefits (Sinatra et al., 2021).

Pedagogical agents can be a useful learning tool! However, to do so, they must be used with caution. Otherwise, we risk losing the learner to extra processing, feeling disconnected, etc. When implementing a pedagogical agent, be sure to tackle the above topics to optimize use!

Key Takeaway: When using a pedagogical agent, learners seem to perform best with more natural characters. Try to optimize use by addressing language use, politeness and enthusiasm, feedback and social memory, personality and interactivity, and gestures.

Read More ($): Sinatra, A. M., Pollard, K. A., Files, B. T., Oiknine, A. H., Ericson, M., & Khooshabeh, P. (2021). Social fidelity in virtual agents: Impacts on presence and learning. Computers in Human Behavior, 114.

Pets of Learning Science Weekly

It’s another BOGO week! Thanks for sharing your sweet babies, Shannon H.! Below we get a close look at a sweet snuggle session between Jumbo the cat and Eleanor the pup 💞

Send us your pet pics at editor@learningscienceweekly.com.

Wondering why we’re including animal photos in a learning science newsletter? It may seem weird, we admit. But we’re banking on the baby schema effect and the “power of Kawaii.” So, send us your cute pet pics -- you’re helping us all learn better!

The LSW Crew

Learning Science Weekly is written and edited by Katie Erhardt, Ph.D.

Have something to share? Want to see something in next week's issue? Send your suggestions: editor@learningscienceweekly.com