Abstract
The paper reports ongoing research toward the design of multimodal affective pedagogical agents that are effective for different types of learners and applications. In particular, the work reported in the paper investigated the extent to which the type of character design (realistic versus stylized) affects students’ perception of an animated agent’s facial emotions, and whether the effects are moderated by learner characteristics (e.g. gender). Eighty-two participants viewed 10 animation clips featuring a stylized character exhibiting 5 different emotions, e.g. happiness, sadness, fear, surprise and anger (2 clips per emotion), and 10 clips featuring a realistic character portraying the same emotional states. The participants were asked to name the emotions and rate their sincerity, intensity, and typicality. The results indicated that for recognition, participants were slightly more likely to recognize the emotions displayed by the stylized agent, although the difference was not statistically significant. The stylized agent was on average rated significantly higher for facial emotion intensity, whereas the differences in ratings for typicality and sincerity across all emotions were not statistically significant. A significant difference in ratings was shown in regard to sadness (within typicality), happiness (within sincerity), fear, anger, sadness and happiness (within intensity) with the stylized agent rated higher. Gender was not a significant correlate across all emotions or for individual emotions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adamo-Villani, N., Beni, G.: Keyboard encoding of facial expressions. In: 8th International Conference on Information Visualization-HCI Symposium (IV04) Proceedings, pp. 324–328. IEEE, London (2004)
Adamo-Villani, N., Wilbur, R.: Two novel technologies for accessible Math and Science Education. IEEE Multimedia Spec. Issue Access. 15, 38–46 (2008)
Adamo-Villani, N., Lestina, J., Anasingaraju, S.: Does character’s visual style affect viewer’s perception of signing avatars? In: Vincenti, G., Bucciero, A., Vaz de Carvalho, C. (eds.) eLEOT 2015. LNICST, vol. 160, pp. 1–8. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-28883-3_1
Adolphs, R.: Perception and emotion: How we recognize facial expressions. Current Dir. Psychol. Sci. 15(5), 222–226 (2006)
Albrecht, I., Schroeder, M., Haber, J., Seidel, H.-S.: Mixed feelings: expression of non-basic emotions in a muscle-based talking head. Virtual Real. (special issue) 8, 201–212 (2005)
Bancroft, T.: Creating Characters with Personality. Watson-Guptil, New York (2016)
Baylor, A., Kim, S.: Designing nonverbal communication for pedagogical agents: when less is more. Comput. Human Behav. 25(2), 450–457 (2009)
Becker, C., Wachsmuth, I.: Modeling primary and secondary emotions for a believable communication agent. In: Proceedings of International Workshop on Emotion and Computing, in Conjunction with the 29th annual German Conference on Artificial Intelligence (KI2006), Bremen, Germany, pp. 31–34 (2006)
Calder, A., Rhodes, G., Johnson, M., Haxby, J.: Oxford Handbook of Face Perception. Oxford University Press, Oxford (2011)
Cissel, K.: A study of the effects of computer animated character body style on perception of facial expression. Purdue University e-Pub, MS thesis (2015). https://docs.lib.purdue.edu/cgttheses/28/
Courgeon, M., Buisine, S., Martin, J.C.: Impact of expressive wrinkles on perception of a virtual character’s facial expressions of emotions. In: Proceedings of 9th International Conference, IVA 2009, Netherlands, pp. 201–214 (2009)
D’Mello, S., Graesser, A.: Emotions during learning with AutoTutor. In: Durlach, P.J., Lesgold, A.M. (eds.) Adaptive Technologies for Training and Education, pp. 117–139. Cambridge University Press, New York (2012)
D’Mello, S., Graesser, A.: AutoTutor and affective autotutor: learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transact. Interact. Intell. Syst. 2(4), 23 (2012)
Davis, C., Kim, J., Kuratate, T., Burnham, D.: Making a thinking-talking head. In: Proceedings of the International Conference on Auditory-Visual Speech Processing (AVSP 2007), Hilvarenbeek. The Netherlands (2007)
Domagk, S.: Do pedagogical agents facilitate learner motivation and learning outcomes? The role of the appeal of agent’s appearance and voice. J. Media Psychol. 22(2), 84–97 (2010)
Ekman, P., Friesen, V.W.: Unmasking the Face. A guide to Recognizing Emotions from Facial Expressions. Malor Books, Cambridge (1975)
Ekman, P., Friesen, V.W.: Manual for the Facial Action Coding System. Consulting Psychologists Press (1977)
Garcia-Rojas, A., et al.: Emotional face expression profiles supported by virtual human ontology. Comp. Anim. Virtual Worlds 17, 259–269 (2006). https://doi.org/10.1002/cav.130
Guo, Y.R., Goh, H.L.: Affect in embodied pedagogical agents: meta-analytic review. J. Educat. Comput. Res. 53(1), 124–149 (2015)
Holmes, J.: Designing agents to support learning by explaining. Comput. Educ. 48, 523–547 (2007)
Huang, T.S., Cohen, I., Hong, P., Li, Y.: IA3: Intelligent Affective Animated Agents. In: Proceedings of ICIG 2000 (2000)
Hyde, J., Carter, E., Kiesler, S., Hodgins, J.K.: Perceptual effects of damped and exaggerated facial motion in animated characters. In: Proceedings of the 10th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2013) (2013)
Intense. (n.d.). In Merriam-Webster. https://www.merriam-webster.com/dictionary/intense
Johnson, W.L., Rickel, J.: Steve: an animated pedagogical agent for procedural training in virtual environments. SIGART Bull. 8, 16–21 (1998)
Kim, S., Baylor, A.: Pedagogical agents as social models to influence learner attitudes. Educat. Technol. 47(01), 23–28 (2007)
Kim, Y., Lim, J.: Gendered socialization with an embodied agent: creating a social and affable mathematics learning environment for middle-grade females. J. Educat. Psychol. 105(4), 1164–1174 (2013)
Kim, Y., Wei, Q.: The impact of user attributes and user choice in an agent-based environment. Comput. Educat. 56, 505–514 (2011)
Kim, Y., Baylor, A., Shen, E.: Pedagogical agents as learning companions: the impact of agent emotion and gender. J. Comput. Assist. Learn. 23, 220–234 (2007)
Lester, J., Voerman, J., Towns, S.J., Callaway, C.: Cosmo: a life-like animated pedagogical agent with deictic believability. In: Notes of the IJCAI 1997 Workshop on Animated Interface Agents: Making Them Intelligent, Nagoya, Japan, pp. 61–70 (1997)
Lester, J., Stone, B., Stelling, G.: Lifelike pedagogical agents for mixed-initiative problem solving in constructivist learning environments. User Model. User-Adap. Inter. 9(1–2), 1–44 (1999)
Lester, J., Towns, S.G., Fitzgerald, P.: Achieving Affective Impact: Visual Emotive Communication in Lifelike Pedagogical Agents. Int. J. Artif. Intell. Educ. 10(3–4), 278–291 (1999)
Lester, J., Converse, S., Kahler, S., Barlow, T., Stone, B., Bhogal, R.: The persona effect: affective impact of animated pedagogical agents. In: Proceedings of CHI 1997, pp. 359–366 (1997)
Lester, J., Converse, S., Stone, B., Kahler, S., Barlow, T.: Animated pedagogical agents and problem-solving effectiveness: a large-scale empirical evaluation. In: Proceedings of the 8th World Conference on Artificial Intelligence in Education, pp. 23–30 (1997)
Lisetti, C., Nasoz, F.: MAUI: a multimodal affective user interface. In: Proceedings 10th ACM Int’l Conf. Multimedia (Multimedia 2002), pp. 161–170 (2002)
Martinez, A., Benavente. R.: The AR Face Database. CVC Technical Report #24 (1998)
Mayer, R.E.: Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In Mayer, R.E. (ed.) The Cambridge handbook of multimedia learning, 2nd ed, pp. 345–368. Cambridge University Press, New York (2014)
Mayer, R.E., DaPra, S.: An embodiment effect in computer-based learning with animated pedagogical agent. J. Exp. Psychol. Appl. 18, 239–252 (2012)
McCloud, S.: Understanding Comics: The Invisible Art. Northampton, Mass (1993)
McDonnell, R., Jorg, S., JMcHugh, J., Newell, F., O’Sullivan, C.: Evaluating the emotional content of human motions on real and virtual characters. In: Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, pp. 67–73 (2008)
Morency, L.P., et al.: Demonstration: a perceptive virtual human interviewer for healthcare applications. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Moreno, R., Flowerday, T.: Students’ choice of animated pedagogical agents in science learning: a test of the similarity attraction hypothesis on gender and ethnicity. Contemp. Educ. Psychol. 31, 186–207 (2006)
Moreno, R., Mayer, R.E.: Interactive Multimodal Learning Environments. Educ. Psychol. Rev. 19, 309–326 (2007)
Nass, C., Isbister, K., Lee, E.J.: Truth is beauty: researching embodied conversational agents. Embodied Conversational Agents, pp. 374–402 (2000)
Paleari, M., Lisetti, C.: Psychologically grounded avatars expressions. In: Proceedings of First Workshop on Emotion and Computing at KI 2006, 29th Annual Conf. on Artificial Intelligence, Bremen, Germany (2006)
Pandzic, I.S., Forcheimer, R.: MPEG4 facial animation – The standard, implementations and applications. John Wiley, Sons, Chichester (2002)
Pelachaud, C.: Modelling multimodal expression of emotion in a virtual agent. Phil. Trans. R. Soc. B 364, 3539–3548 (2009)
Powers, D., Leibbrandt, R., Luerssen, M., Lewis, T., Lawson, M.: PETA-a pedagogical embodied teaching agent. In: Proceedings of PETRA08: 1st international conference on PErvasive Technologies Related to Assistive Environments, Athens (2008)
Riek, D.L., Rabinowitch, T.C., Chakrabarti, B., Robinson, P.: How anthropomorphism affects empathy toward robots. In: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction, pp. 245–246 (2009)
Rosenberg-Kima, R.B., Baylor, A., Plant, A., Doerr, C.: Interface agents as social models for female students: the effects of agent visual presence and appearance on female students’ attitudes and beliefs. Comput. Hum. Behav. 24, 2741–2756 (2008)
Ruttkay, Z., Dormann, C., Noot, H.: Embodied conversational agents on a common ground. In: Ruttkay, Z., Pelachaud, C. (eds.) From Brows to Trust. HIS, vol. 7, pp. 27–66. Springer, Dordrecht (2004). https://doi.org/10.1007/1-4020-2730-3_2
Scherer, K.R.: Appraisal considered as a process of multilevel sequential checking. In: Scherer, K., Schorr, A., Johnstone, T. (eds.) Appraisal processes in emotion: theory, methods, research, pp. 92–119. Oxford University Press, New York (2001)
Schroeder, N.L., Adesope, O., Barouch Gilbert, R.: How effective are pedagogical agents for learning? A Meta-Analytic Review. J. Educ.Comput. Res. 49(1), 1–39 (2013)
Sincere. (n.d). In Oxford Dictionaries Online. https://en.oxforddictionaries.com/definition/sincere
Thomas, F., Johnston, O.: The Illusion of Life. Disney Animation. Disney Editions, Los Angeles (1995)
Typical. (n.d.). In Oxford Dictionaries online. https://en.oxforddictionaries.com/definition/typical
Wallraven, C., Breidt, M., Cunningham, D.W., Bulthoff, H.H.: Evaluating the perceptual realism of animated facial expressions. ACM Transact. Appl. Percept. 23, 1–20 (2007). http://dl.acm.org/citation.cfm?id=127876456
Wang, N., Johnson, L., Mayer, R.E., Rizzo, P., Shaw, E., Collins, H.: The politeness effect: pedagogical agents and learning outcomes. Int. J. Hum. Comput. Stud. 66, 98–112 (2008)
Welch, R., Blackmon, T., Liu, A., Stark, L.: The effects of pictorial realism, delay of visual feedback, and observer interactivity on the subjective sense of presence. Presence Teleoperators Virtual Environ. 5, 263–273 (1996)
Willcox, G.: The feeling wheel: a tool for expanding awareness of emotions and increasing spontaneity and intimacy. Transact. Anal. J. 12(4), 274–276 (1982)
Woo, H.L.: Designing multimedia learning environments using animated pedagogical agents: factors and issues. J. Comput. Assist. Learn. 25(3), 203–218 (2009)
Zhou, L., Mohammed, A., Zhang, D.: Mobile personal information management agent: Supporting natural language interface and application integration. Inf. Process. Manage. 48(1), 23–31 (2012)
Acknowledgments
This work is supported by NSF-Cyberlearning award 1821894, and by Purdue University Instructional Innovation Grant 2017–2019. The authors would like to thank Purdue Statistical Consulting for their help with the statistical analysis of the data.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Adamo, N., Dib, H.N., Villani, N.J. (2019). Animated Agents’ Facial Emotions: Does the Agent Design Make a Difference?. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2019. Lecture Notes in Computer Science(), vol 11613. Springer, Cham. https://doi.org/10.1007/978-3-030-25965-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-25965-5_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25964-8
Online ISBN: 978-3-030-25965-5
eBook Packages: Computer ScienceComputer Science (R0)