A Toolkit for Creating Intelligent Virtual Humans in Extended Reality

This paper introduces an initial implementation and a preliminary evaluation of a toolkit for providing Intelligent Virtual Humans in Extended Reality (XR) environments. These virtual humans may serve as avatars for users, known as Smart Avatars, or artificial agents, known as Intelligent Virtual Agents. The presented toolkit consists of five modules that contribute to producing realistic human-to-human and human-to-agent interactions in XR. The primary focus lies in enabling natural, multimodal communication and interaction through a range of expressive capabilities, including speech, facial expressions, gaze tracking, and full-body animations. By enhancing the realism and responsiveness of virtual humans, this paper strives to facilitate their use in a variety of metaverse applications.

Conference paper