Research and Prospects of Multimodal Technology of AIGC in NPC Dialogue Generation

Abstract

In today’s technologically advanced world, in-game NPC dialogue with players is a crucial component of modern game production, directly impacting player immersion and engagement. The creation of large language models such as GPT-4V and VIMA, along with other advances in multimodal learning methods, has significantly enhanced the ability of game NPC systems to recognize and respond to diverse signals and complex scenarios. This paper examines the current state of the art in multimodal dialogue production in great detail, covering common methods, widely used datasets, and evaluation criteria. This work proposes solutions to these issues, including leveraging lightweight model structures, developing effective methods for aligning data from various modalities, and creating improved multimodal datasets specifically designed for gaming environments. This paper aims to inform the next generation of NPC dialogue systems using multimodal AIGC by examining the strengths and weaknesses of current approaches. This will help in-game dialogue become more meaningful, characters more aware of their surroundings, and interactions feel more realistic.