Implementation and effect evaluation of dynamic difficulty adjustment based on reinforcement learning in Multiplayer Online Battle Arena games
DOI:
https://doi.org/10.61173/r8885t34Keywords:
Dynamic Difficulty Adjustment, Reinforcement Learning, MOBA Games, Player Experience, Game Design OptimizationAbstract
This paper explores the implementation and impact of Dynamic Difficulty Adjustment (DDA) in Multiplayer Online Battle Arena (MOBA) games using reinforcement learning algorithms. It evaluates the effectiveness of DDA in improving player experience by dynamically adjusting game difficulty based on real-time player performance data. The study conducts empirical research on League of Legends, involving players of varying skill levels and using algorithms such as DQN, PPO, A2C, SAC, and TD3. The results indicate significant improvements in player engagement, satisfaction, and retention with the application of DDA. The paper suggests further optimization strategies for DDA and discusses its long-term effects on player behavior, highlighting the potential of reinforcement learning in game design.References
[1] Chen, S. (2024). Literature review of application of AI in improving gaming experience: NPC behavior and dynamic difficulty adjustment. LUT Publications. https://lutpub.lut.fi/ handle/10024/167834
[2] Mortazavi, F., Moradi, H., & Vahabie, A. H. (2024). Dynamic difficulty adjustment approaches in video games: a systematic literature review. Multimedia Tools and Applications. https:// link.springer.com/article/10.1007/s11042-024-18768-x
[3] Sepulveda, G. K., Besoain, F., et al. (2019). Exploring dynamic difficulty adjustment in videogames. 2019 IEEE CHILEAN Conference on Electrical, Electronics Engineering, Information Theory, Communications and Signal Processing. https://ieeexplore.ieee.org/abstract/document/8988068/
[4] Bonti, A., Palaparthi, M., Jiang, X., & Pham, T. (2021). Framework Design and Implementation for Education Using Dynamic Difficulty Adjustment Based on Deep Reinforcement Learning and Mathematical Approach. International Conference on Advances in Computing and Information Science. https:// link.springer.com/chapter/10.1007/978-3-030-98005-4_17
[5] Bicalho, L. F., Baffa, A., & Feijó, B. (2023). A Dynamic Difficulty Adjustment Algorithm With Generic Player Behavior Classification Unity Plugin In Single Player Games. 22nd Brazilian Symposium on Games. https://dl.acm.org/doi/ abs/10.1145/3631085.3631286
[6] Cheng, J. (2021). Research on Dynamic Task Scheduling Algorithm and Platform Based on Reinforcement Learning. East China Normal University. https://kns.cnki. net/kcms2/article/abstract?v=j6HAoO1nZAwylatyM- M J b T N q a R j w 2 2 v i B p u K l w y l Z E y f _ N1gneJjhGHj2CQCgK6ty80Mks5m_sdCz02QqhcXAP3Y2kaj MVT4gubA6Q6KbAJ9tYS6bkECQcOIn4szyDE1ScCPeTidgIn enYLREztjVw==&uniplatform=NZKPT&language=CHS
[7] Gao, M. (2023). Research on the Application of Big Data Technology in the MOBA Game Industry. Digital Communication World. https://kns.cnki.net/kcms2/article/ abstract?v=j6HAoO1nZAy62VJHiYdoLPnvBbs9irXv GxTR8PwXlx4A1b8D-XT0KRj3q4SIALuSRSZXh5 AjYlLlVMgkzjL8Km3J92ykNwFBh_2nHvH4Up6dS- zT5OSuoQvHl70luYk88uiBYFIfhIraeI_pwxJ4DQ==&uniplatfo rm=NZKPT&language=CHS
[8] Li, H. (2015). Research on Optimal Control and Differential Game of Adaptive Dynamic Programming Based on Data. Institute of Automation, Chinese Academy of Sciences. http:// dpaper.las.ac.cn/Dpaper/detail/detailNew?paperID=20009564
[9] Shao, K. (2019). Game Intelligent Decision Making Based on Deep Reinforcement Learning. Institute of Automation, Chinese Academy of Sciences. http://dpaper.las.ac.cn/Dpaper/ detail/detailNew?paperID=20157285
[10] Pan, F. (2021). Optimization and Application of Reinforcement Learning Strategy Based on Uncertainty Estimation. Institute of Computing Technology. http://dpaper.las. ac.cn/Dpaper/detail/detailNew?paperID=20195223
[11] Zhang, Y. (2018). Research on the Application of Deep Reinforcement Learning in Multi-Machine Combat Tactical Decision Making. Institute of Automation, Chinese Academy of Sciences. http://dpaper.las.ac.cn/Dpaper/detail/ detailNew?paperID=20141496
[12] Fisher, N., & Kulshreshth, A. K. (2024). Exploring Dynamic Difficulty Adjustment Methods for Video Games. Virtual Worlds. Dean&Francis Jingtian Zhang https://doi.org/10.3390/virtualworlds3020012
[13] Mortazavi, F., Moradi, H., & Vahabie, A.-H. (2024). Dynamic difficulty adjustment approaches in video games: a systematic literature review. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-024-18768-x
[14] Deepchecks Community Blog. (2024). Reinforcement Learning Applications: From Gaming to Real-World. Deepchecks. https://www.deepchecks.com/reinforcement-learning-applications-from-gaming-to-real-world/
[15] What video games use reinforcement learning? (2024). Quora. https://www.quora.com/What-video-games-use-reinforcement-learning
[16] Szita, I. (2012). Reinforcement Learning in Games. Adaptation, Learning, and Optimization. https://doi.org/10.1007/978-3-642-27645-3_17
[17] Cinnamon AI. (2022). Reinforcement Learning in Games and Entertainment. Medium. https://cinnamonai.medium. com/reinforcement-learning-in-games-and-entertainment- 47ffb22b9b9e
[18] Tigran Hovsepyan. (2024). Reinforcement Learning: Game-Level Design Technique. Plat.AI. https://plat.ai/blog/ reinforcement-learning-in-game-ai/
[19] Mirna Paula Silva. (2017). Dynamic Difficulty Adjustment on MOBA Games. arXiv.org. https://arxiv.org/abs/1706.02796v1
[20] Zohaib, M. (2018). Dynamic Difficulty Adjustment (DDA) in Computer Games: A Review. Advances in Human-Computer Interaction. https://doi.org/10.1155/2018/5681652
[21] Lopes, J. C., & Lopes, R. P. (2022). A Review of Dynamic Difficulty Adjustment Methods for Serious Games. Communications in Computer and Information Science. https://doi.org/10.1007/978-3-031-23236-7_11
[22] Kun Shao. (2019). A Survey of Deep Reinforcement Learning in Video Games. arXiv.org. https://arxiv.org/ abs/1912.10944v2
[23] Souchleris, K., Sidiropoulos, G. K., & Papakostas, G. A. (2023). Reinforcement Learning in Game Industry—Review, Prospects and Challenges. Applied Sciences. https://doi.org/10.3390/app13042443
[24] Chineng Vang. (2023). The Impact of Dynamic Difficulty Adjustment on Player Experience in Video Games. umn.edu. https://digitalcommons.morris.umn.edu/cgi/viewcontent. cgi?article=1105&context=horizons
[25] Mirna Paula Silva. (2024). [PDF] Dynamic difficulty adjustment on MOBA games | Semantic Scholar. semanticscholar.org. https://www.semanticscholar.org/paper/Dynamic-difficulty-adjustment-on-MOBA-games-Silva-Silva/25ff204f14fb85f554f00aa6f2697ea7f72bfc5c
[26] Hunicke, R. (2005). The case for dynamic difficulty adjustment in games. Proceedings of the ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE ’05), June 2005, Valencia, Spain, 429–433.
[27] Zohaib, M. (2018). Dynamic Difficulty Adjustment (DDA) in Computer Games: A Review. Advances in Human-Computer Interaction, 1–12.
[28] Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529. https://www.nature.com/articles/nature14236
[29] Silva, M. P., Silva, V. do N., & Chaimowicz, L. (2017). Dynamic difficulty adjustment on MOBA games. Entertainment Computing, 18, 103–123.
[30] Deheng Ye. (2021). Towards Playing Full MOBA Games with Deep Reinforcement Learning. arXiv.org. https://arxiv.org/ abs/2011.12692v4
Downloads
Published
Issue
Section
License
Copyright (c) 2024 by the authors.
This work is licensed under a Creative Commons Attribution 4.0 International License.
