Abstract

Energy Communities (ECs) are emerging frameworks where citizens collectively share renewable energy. Leveraging knowledge about this topic is challenging because of how varied these types of communities might be and how many actors are involved in decision-making. We are developing En-join, a game in which the player has to solve open-ended challenges that are mediated and evaluated by conversational agents that represent members of an EC. We implemented and prompted an LLM (Phi-4) to perform role-playing and evaluation simultaneously. We tested prompt variants indicating personality and behavior and meta-evaluated the evaluation performance using six predefined answers across three levels. Our results suggest that indicating social preferences noticeably affects the evaluation behavior. We contribute to the field of games and serious games by showing how LLMs can be used as conversational characters and evaluator agents simultaneously and suggest that role-playing might be affecting evaluation behavior in any LLM implementations.