Abstract

Energy Communities (ECs) are emerging frameworks where citizens collectively share renewable energy. Leveraging knowledge about this topic is challenging because of how varied these types of communities might be and how many actors are involved in decision-making. We are developing En-join, a game in which the player has to solve open-ended challenges that are mediated and evaluated by conversational agents that represent members of an EC. We implemented and prompted an LLM (Phi-4) to perform role-playing and evaluation simultaneously. We tested prompt variants indicating personality and behavior and meta-evaluated the evaluation performance using six predefined answers across three levels. Our results suggest that indicating social preferences noticeably affects the evaluation behavior. We contribute to the field of games and serious games by showing how LLMs can be used as conversational characters and evaluator agents simultaneously and suggest that role-playing might be affecting evaluation behavior in any LLM implementations.

Abstract

Building on previous work on incorporating large language models (LLM) in gaming, we investigate the possibility of implementing LLM as evaluating agents of open-ended challenges in serious games and its potential to facilitate a meaningful experience for the player. We contribute with a sustainability game prototype in a single natural language prompt about energy communities and we tested it with 13 participants inside ChatGPT-3.5. Two participants were already aware of energy communities before the game, and eight of the remaining 11 gained valuable knowledge about the specific topic. Comparing ChatGPT-3.5 evaluations of players’ interaction with an expert’s assessment, ChatGPT-3.5 correctly evaluated 81% of player’s answers. Our results are encouraging and show the potential of using LLMs as mediating agents in educational games, while also allowing easy prototyping of games through natural language prompts.

Video Preview

Abstract:
This paper presents the results of an empirical exploration of 10 cursor movement metrics designed to measure respondent hesitation in online surveys. As a use case, this work considers an online survey aimed at exploring how people gauge the electricity consumption of domestic appliances. The cursor metrics were derived computationally from the mouse trajectories when rating the consumption of each appliance and analyzed using Multidimensional Scaling, Jenks Natural Breaks, and the Jaccard Similarity Index techniques. The results show that despite the fact that the metrics measure different aspects of the mouse trajectories, there is an agreement with respect to the appliances that generated higher levels of hesitation. The paper concludes with an outline of future work that should be carried out in order to further understand how cursor trajectories can be used to measure respondent hesitation.

Poster:

Reproducible Research:

Data (scripts will be added soon)

Abstract

In order to promote a more effective understanding of electricity consumption in households, and leverage the potential of current and future energy conservation initiatives, we have developed an online survey in which visual icons of various electrical devices coupled with a Likert scale are used to investigate people’s perception of the amount of electricity consumed by a variety of individual domestic appliances. In this paper we present initial results from a pilot study conducted with 293 participants from over 50 countries and outline our plans for future work. Our initial results suggest that despite a considerable number of respondents was able to give the correct answers, many others seem to lack an understanding of appliance electricity consumption, and thus further work should be carried in this direction.

Reproducible Research:

Data