Date
Publisher
arXiv
Interactive systems such as chatbots and games are increasingly used to persuade and educate on sustainability-related topics, yet it remains unclear how different delivery formats shape learning and persuasive outcomes when content is held constant. Grounding on identical arguments and factual content across conditions, we present a controlled user study comparing three modes of information delivery: static essays, conversational chatbots, and narrative text-based games. Across subjective measures, the chatbot condition consistently outperformed the other modes and increased perceived importance of the topic. However, perceived learning did not reliably align with objective outcomes: participants in the text-based game condition reported learning less than those reading essays, yet achieved higher scores on a delayed (24-hour) knowledge quiz. Additional exploratory analyses further suggest that common engagement proxies, such as verbosity and interaction length, are more closely related to subjective experience than to actual learning. These findings highlight a dissociation between how persuasive experiences feel and what participants retain, and point to important design trade-offs between interactivity, realism, and learning in persuasive systems and serious games.
Study at a glance
- Question: How do essays, chatbots, and text-based games affect user experience, persuasion, and factual knowledge retention?
- Setting and data: University-recruited adult volunteers, 43 participants, mostly aged 18 to 34.
- Method: Between-subjects user study with pre- and post-questionnaires and a delayed 24-hour knowledge quiz.
Why it matters
Interactive systems like chatbots and games are increasingly used for sustainability education, but evidence is limited on how different delivery modes affect learning and persuasion when factual content is kept identical.
Key findings
- Chatbot interactions led to significantly greater perceived increases in topic importance than static essays (beta = 2.22), while intention to act did not differ across modes.
- Perceived learning diverged from retention: game participants reported learning less than essay readers, yet scored higher on the delayed 24-hour factual knowledge quiz.
- Interaction log metrics such as word counts, turns, and session duration were associated with subjective impressions but did not reliably predict delayed objective learning.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
