Date
Publisher
arXiv
Large language models (LLMs) exhibit dynamic capabilities and appear to comprehend complex and ambiguous natural language prompts. However, calibrating LLM interactions is challenging for interface designers and end-users alike. A central issue is our limited grasp of how human cognitive processes begin with a goal and form intentions for executing actions, a blindspot even in established interaction models such as Norman's gulfs of execution and evaluation. To address this gap, we theorize how end-users 'envision' translating their goals into clear intentions and craft prompts to obtain the desired LLM response. We define a process of Envisioning by highlighting three misalignments: (1) knowing whether LLMs can accomplish the task, (2) how to instruct the LLM to do the task, and (3) how to evaluate the success of the LLM's output in meeting the goal. Finally, we make recommendations to narrow the envisioning gulf in human-LLM interactions.
Study at a glance
- Question: What challenges do people face when turning their goals into prompts for AI models?
- Setting and data: Not stated in the abstract
- Method: Theoretical framework
Why it matters
Calibrating interactions with large language models is challenging for designers and users alike. A central issue is our limited grasp of how human cognitive processes begin with a goal, form intentions, and craft prompts.
Key findings
- Users face significant cognitive challenges translating their goals into clear intentions and effective prompts when interacting with large language models.
- Users often struggle to determine whether a language model can accomplish a task and how to instruct it effectively.
- Users face difficulty evaluating whether a language model's generated response successfully meets their original goal.
Summary drafted with AI from the paper’s abstract and may contain errors. See the full paper for details.
Who is the user?
Why use AI?
Study design
