Irresponsible Counselors: Large Language Models And The Loneliness Of Modern Humans

Date

02/2026

AI Overview

Why it matters

People increasingly consult large language models as multi-specialist advisors for personal and high-stakes decisions, yet these stateless systems lack real subjectivity, intention, or clear human responsibility when harms occur.

Study at a glance

Question: What ethical and political problems emerge when people use large language models as personal counselors?
Setting and data: Not stated in the paper
Method: Conceptual and policy analysis
AI tools: ChatGPT, Replika, Wysa, Claude, Gemini

Key findings

Users experience AI advice through four overlapping illusions: understanding, neutrality, relationship and care, and control and responsibility, despite interacting with a stateless statistical system.
Technical features like next-token prediction, statelessness, structural hallucination, and sycophancy prevent AI models from having a personal moral history or being held responsible as counseling agents.
Machine counseling risks shifting the burden of filtering advice onto vulnerable populations who lack access to human care, creating a two-tiered system of counseling quality.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Large language models (LLMs) have rapidly shifted from peripheral assistive tools to constant companions in everyday and even high stakes human decision making. Many users now consult these models about health, intimate relationships, finance, education, and identity, because LLMs are, in practice, multi domain, inexpensive, always available, and seemingly nonjudgmental. At the same time, from a technical perspective these models rely on transformer architectures, exhibit highly unpredictable behavior in detail, and are fundamentally stateless; conceptually, they lack any real subjectivity, intention, or responsibility. This article argues that the combination of this technical architecture with the social position of LLMs as multis pecialist counselors in an age of human loneliness produces a new kind of advisory intimacy without a subject. In this new relation, model outputs are experienced as if they contained deep understanding, neutrality, emotional support, and user level control, while at the deeper level there is no human agent who is straightforwardly responsible or answerable. By reviewing dominant strands of AI ethics critique, we show that focusing only on developer liability, data bias, or emotional attachment to chatbots is insufficient to capture this configuration. We then explore the ethical and political implications of this advisory intimacy without a subject for policy-making, for justice in access to counseling, and for how we understand loneliness in the contemporary world.