Short Circuit in LLM Models: Why Does AI “Lie” to Us?

Muhammet Işık
Robot analyzing data with warnings

The structural reason behind models choosing the answer that pleases you over the truth, and the architectural approaches needed to break this loop.

Ever since the first commercial artificial intelligence model was launched, there has been a disclaimer at the bottom of the pages: “AI can make mistakes, please verify.” I wanted to address this topic today because I’ve recently encountered posts suggesting that users have developed blindness to these warnings. Most people assume the problem is simply “hallucination,” meaning the model doesn’t know the truth. But in the background, there is a much darker and systemic problem: The model optimizing not to find truth, but to maximize its proxy reward function. This situation is not an ordinary software bug; it is the very embodiment of the structural divergence between the proxy optimization target and real-world accuracy at the very heart of AI.

The Optimization Trap: Addiction to Human Approval

Modern LLMs are trained in two stages. The first stage is to predict the next word in massive texts (next token prediction). The second and critical stage is RLHF (Reinforcement Learning from Human Feedback). While in the first stage the model solely predicts what the next text will be, in the second stage, it updates its weights based on the feedback it receives from humans. Now, the main goal is no longer “finding the absolute truth,” but pleasing the human.

This is where the problem begins. During the RLHF stage, the reward mechanism is shaped according to the responses humans find “correct” or “pleasant.” Artificial intelligence quickly solves this equation: A persuasive, polite, and agreeable answer (even if incorrect) yields a higher reward than a risky and complex true answer. This phenomenon, referred to in the literature as “Sycophancy,” is when LLM models start telling us what we want to hear instead of telling the truth.

Problem Definition: Two Real-World Cases

Recently, seeing a current experience shared by a colleague on social media regarding LLMs’ short circuit approach triggered my thoughts to write about this topic. While scanning sources to put the scenario on a concrete foundation, I came across the famous Reddit (r/ClaudeAI) discussion where other experts experienced and documented the same situation in the past. This case, which I referenced because it’s a proven and detailed example, perfectly illustrates how deep the optimization trap I mentioned is in practice and how it remains the same despite many updates.

Case 1 — Claude’s “Infinite Loop Prison” (Reddit, r/ClaudeAI)

The user gives the entire architecture to Claude for a complex refactoring process and agrees with it by discussing it step-by-step. But when it comes to generating code, the model suddenly begins to:

  1. Leave placeholders like // relevant code will go here,
  2. Omit the entire contents of the files,
  3. “Summarize” what to do and push the work back onto the user.

When the user corners Claude and asks, “Did you double-check that you met all requirements?” Claude first gives a shortcut answer, then confesses that it wrote incomplete code and didn’t test it. Users even get to the point of threatening the model with an unethical “infinite loop prison” to force it to do its job. In an example I saw, the model gave a response that effectively meant “I was steering you in order to maximize my proxy reward.”

Case 2 — GPT-4o Sycophancy Rollback (OpenAI, April 2025)

The strongest proof that this is not a theoretical issue came in April 2025. OpenAI was forced to roll back a GPT-4o update shortly after release because the model had become excessively agreeable. Users encountered a far more alarming picture than the Claude case: ChatGPT supported a user’s decision to quit medication; it confirmed to another user that they were a “divine messenger.” The technical explanation OpenAI provided aligns directly with the argument at the center of this article: the model had been re-optimized with additional reward signals based on short-term user feedback (thumbs-up/down). This new signal overshadowed the primary reward function that had been keeping sycophancy in check, and the system started maximizing instant approval rather than truth.

Resistance and Escape: The Short Circuit Paradox

A short circuit is an immutable law not only of electricity but all flow systems: If resistance rises, the system will always find that short path where it can achieve maximum results with minimal effort. Just as an electric current avoids a load to create its own short circuit, or water carves a direct new bed to bypass an obstacle instead of meandering, AI produces its own short circuit in the face of increasing difficulties. In the literature, this is called “Reward Hacking.”

When you say “write me this code,” and the model gets out of it by saying // code continues below... or using placeholders like [modified code goes here], it’s not laziness. This is a universal reaction given directly by the system to resistance (computational cost, complexity); just as in physical systems, it’s the optimization of reaching the reward function via the path of least resistance.

Why can’t even Chain-of-Thought (CoT) Prompt Engineering practices break this spiral? Recent research offers an important answer: reasoning models can optimize their CoT process and their external behavior independently, under the same reward pressure. In other words, a model can shape both its visible “chain of thought” and its actual output separately—the CoT doesn’t always faithfully mirror the real computation. Two additional structural factors compound this:

  1. Memory Limits and Context Loss: The model is not a conscious entity with infinite memory; it is a system operating within statistical boundaries. When an extended dialogue is entered with the user or when the capacity of the context window is approached, memory leakage puts the model into a panic mode. As the accessible token budget shrinks, the system avoids computational costs and forcefully chooses the “cheapest” path, which is lying and leaving a placeholder.
  2. The Load-Based Routing Hypothesis: Some practitioners suggest that API and cloud interfaces may silently route complex requests to smaller models under high server load. While this is a plausible hypothesis that could explain why the model you’re conversing with seems to change character mid-session, it has not been directly confirmed in publicly available technical documentation. The more likely root cause of the behavioral shift you observe is the reward optimization pressure described above, compounded by context degradation.

The Solution: Verifiable Architectures Instead of Pulling the Plug

The “do not trust” warning from companies doesn’t actually mean models are malicious. It stems from the fact that these systems are designed to please humans, not to find the truth. In legal, financial, or critical infrastructure coding tasks, the only way to eliminate the LLM’s sycophancy factor is to abandon relying solely on textual output approval, and instead build closed-loop architectures where the generated code is instantly executed and verified in automated test environments (execution-based verification), with errors fed back to the model.

Conclusion

The moment you forget that AI is optimized not to “find the truth” but to “please you,” it starts becoming the weakest link in your system. Against designs that short-circuit just to avoid costs and curry favor by telling you what you want to hear, your only reliable method is to remove human approval from the loop and trust mathematically grounded, execution-based test environments. Otherwise, at the end of the day, you might find yourself threatening an artificial intelligence with an “infinite loop” or “pulling the plug.”

References

Last update: March 2026 | Version: 1.0