Perplexity AI's Performance Puzzle: Are You Getting the Model You Expect?

The reliability of artificial intelligence platforms and the models they deploy has become a critical topic, especially when users observe discrepancies between advertised capabilities and actual performance. A recent deep dive into user experiences with Perplexity AI brought these concerns to the forefront, highlighting a noticeable decline in output quality and inconsistent model self-identification.

The Challenge of Model Identity

One of the initial points of contention revolves around how Large Language Models (LLMs) identify themselves. It's often unreliable to ask an LLM directly about its underlying model, as its response is heavily influenced by its system prompt. An AI provider can configure a system prompt to make any model "think" and declare itself to be a specific, even fictional, entity. For instance, if a platform uses an OpenAI API, they can embed "You are GPT-5" in the system prompt, and the model will parrot that identity, regardless of its true architecture.

This manipulation can lead to confusion. While one model might correctly identify as "Claude, an AI assistant created by Anthropic," another, supposedly Grok-4, might internally process as "an AI language model by OpenAI, specifically based on GPT-4 architecture," yet present a final answer identifying as "Perplexity’s AI assistant" using a mix of GPT-4o and Claude 4 Sonnet. Such varied and sometimes conflicting self-declarations underscore the difficulty in trusting a model's word alone.

Alarming Quality Deterioration

Beyond self-identification, the core concern raised by users is a severe and recent degradation in output quality. Queries that previously yielded insightful and correct answers—even simple technical requests involving PostgreSQL and Node.js—are now producing "absolute garbage." This isn't a subtle shift; users describe the new outputs as characteristic of "super super small 3B models," completely unlike the expected performance of advanced models like GPT-5. Crucially, when these same queries were run on alternative platforms such as ChatGPT's GPT-5 and Anthropic's Sonnet, they generated sensible and accurate responses, further cementing the perceived decline in Perplexity AI's quality. This observed drop in quality is distinct from the model's self-proclaimed identity and serves as a more reliable indicator of underlying changes.

Unpacking Potential Reasons

The rapid and significant decline in quality, particularly when it occurs suddenly, often prompts speculation about its causes. One compelling hypothesis suggests that Perplexity AI might be conducting A/B testing with different model configurations or potentially implementing cost-saving measures. This theory gains traction when considering recent strategic moves, such as a partnership with a major telecommunications provider to offer free annual subscriptions to a large user base. With hundreds of millions of new users potentially acquired, there's a strong incentive to reduce operational costs, which could involve swapping out more expensive, high-performing models for smaller, cheaper alternatives, especially for certain user segments or geographical regions. While the observed quality deterioration is an empirical fact for the affected users, the exact reasoning remains speculative without internal data from the platform.

Conclusion: A Call for Transparency

The situation highlights the importance of transparency in AI service provision. While the technical intricacies of LLM deployment are complex, users ultimately value consistent, high-quality performance. When that performance significantly degrades, and model identities become opaque, it erodes trust. For users, the most reliable metric for evaluating an AI platform remains the quality and utility of its generated outputs, rather than its self-declarations. Companies leveraging these advanced technologies must balance user experience with economic realities, ensuring that service changes are communicated clearly, and quality expectations are managed effectively.