GPT-6, Hallucinations Reduced but Vulnerable to Document-Based Attacks

OpenAI's latest model "GPT-6 Astra" has demonstrated reduced "hallucinations" where AI generates information that contradicts facts, compared to previous-generation models. Meanwhile, it has also been revealed that the model remains vulnerable to attacks called "indirect prompt injection," where malicious instructions are embedded within documents in a concealed manner.

Prompt injection is an attack technique in which hidden instructions separate from legitimate instructions are sent to an AI model, causing it to perform unintended actions. There are two types: "direct injection," where attackers send commands directly to the model, and "indirect injection," where commands are embedded within documents or web pages that the AI reads. As "AI agents"—systems where AI autonomously reads external information while performing tasks—become more widespread, attention to this issue is growing.

GPT-6 Astra achieves a 99.99% success rate in blocking direct attacks, demonstrating strong robustness in this regard. However, in indirect scenarios where attacks are concealed within documents referenced by AI, results show that attacks succeeded in 8.5% of cases. By comparison, Anthropic's "Claude Opus 5" achieved a 4.8% success rate for the same indirect attacks, lower than GPT-6 Astra.

While 8.5% may appear small on the surface, it represents a significant risk in scenarios where AI agents handle actual business data or personal information. For example, if an AI system autonomously processes large volumes of emails or contracts, it implies that roughly 8 to 9 out of 100 transactions could be compromised by attacks. In real operational environments, a single breach could result in serious information leaks or operational errors.

As AI agent applications expand, opportunities for models to read external content continue to increase. Attack vectors for concealing instructions are virtually unlimited—from email attachments and web page content to shared documents. These indirect attack pathways are harder to defend against compared to direct attacks and remain an unsolved challenge for current leading models.

The comparison between GPT-6 Astra and Claude Opus 5 reveals that companies are engaged in ongoing security competition. Hallucination reduction and attack resistance represent evaluation criteria distinct from model "intelligence" and are increasingly likely to be weighted heavily going forward. Particularly as enterprises begin embedding AI agents at the core of their operations, robustness assessments of this kind will become critical factors in model selection decisions.