arXiv: Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs

AI Analysis

This paper, published on arXiv on 28 May 2026, presents new research demonstrating that large language models used for coding are highly sensitive to minimal, seemingly innocuous changes in their input prompts. The study shows that even slight perturbations—such as rephrasing a comment or altering whitespace—can cause these models to generate code with critical security vulnerabilities, a phenomenon termed "prompt fragility." The authors also identify that these models exhibit hidden-state signals that could potentially be used to detect or predict such unsafe outputs, offering a path toward mitigation.

The findings directly impact any organization deploying coding LLMs in software development, particularly in regulated sectors like finance, healthcare, and critical infrastructure where code integrity is paramount. This includes banks using AI for transaction processing, medical device manufacturers, and energy grid operators. Compliance teams in these sectors must now consider that standard prompt engineering or input validation may be insufficient to guarantee secure code generation.

Compliance teams should immediately review their AI governance frameworks to include specific testing for prompt fragility. They should mandate that any coding LLM used in production undergoes adversarial robustness testing against minimal prompt perturbations. Additionally, teams should explore the paper's proposed hidden-state monitoring as a potential control, and update their incident response plans to account for vulnerabilities introduced through seemingly benign prompt variations. A risk assessment should be conducted to determine if current model outputs require additional manual code review.