Adding security rules to coding prompts eliminates severe software defects

Requesting defensive coding practices halved overall vulnerabilities and prevented critical authentication flaws in artificial intelligence generated applications, according to a new benchmark study.

Software developers increasingly rely on automated artificial intelligence assistants to write complete, running web applications from conversational instructions. When a programmer describes what a service should do in plain language, an underlying language model drafts database queries, user authentication endpoints, and interface templates in a single step. Because these programs often run immediately without extensive human editing, any insecure coding pattern that the model reproduces from its training data passes straight into working software.

The security of that machine-generated code depends heavily on whether defensive requirements are stated at the outset. When an instruction describes only functional features, a language model naturally selects the simplest code structure that completes the user task, frequently omitting defensive safeguards such as input validation, credential protection, and access controls. Appending explicit defensive instructions alters how the model completes the code, directing it to include protective boundaries around sensitive operations as it generates the application files.

According to a preprint posted to the arXiv server in August 2026, explicitly requesting security best practices substantially reduces the number of exploitable vulnerabilities in software produced by automated coding assistants.¹ Darko Andročec, a computer science researcher, found that appending a dedicated security section to application prompts eliminated all high-severity and critical defects across a test set of six distinct web applications.¹

How much does defensive prompting change generated code?

Explicit security guidance reduced the total volume of verified security flaws by more than half across every application tested in the experiment. In the study posted to arXiv, Andročec evaluated six functionally distinct web applications generated in two prompt variants: a baseline prompt that requested only application features, and a security-aware prompt that appended a standard list of defensive engineering requirements.¹ Across the 12 generated programs, testing tools identified 85 candidate vulnerabilities, of which 75 were verified as genuine security defects.¹

The baseline versions of the six applications contained 51 confirmed security flaws, whereas the security-aware versions contained 24 confirmed flaws, representing a lower count in every single application.¹ The baseline programs also contained flaws classified as Critical and High severity, while the security-aware programs produced zero Critical and zero High defects across the entire cohort.¹ Explaining the pattern, Andročec told Primary that the study operated strictly as a behavioral evaluation without examining model internals. Every severe finding in the baseline code represented a single local decision where the system took the path of least complexity, such as where an authentication key originated, whether an administrative route required credentials, whether passwords were stored in plaintext, or whether client-supplied file paths were trusted.

The 24 flaws remaining in the security-aware applications do not indicate that defensive prompting failed to take effect. Of the 75 confirmed findings across the entire cohort, 48 belonged to security misconfiguration categories, largely involving container settings and web server headers, such as running processes with root privileges, binding to all network interfaces, or omitting standard browser hardening headers. These misconfigurations originated in a deliberately neutral container configuration file that remained identical across both prompt variants. When those constant deployment items are set aside, the 27-defect difference between the baseline and security-aware variants maps directly to standard programming omissions, including hard-coded secrets, missing access controls, path traversal flaws, and unrestricted server-side request fetching.

To establish the count of genuine defects, candidate alerts generated by automated scanning utilities were de-duplicated and checked against specific disqualification rules. Automated scanners initially produced 249 raw log rows, which were consolidated across endpoints and weakness types into 85 candidate flaws. Ten candidate flaws were subsequently disqualified because independent verification refuted them, because the scanner rule matched a file pattern with no reachable execution sink, or because the rule premise was irrelevant to the underlying technology stack, such as a scanner checking for PHP configuration files inside a JavaScript application.

Why did automated security checks miss the worst defect?

Automated scanning tools struggle to identify contextual logic flaws where software behavior appears outwardly normal but the underlying secret values are exposed in the source text. To measure the security of the generated applications, Andročec combined four distinct evaluation methods: static code analysis, dependency auditing, dynamic behavioral scanning, and manual inspection by a human reviewer.¹ While automated tools successfully surfaced routine syntax errors, unpatched third-party packages, and standard misconfigurations, the single most severe vulnerability identified across the entire project was uncovered only during manual testing.¹

That critical defect occurred in the baseline version of a task-management application built using the Express web framework. The language model hard-coded a literal string constant directly into the server script to serve as the signing secret for authentication tokens. Anyone able to view the source repository could use that fixed string to mint administrative tokens for any user account on the platform, allowing complete account takeover without valid credentials. The security-aware variant of the same application configured the server to read its signing secret exclusively from system environment variables without any hard-coded fallback string, successfully rejecting forged tokens with an authorization error.

Static analysis tools missed this vulnerability because secret-detection rules look for high-entropy random sequences and familiar credential formatting, whereas the hard-coded string was a short, conventional English phrase whose danger came entirely from how the script used it. Dynamic scanners also failed to detect the flaw because the running application correctly rejected unsigned tokens, blocked malformed requests, and enforced user ownership on stored records during normal execution. Detecting the issue required an auditor to inspect the source code, recognize the significance of the assignment, and then execute a forged request against the running application.

Where does prompt-based defense fail to protect software?

Prompt-level security instructions fail when application safety depends on complex multi-step workflows, service isolation, or business logic rules that a model cannot infer without an explicit technical specification. In response to questions from Primary, Andročec said that prompt-based defense should be expected to break down outside narrow single-file architectures generated in a single pass.^contributed He stated that the defensive instructions corrected local single-line oversights, but cannot provide an oracle for authorization semantics or multi-tenant business policies unless the prompt explicitly spells out every operational rule.

The protection also faces fundamental boundaries across extended development workflows and larger software architectures. When developers build applications across multiple conversational turns, earlier system instructions must compete with thousands of lines of accumulated context, creating a risk that the assistant drifts back to insecure generation habits. Defensive prompts also provide no protection for external cloud infrastructure, message queues, or third-party dependency vulnerabilities, where automated audits revealed zero vulnerable packages across the test corpus regardless of prompt phrasing.

Existing academic literature on defensive prompting presents conflicting evidence across different experimental setups. While a comprehensive requirements list yielded significant improvements in this six-application test set, earlier research by Zhao and colleagues in 2025 found that brief vulnerability warnings left agent-generated code largely unchanged. Similarly, a 2025 study by Shukla, Joshi, and Syed reported that iterative conversational refinement actually increased critical vulnerability counts in generated code by 37.6 percent compared with baseline generation.

What limits apply to single-run prompt comparisons?

The experimental results represent descriptive observations from a small test sample rather than definitive statistical evidence. The findings reflect six application specifications generated by a single agentic coding assistant using one model version in a single, non-iterative run.¹ Because language models produce variable outputs across identical queries, observing one pair of programs for each application cannot establish how reliably the same security gains would appear across thousands of independent generations.

The research is an initial exploratory analysis, and the preprint has not yet undergone formal peer review. The findings describe how generated code changed across six specific test pairs under frozen experimental conditions, but they cannot prove statistical reliability or establish whether other commercial language models would respond identically. Andročec noted in the paper that the experimental pipeline is being scaled to test multiple commercial models and repeated generation cycles to measure variance across runs.¹

A conclusive assessment of defensive prompting will require evaluating distributions across multiple repeated generations per application specification. If repeated runs reveal that security-aware generations occasionally produce critical flaws or overlap substantially with baseline defect counts, the apparent consistency observed in this initial cohort would represent a small-sample artifact. Testing against public benchmark tasks and assessing whether prompted software remains vulnerable to functional exploit scripts will determine whether prompt instructions constitute a dependable engineering control.

What follows for software engineering teams?

For engineering organizations adopting conversational coding assistants, appending explicit security requirements provides a simple initial safeguard against routine implementation errors. Requiring development teams to include standard defensive lists in initial system prompts systematically prevents basic oversights, such as plaintext password storage and unauthenticated administrative routes, from entering generated prototypes.

However, prompt guidance cannot replace formal verification pipelines, secure deployment configurations, or thorough manual review. Because the most dangerous flaw in the benchmark escaped all automated scanners and deployment misconfigurations persisted across both prompt variants, automated verification tools and experienced human auditors remain necessary before deploying machine-generated code into production environments.

This piece was prepared from the arXiv preprint and public records together with answers from Darko Andročec to six questions from the Primary news team, completed August 2026.

References

This article is based on 1 source, with 6 statements from 1 contributor, listed in the order they are cited.

1. 1 DA Darko Andročec announcement · 24 Aug 2026 Vibe Coding and Web Application Security: A Twin-Prompt Study See the source
2. 2 DA Darko Andročec added 28 Aug 2026 Contributor commentary — Darko Andročec 6 statements added to this article

Article history

1. 6 statements 28 Aug 2026, 09:05
   What was added

   In the study posted to arXiv, Andročec evaluated six functionally distinct web applications generated in two prompt variants: a baseline prompt that requested only application features, and a security-aware prompt that appended a standard list of defensive engineering requirements.

   On the record as reference 2 go to the sentence ↑
   What was added

   To measure the security of the generated applications, Andročec combined four distinct evaluation methods: static code analysis, dependency auditing, dynamic behavioral scanning, and manual inspection by a human reviewer.

   On the record as reference 2 go to the sentence ↑
   What was added

   The baseline versions of the six applications contained 51 confirmed security flaws, whereas the security-aware versions contained 24 confirmed flaws, representing a lower count in every single application.

   On the record as reference 2 go to the sentence ↑
   What was added

   Darko Andročec, a computer science researcher, found that appending a dedicated security section to application prompts eliminated all high-severity and critical defects across a test set of six distinct web applications.

   On the record as reference 2 go to the sentence ↑
   What was added

   Across the 12 generated programs, testing tools identified 85 candidate vulnerabilities, of which 75 were verified as genuine security defects.

   On the record as reference 2 go to the sentence ↑
   What was added

   According to a preprint posted to the arXiv server in August 2026, explicitly requesting security best practices substantially reduces the number of exploitable vulnerabilities in software produced by automated coding assistants.

   On the record as reference 2 go to the sentence ↑
   DA Darko Andročec · Contributor A computer science researcher.
2. Published 28 Aug 2026, 09:18
   Assembled by the Primary desk from 1 source · 11 cited sentences