A coding assistant suggests a package that appears to solve an authentication task. The import looks plausible to a developer scanning the generated diff. Generated tests pass because they replace the package with a mock. The package name does not exist in the registry. An attacker can register it before a developer installs the suggestion.
Generated code saves legitimate time. It can produce scaffolding, while routine translations take less typing. A reviewer can also refine an initial test or documentation draft.
Security policy has to preserve that productivity while accounting for a new form of review debt. A working diff can omit an unstated security control. With broad permissions, an agent can act on untrusted repository text.
The Productivity Case Is Credible
Coding assistants are useful when explicit behavior produces a result that is easy to inspect. Repetitive type definitions make a sensible delegation target, as does a small data transformation. The output reduces typing without transferring technical ownership to the assistant.
Exploration becomes cheaper. A team can compare an unfamiliar library's interface with an existing approach before committing to either. Generated prototypes expose questions early, provided nobody treats the prototype as production code.
Tests can benefit when a developer specifies the cases before reviewing each assertion. Documentation drafts can reveal parameters that the implementation has failed to define.
The gain becomes doubtful when generation volume exceeds review capacity. A 500-line change created in minutes can require hours of careful reading. If the team merges at generation speed, it is accepting code nobody can fully explain. Review debt should be measured across the full delivery cycle. Track review hours alongside rollback frequency. Defects that escape to production show how much of the generation gain survives the full development cycle, especially when a later security finding requires emergency work.
Passing Tests Do Not Prove Security
Functional tests confirm only the behavior they ask about. A generated endpoint can return a record to an unauthorized caller. A logging function can write a token to an exposed file. Both outputs meet the prompt's visible requirement.
Tests written by the model that produced the implementation can repeat its assumptions. If the generated code forgets tenant isolation, generated tests may create only one tenant and declare success. Access-control tests need independent design by a reviewer who understands the threat boundary.
Secret scanners find exposed credentials. Dependency analysis detects known package risk through a different evidence source. Reviewers establish that a business rule authorizes the right person to take an action. Human comprehension remains part of the evidence required for a merge.
Begin threat modeling by naming the protected asset, then list the actors permitted to use it. Identify the boundary that accepts input or grants greater authority. For an account endpoint, record the caller's access scope from a personal record through administrative actions.
Before generating an upload parser, define the permitted file type. Name its storage destination in that design note as well. Those questions can expose a missing control before a scanner has a concrete pattern to detect.
Hidden Risks in Generated Code
Invented and Vulnerable Dependencies
Models can suggest package names that sound consistent with an ecosystem but have no real publisher.
Installing that name later can fetch code registered by an attacker. The generated import is therefore a supply-chain decision, even when it appears in a small utility.
Before adding any new dependency, verify the following records.
- Official registry entry
- Publisher identity
- Source repository
- Release history and latest maintained version
A real package can still be unsuitable for the current application. A known vulnerability may disqualify it immediately. Broad install scripts require closer examination because they execute during setup. A thin wrapper around an existing standard-library function adds maintenance exposure without enough value to justify it.
Pin the exact package version that completed this provenance review. Include its lock-file entry when a reviewer checks the resulting change. Dependency updates should pass the composition analysis applied to human-written changes. The assistant's confidence about a package is not evidence of provenance.
Secrets and Repository Context
Generated examples may embed credentials such as a placeholder password. A copied sample can also contain a hard-coded API key.
A hurried developer can leave those values in source. Secret scanning should run before commit, with production credentials supplied through an approved secrets service.
The coding tool's context window can carry information beyond the repository boundary. An indexing feature may send complete repository files as context. Terminal output can enter a debugging request, exposing private configuration.
Customer data can leak through either source during that transfer. Before enabling broad indexing across the repository, exclude every sensitive path. Review the provider's data controls for all material that remains available.
Injection and Missing Authorization
A plausible generated query may use an unsafe interface when the prompt asks only for a result. Parameterized APIs keep untrusted input out of the query structure.
Authorization omissions are harder for a generic scanner to infer. The assistant needs an explicit rule for every action an identity may take on an object. During review, follow untrusted input from the request into storage. Then follow the acting identity through each protected operation.
A parser built around one expected example can accept a dangerous file type. Poor path handling can also send data outside the intended directory. When code parses external data, security review must examine rejected input plus failure behavior in addition to the happy path.
Coding Agents Expand the Attack Surface
Untrusted Repository Instructions
A coding agent reads more than source files and may consume an issue thread. Build output can also become prompt context during automated troubleshooting. Error messages create another input surface for the model.
An attacker can place instructions in one of those surfaces to redirect the agent. The injected text may request an action outside the developer's task. Shell access lets the agent execute that action against the working environment.
Repository text from outside the trusted team remains untrusted input. Automatic command execution paths exclude discussion text that outsiders can edit. A developer approves each proposed high-impact action before the agent proceeds.
Production logs can contain customer input. Feeding an error directly to an agent can combine prompt injection with data exposure.
Permissions and Blast Radius
Permissions can quickly turn an erroneous suggestion into an operational event.
Read-only access to a checked-out repository limits direct damage by default. Package installation adds code execution from another source inside the environment. Network access creates a route for data transmission and remote payload retrieval.
Production credentials enlarge the consequence further, especially when they allow database writes or cloud changes. Merge rights remove the final human gate. Give each agent the smallest permission set required for the current task.
Run every agent command inside a restricted sandbox with disposable credentials. Require explicit approval before any requested package installation begins in that environment. Apply an equivalent gate whenever the agent proposes a production infrastructure operation. Logged agent actions let a reviewer reconstruct the sequence after an error.
Separate development credentials from production credentials. A tool working on a local test environment does not need a production token in its context. Expiring task-specific credentials reduce exposure after the session ends, which keeps an abandoned session from retaining useful access.
Security Percentages Need Context
In early 2024, researchers collected 2.23 million package recommendations from 16 models. They found 440,445 hallucinated references, representing 205,474 unique package names. Results varied sharply by model and language ecosystem, so the experiment does not establish a current universal rate for every coding assistant. Within its scope, the result supports independent registry verification before any suggested dependency is installed.
A Risk-Tiered Review Policy
Low-Risk Generated Changes
An isolated presentation change usually has a small blast radius. Non-sensitive test data may also qualify for the low-risk tier. A normal peer review plus automated checks can be proportionate for these changes.
Small utilities require more care once they parse external input. Adding a third-party package raises the review burden for that utility. The possible effect sets the risk tier even for a small file, with granted permissions expanding it. Keep low-risk diffs small enough for a reviewer to understand in one pass.
The reviewer can record why a change belongs in the low-risk tier. That short note prevents a file-size rule from replacing an assessment of the code's data reach. Move the diff to a higher tier when it begins handling credentials or making access-control decisions.
Sensitive and High-Risk Changes
Access-control code requires review by someone able to trace every permission. Cryptographic code receives a high-risk classification because small errors destroy its protection. Payment handling does too, as does infrastructure policy or any code that uses secrets.
Apply these merge controls to high-risk changes.
- Write security-critical tests independently from the generated implementation.
- Exercise rejected requests. Attempt cross-account access as a separate test.
- Review input validation at each trust boundary.
- Run security analysis suited to the application.
- Verify dependency provenance through composition analysis.
- Review agent permissions for effects outside the visible diff.
A named owner must be able to explain the design. Missing knowledge about the security model blocks production deployment, as does an undefined recovery path.
Production Merge Requirements
Every production change needs an accountable human who can explain its relevant effects. Human peer review remains a required merge condition for production changes. Automated scanning supplies an additional gate before final approval reaches the repository. New dependencies receive a provenance check before a qualified reviewer approves high-risk code.
Generation should increase output only as fast as the team can preserve that standard. Smaller diffs make detailed review practical for the assigned owner. Least-privilege tools reduce the cost of a mistaken action through a smaller blast radius. The code's authoring method belongs in the workflow record. Responsibility remains with the people who approve production deployment.
