AI Writes Code. Engineers Deliver Systems.

Enterprise software delivery has a new drafting layer. Assistants can produce service stubs, UI components, migration scripts and test scaffolds at a speed that would have looked implausible a few years ago. That capability is useful. It is also easy to misread. Writing code and delivering a production system are not the same activity. One produces text that compiles. The other produces behaviour an organisation can operate, secure, change and defend when something goes wrong. For engineering leaders, the distinction matters because accountability does not transfer to the model that suggested the diff. It remains with the team that merged it.

Introduction

Most organisations now have engineers using AI assistance daily. That is rational. Routine boilerplate, unfamiliar API surface area and first-pass refactors are legitimate places to save time. The risk appears when leadership treats generation volume as delivery progress, or when review standards quietly loosen because the code “looks finished.” Fluent output can hide weak assumptions: a plausible method name that does not exist, a dependency chosen for convenience rather than supportability, a happy-path implementation that never meets peak load.

This article walks through why generated code is not automatically production ready, then examines the practical failure modes that matter in enterprise estates — security, maintainability, business logic, architecture, testing and accountability — before closing with recommendations teams can apply without ceremony or hype.

Why generated code is not automatically production ready

A model optimises for plausible continuation given the prompt and the context it can see. It does not own your production invariants. It has not sat in the incident review for last quarter’s payment double-charge. It does not know that a field is write-once for audit reasons, or that a partner API rejects retries after sixty seconds. When those constraints are missing from the prompt — and they usually are — the assistant fills the gap with patterns that look correct in isolation.

Hallucinated APIs are a common example. An assistant invents a client method, a webhook event name or a SDK option that matches naming conventions in training data. The code reviews cleanly if nobody checks the vendor documentation. Integration tests fail later — or worse, a mocked suite passes and the defect reaches staging. Dependency risks follow a similar pattern: the model suggests a package that solves the immediate problem, without regard to licence posture, release cadence, known CVEs or whether your organisation already standardises on another library for the same job.

Performance issues are quieter. Generated code often chooses the readable algorithm over the operable one: N+1 queries across an order history endpoint, unbounded in-memory aggregation for a reporting job, synchronous calls in a request path that already sits near its latency budget. Technical debt accumulates when these drafts are merged at volume without the same scrutiny applied to human-authored changes. The system compiles. The organisation inherits the cost.

Production readiness is therefore a property of the change after engineering work — review, validation, hardening and ownership — not a property of the tool that drafted it.

Security risks

Security defects in generated code are dangerous precisely because they can look conventional. An assistant may store a session token where many tutorials store it, log a request body that includes personal data, or construct a SQL string by concatenation because the surrounding example did. None of those choices require malice. They require only incomplete context and a reviewer who trusts fluency.

In enterprise delivery, the failure modes that matter most are familiar: broken authorisation on “internal” endpoints, overly broad service credentials, secrets embedded in configuration examples that somehow survive into a branch, and missing audit events on actions that move money or change access. AI acceleration increases the rate at which such drafts can appear. It does not increase the rate at which threat models update themselves.

A practical pattern: a team asks an assistant to “add an admin export for customer records.” The draft returns a CSV endpoint behind a role check that only validates authentication, not the finer permission required for bulk export. Unit tests assert that a logged-in user receives 200. The authorisation gap ships until compliance or a customer incident surfaces it. The corrective discipline is unchanged — security review of privileged paths, least privilege by default, and tests that assert denial as carefully as success.

Maintainability

Maintainability is where generated volume collides with long-lived systems. Assistants are good at local solutions. They are weaker at preserving a coherent house style across services unless that style is explicit in documentation, lint rules and examples. Left alone, teams accumulate parallel helpers for the same concern, inconsistent error shapes and copy-pasted validation that drifts apart over successive prompts.

Technical debt here is not abstract. Every duplicated retry policy is a future incident with two behaviours. Every unexplained magic constant becomes a mystery during onboarding. Every “temporary” wrapper generated to bridge two modules becomes a permanent seam nobody wants to touch. AI does not create that problem uniquely — but it can create more of it per sprint if merge criteria reward speed over clarity.

Strong teams treat generated diffs as proposals that must meet the same maintainability bar as human work: clear ownership, readable structure, minimal novelty in dependencies, and comments only where intent is non-obvious. If the draft cannot be explained by the engineer who merges it, it is not ready.

Business logic validation

Business rules are institutional knowledge. A wholesale pricing exception, a refund window tied to fulfilment state, a multi-entity accounting constraint or a clinical scheduling rule is not something a model can invent safely from a thin ticket. It can express a rule once a human has specified it. It can also invent a plausible rule that is wrong in ways that look minor until finance, operations or a regulator notices.

Validation therefore belongs in the engineering process, not in hope that the prompt was complete. Acceptance criteria should name the states that matter. Scenario matrices should include partial failures and conflicting updates. Product owners and domain experts should review behaviour on critical paths, not only screenshots of the happy path. Where AI drafts the implementation, humans still certify that the behaviour matches the business.

Example: an assistant implements “cancel order if unpaid after 48 hours.” The real rule excludes orders already allocated to warehouse pick waves and treats partial payments differently by market. The generated job cancels aggressively. Warehouse and customer-service load rises. The defect was not a syntax error — it was missing business validation before merge.

Architecture review

Architecture review becomes more important when implementation is cheap. Teams can scaffold services, queues and adapters quickly. Without deliberate boundaries, they create architectural drift at higher speed: chatty synchronous chains, shared databases across domains, and “AI feature” modules that reach into every system of record because the prompt asked for convenience.

Review should ask structural questions before debating style. Who owns this data? What happens when the dependency times out? Is this call synchronous because it must be, or because the draft made it easy? Can this change be rolled back without corrupting state? Does this introduce a new dependency that the platform team must now support?

Performance and operability sit inside architecture, not beside it. A generated batch job that scans full tables nightly may pass functional review and still exhaust shared database capacity during month-end. An event consumer without idempotency may look correct until a broker redelivers. Architecture review is where those risks are cheaper to catch than in production.

Automated testing

Automated testing is the control that scales with generation volume. If assistants increase the number of changes per week, verification capacity must rise with them — or risk compounds. Generated unit tests are useful starting points and unreliable finishing lines. Models often assert the implementation they just wrote, missing the business failure modes you care about.

Enterprise suites still need contract tests at service boundaries, integration coverage against real behaviour, regression protection for money, identity and permission paths, and load or soak checks where latency and contention matter. For any AI-facing behaviour, evaluation sets and monitoring matter as much as classical assertions. The point is not ceremony. The point is evidence that the system does what the organisation claims under conditions that resemble production.

A useful rule: if a path can move money, personal data or operational state, do not accept “the assistant wrote tests” as assurance. Require human-owned scenarios, including denial cases, and keep those scenarios in the same review as the implementation.

Human accountability

Tools do not hold production accountability. Engineers and their leaders do. When a change causes an incident, customers, auditors and executives will not accept “the model suggested it” as an explanation. The merge author, the reviewers and the owning team remain responsible for what runs.

That accountability should be visible in process. Pull requests need a human owner. Critical paths need named reviewers. Release notes need an engineer who can explain behaviour. On-call rotations need people who understand the system well enough to act — not only people who can re-prompt an assistant during an outage.

AI can expand what a competent team supervises. It cannot absorb the duty of care that comes with operating software inside a real business. Organisations that keep that distinction clear get the productivity benefit without confusing drafting assistance for delivery ownership.

Practical recommendations

Engineering teams can adopt AI assistance without lowering the production bar. The following practices are deliberately ordinary — because ordinary engineering controls are what keep generated code safe enough to ship.

• Treat every AI draft as a proposal. The merging engineer must be able to explain the change and its failure modes.
• Verify APIs, events and SDK options against current vendor documentation — do not trust invented method names.
• Constrain dependencies: prefer approved libraries, check licences and advisories, and reject one-off packages without an owner.
• Require security review on authentication, authorisation, secrets, logging and any bulk export or admin path.
• Keep architecture decisions explicit: ownership, timeouts, idempotency, rollback and non-functional budgets before large merges.
• Validate business rules with domain owners on critical flows; write scenario matrices that include partial failure.
• Raise automated testing with generation volume — contract, integration and denial-path coverage, not only happy-path unit tests.
• Measure delivery by production outcomes and lead time, not by lines generated or pull requests opened.
• Preserve human accountability: named owners, competent review, and on-call engineers who understand the system.

Conclusion

AI writes code. That statement is now mundane in many engineering organisations. The work that still decides whether software is fit for enterprise production is less glamorous and more consequential: security review, maintainable structure, validated business logic, architectural judgement, automated verification and clear ownership after release.

Generated drafts can accelerate competent teams. They cannot replace the engineering system those teams operate. Hallucinated APIs, risky dependencies, silent performance problems and accumulating technical debt are not abstract concerns — they are ordinary failure modes when fluency is mistaken for readiness. Production systems remain the responsibility of the engineers who deliver them. That is not a nostalgic preference. It is how accountability works when software runs the business.