Review checklists for AI-authored changes

Code review checklists for AI-generated pull requests require different validation steps than human-authored changes to catch missing context and assumptions.

When an AI agent ships a pull request that touches twelve files, adds three dependencies, and claims to "implement the feature as requested," you face a different review problem than human-authored code. The PR passes CI, the diff looks plausible, but you have no mental model of the author's reasoning. Traditional code review asks "does this implementation match the spec?" Reviewing AI-generated code forces you to also ask "did the agent understand the actual problem?" and "what constraints did it ignore because I didn't make them explicit?"

The challenge isn't that AI writes worse code—often it's cleaner and more consistent than the average junior engineer. The problem is provenance: you can't assume the agent considered error paths you would have caught in conversation, or that it prioritized the same tradeoffs. A human might comment "went with approach A over B because we need to support legacy clients." The agent just does something that compiles.

Separate "does it work" from "should we ship it"

Traditional review conflates these questions because a human engineer who wrote working code probably thought through the implications. With AI-authored changes, split your review into two passes:

Pass one: functional correctness. Run the code locally. Exercise the new behavior manually if it's user-facing. Check that CI coverage actually exercises the new paths—agents are excellent at writing code that passes shallow tests but breaks on edge cases. This pass answers "did the agent solve the narrow problem?"

Pass two: systemic impact. Now ignore whether it works and evaluate whether this change moves the codebase in a direction you want. Did it introduce a new pattern that will need maintenance? Does it bypass existing abstractions? If five more features get implemented the same way, will you regret the precedent? This is where you catch changes that technically function but create long-term friction.

A common failure mode: an agent adds a new background job to process user uploads, picks a reasonable library, writes clean implementation code—but doesn't consider that your existing job infrastructure already has retry logic, monitoring, and rate limiting. The code works. The change creates a parallel system you'll have to maintain separately.

Check constraint boundaries explicitly

Agents follow instructions literally. If you ask for "a way to export reports to PDF" without specifying performance constraints, storage limits, or format requirements, you might get beautiful code that generates 40MB PDFs in memory and crashes on datasets over 1000 rows.

Build a review checklist that surfaces the constraints you normally communicate implicitly:

Performance expectations: Did the agent add queries in loops? Does it load entire datasets into memory? If this code runs on every request, what happens at 100 concurrent users?
Error handling scope: AI tends to handle errors syntactically (catch blocks exist) but not semantically (does it retry idempotent operations vs. user errors appropriately?). Check what happens when external services are down, when input is malformed, when the user cancels mid-operation.
Data model assumptions: Agents often assume clean, normalized data. Check joins that assume referential integrity without DB constraints. Look for code that assumes unique usernames when your schema allows duplicates with different domains.
Security boundaries: Did it validate user input at the right layer? Does it respect existing permission checks? If it added API endpoints, do they require authentication?

For example, we've seen agents implement "add pagination to search results" by adding LIMIT and OFFSET parameters—correct for small datasets, but missing index hints and potentially causing full table scans at scale. The code works in development. Your checklist catches that production has 10M rows.

Trace dependency changes with extra scrutiny

When an AI agent adds a dependency, it's optimizing for "solve the immediate problem" without the context that humans accumulate about your dependency philosophy. You might have a policy against crypto libraries that aren't FIPS-certified, or prefer native implementations over FFI wrappers for security-sensitive operations. The agent doesn't know.

For any new dependency in an AI-generated change:

1. Check if existing code already solves this. Agents don't have perfect knowledge of your codebase. We've seen PRs that add new HTTP clients when three others already exist, or import date-parsing libraries when the standard library suffices.

2. Evaluate maintenance risk. Look at the library's release cadence, issue tracker, security history. An agent might pick a library with 50k GitHub stars but no releases in two years.

3. Consider bundle size / binary size impact if relevant to your domain. Agents optimize for implementation simplicity, not deployment artifacts.

4. Verify license compatibility. This is usually automated in CI, but agents occasionally suggest dependencies with licenses your legal team would reject.

When the agent adds leftpad to solve a string formatting problem, you catch it here. When it adds moment.js to a frontend bundle in 2025, you ask why it didn't use native Intl APIs.

Validate test coverage against actual risk

AI is excellent at writing tests that mirror implementation structure—unit tests for every function, mocks for every external call. What it struggles with is writing tests that match the risk profile of the change.

Ask these questions:

Do the tests cover the failure modes you actually fear? If the change involves authentication, are there tests for expired tokens, wrong credentials, missing auth headers—not just the happy path?
Do integration tests exercise the seams? Unit tests might mock external services correctly, but does an integration test confirm the actual API contract matches what the agent assumed?
Are there regression tests for constraints you specified? If you asked for "pagination with max 100 results per page," is there a test that verifies trying to request 101 fails correctly?

A practical check: look at the test file additions. If the agent added 200 lines of implementation and 50 lines of tests that only call the new functions once each with valid inputs, the coverage is illusory.

Use compilation and validation as gates, not just signals

In traditional review, CI failures are signals to investigate. With AI-generated code, treat them as hard gates—but expand what "compilation" means. If the agent makes a change that passes TypeScript's type checker but violates your API contract linters, or passes unit tests but fails a smoke test in a deployed preview environment, that's a compilation failure for your purposes.

This is where Goatfied's edit loop becomes relevant: plan -> constrain -> edit -> validate -> retry. The "validate" step isn't just tsc or cargo check, it's the full set of automated constraints you can articulate. If your codebase has custom linters that enforce "all database queries must have timeouts," those run before the PR reaches human review.

The more constraints you can encode as automated checks, the less mental energy you spend catching violations. When an AI-generated PR reaches you, you can assume it passed explicit rules. Your job is catching the implicit constraints you haven't automated yet—and adding them to your validation suite for next time.

Look for "works but weird" patterns

AI agents occasionally produce code that is technically correct but stylistically foreign to your codebase. A change might use promise chains in a repo that's standardized on async/await, or implement a feature with classes in a functional codebase. This isn't wrong, but it creates maintenance friction.

Watch for:

Inconsistent idioms: Different variable naming conventions, error handling patterns, or file organization than surrounding code.
Over-abstraction or under-abstraction: Agents sometimes create elaborate inheritance hierarchies for simple features, or inline everything when your codebase prefers small composed functions.
Comments that explain implementation instead of intent: Human-written comments often say "why"; AI-written comments often say "what." If every comment is a literal description of the next line, they're noise.

These patterns aren't bugs, but they signal that the agent didn't absorb your codebase's implicit style. You can accept the change and refactor, or send it back with specific style constraints added to the prompt for regeneration.