Reliability In Coding Agents
==For coding agents, reliability means the system can produce, verify, and approve code changes in a way that preserves intent, authority, state, evidence, ordering, and recovery across failures, rather than merely pr...
==For coding agents, reliability means the system can produce, verify, and approve code changes in a way that preserves intent, authority, state, evidence, ordering, and recovery across failures, rather than merely producing plausible code once.==[:cite[1]{ln=1}][:cite[1]{ln=2}][:cite[2]{ln=2}][:cite[2]{ln=3}] More plainly: ==the code output is not the main reliability question; the main question is whether the surrounding system gives you a trustworthy result.==[:cite[3]{ln=4}][:cite[3]{ln=5}] The paper says a run can look successful even when the record does not show whether it would reproduce, whether the tests really covered the behavior, or whether the reviewer saw the risky decisions.[:cite[3]{ln=2}][:cite[3]{ln=3}] That is why it treats a coding agent as just one component inside a larger system that includes evaluation, governance, context management, review, and scheduling.[:cite[3]{ln=6}] What reliability means operationally ==A coding agent stops being “just a model” problem once its runs persist, wait on services, compete with other runs, edit shared code, or create external effects.==[:cite[2]{ln=2}][:cite[2]{ln=3}] At that point, reliability belongs to the whole operating system around the model, not to the model alone.[:cite[1]{ln=2}][:cite[2]{ln=3}] The paper defines that surrounding system in terms of harness, execution state, retrieval, memory and state management, permissions, review interfaces, and resource allocation.[:cite[1]{ln=2}] It also says reliability is a ==dependency chain==, not one metric.[:cite[5]{ln=1}][:cite[4]{ln=6}] In that chain, measurement gives a credible comparison, grading gives a defensible verdict, containment and recovery give a bounded recoverable record, retrieval and context give current usable evidence, review and accountability give a challengeable decision, and allocation and cost give a measured commitment.[:cite[5]{ln=2}][:cite[5]{ln=3}][:cite[5]{ln=4}][:cite[5]{ln=5}][:cite[5]{ln=6}][:cite[5]{ln=7}][:cite[6]{ln=1}][:cite[6]{ln=2}] ==Each layer determines what the next layer is allowed to trust.==[:cite[5]{ln=8}][:cite[6]{ln=2}] If evidence is lost upstream, downstream confidence does not fix it.[:cite[5]{ln=9}][:cite[5]{ln=10}][:cite[4]{ln=2}] So what counts as a reliable coding agent? A reliable coding agent system should be able to answer four basic questions: ==what promise the system makes, which component owns that promise, what state survives failure, and what experiment would falsify the guarantee.==[:cite[7]{ln=1}] If you cannot answer those questions, the paper’s view is that you do not yet have a reliable guarantee.[:cite[7]{ln=1}][:cite[7]{ln=2}] In practice, that means: ==Correctness must be grounded in execution, not in confidence or appearance.==[:cite[8]{ln=4}][:cite[9]{ln=4}][:cite[9]{ln=8}] The paper is explicit that a plausible patch is only text until the repository accepts it, because it may compile incorrectly, violate constraints, depend on hidden workspace state, or fail broader behavior outside the visible test.[:cite[9]{ln=5}][:cite[9]{ln=6}] ==Retries should only happen when the system gained new external evidence.==[:cite[10]{ln=1}][:cite[10]{ln=10}] Repeating the same attempt with the same prompt, evidence, and workspace does not count as learning from failure.[:cite[10]{ln=1}][:cite[10]{ln=4}] ==Reliability must be checked against local work, not just public benchmark scores.==[:cite[11]{ln=1}][:cite[11]{ln=4}][:cite[12]{ln=4}][:cite[12]{ln=6}] The paper warns that benchmark success may say little about migration work, security fixes, build repair, or long running feature work unless the workload and operating conditions actually overlap.[:cite[12]{ln=4}][:cite[12]{ln=6}] ==The system must preserve the evidence needed for challenge and review.==[:cite[13]{ln=4}][:cite[14]{ln=2}][:cite[15]{ln=1}] At organizational scale, neither the agent nor the reviewer sees the whole system, so the surrounding system has to record the evidence the reviewer needs.[:cite[13]{ln=4}] ==Reliability includes recovery and control boundaries, not just success on the happy path.==[:cite[16]{ln=2}][:cite[5]{ln=4}][:cite[17]{ln=2}] The paper stresses containment, durable state, recovery, and human oversight because outcome alone does not explain whether success was trustworthy.[:cite[16]{ln=4}][:cite[17]{ln=1}][:cite[17]{ln=2}] A usable checklist If you want the paper’s standard in one checklist, here it is: [ ] Is the acceptance decision based on observed execution or another appropriate oracle, not just the agent’s answer?[:cite[8]{ln=4}][:cite[9]{ln=4}][:cite[15]{ln=1}][:cite[15]{ln=2}] [ ] Does the system record enough evidence to show why the candidate was accepted or rejected?[:cite[3]{ln=3}][:cite[13]{ln=4}][:cite[14]{ln=2}][:cite[14]{ln=3}] [ ] If a retry happens, did some external observation actually change th...