When long-horizon LLM agents fail
A research team at Villanova University in the United States has published an arXiv preprint (arXiv:2611.13254, dated 9 November 2026) that traces how long-horizon large language model (LLM) agents fail from start to finish. After analysing 95 failed runs across six multi-agent architectures, the authors found that the root cause of failure entered at a median of 18% of the way through a run. In 72% of failed runs, the execution history already contained recorded evidence that directly contradicted the error before the run collapsed. In 88% of those cases, no agent in the system acted on that evidence.
The paper is a preprint and has not yet been peer reviewed. Its data come from two public corpora of agent failure logs, known as Who&When and MAD. Even with that caveat, the findings challenge a common assumption about AI agents. Many teams assume that agents mostly fail through late-stage reasoning errors or final-step hallucinations that a reviewer can catch by checking the finished output. This study points to early state corruption, followed by a failure to cross-check that state against evidence the agents already hold.
Where Australian environmental consultancies, law firms, councils and developers trial agentic AI for desktop studies, data handling, document review and report drafting, the question this research raises is where quality control sits in the workflow. The study shows agent errors are typically seeded early, remain visible in the record, and are not corrected by the agents themselves. A review of the finished product alone will often come too late to catch them cheaply.
Early faults, ignored evidence and failed recovery
The study found that faults in failed agent runs enter early, at a median point of 0.18 of the total execution. These early faults were predominantly faulty initial operational decisions or fabrications about the state of the environment the agent was working in. In practical terms, an agent asserts that a file, tool result or data value is something it is not, and later steps build on that assertion. The authors report that these faults occurred well before human evaluators or automated benchmarks flagged a decisive error. Conventional evaluation therefore tends to locate failure at the point of visible collapse rather than at the point of origin.
The most striking finding concerns evidence that was available but ignored. In 72% of failed runs, the trajectory (the logged sequence of agent actions, tool calls and observations) contained concrete evidence that contradicted the faulty belief. In 88% of those runs, none of the collaborating agents registered or responded to the contradiction, and the system continued along the flawed path. The authors conclude that multi-agent systems cannot be relied on to self-correct simply because agents share prompts or context. Agents routinely ignored contradictory data that was already present in their own context windows.
Detection on its own did not solve the problem. In 31% of runs, the system did register that an action had deviated from the expected outcome. The existing control mechanisms then failed to clear or re-initialise the corrupted state, and the errors continued to compound. The authors estimate that restoring the agent’s internal state when a discrepancy was detected would have prevented failure in 35 of the 95 runs analysed, or about 37%. That figure supports rollback to a known clean state as a recovery strategy, as opposed to asking the agent to repair its reasoning within the same conversation.
The paper also identifies a measurement problem in agent benchmarks. In 45% of failure logs from a hand-crafted benchmark, the cause was an external provider content-filter timeout or crash. In those cases the test harness silently emitted a synthetic answer that no agent had generated, and the result was recorded as an agent reasoning failure. To address both issues, the authors formalise an “episode record” protocol that maps each fault across four stages: origin, propagation, detection and consequence. They recommend that multi-agent runtimes include independent trajectory monitors with external state-rollback boundaries, instead of relying on agents reflecting on their own conversation.

Australian context: AI governance and data quality
The paper does not address environmental consulting or Australian regulation directly. Its relevance here lies in professional services governance and in how firms assure the quality of AI-assisted work. The Australian Government’s Voluntary AI Safety Standard, released in September 2024, sets out 10 guardrails for organisations deploying AI. These include testing systems and monitoring them once deployed, and enabling meaningful human control or intervention. The Villanova findings suggest that, for agentic systems, monitoring while the system runs is likely to catch problems that a check of the final output would miss.
Contaminated land practice already has a well-established framework for this kind of discipline. Schedule B2 of the National Environment Protection (Assessment of Site Contamination) Measure 1999, as amended in 2013 (the ASC NEPM), applies the data quality objectives (DQO) process. That process requires practitioners to define what the data must demonstrate, and to what level of confidence, before relying on it. Laboratory data are validated against QA/QC criteria such as relative percent differences for duplicates, blank results and holding times before they are used in decisions. The logical step for firms using AI agents is to treat agent outputs the same way. Extracted values, compiled site histories and summarised records are data that need validation against primary sources before anyone relies on them.
Accountability for AI-assisted work in Australian contaminated land practice remains with the practitioner who signs the report. Using an agent to compile or draft parts of that report does not change where responsibility sits.
References and related sources
- Primary source: arxiv.org
- agitracker.io
- arxiv.org
- https://arxiv.org/abs/2611.13254`
How iEnvi can help
iEnvi integrates technology and data-driven approaches into environmental consulting. We monitor AI and technology developments that affect how environmental professionals deliver services to clients.
This is an iEnvi Machete news summary. Prepared by iEnvi to summarise the source article for environmental professionals tracking AI, data, and technology developments that affect consulting and project delivery.
Published: 14 Nov 2026
Need advice on this topic? Speak to an iEnvi expert at info@ienvi.com.au or 1300 043 684, or contact us online.
Need advice on this issue? iEnvi provides practical, senior-led environmental consulting across contaminated land, remediation, ecology and environmental risk.