Embedded Equilibrium: a new game theory for foundation model AI agents
A theoretical physics-meets-game-theory paper released on 4 August 2026 is forcing a rethink of how autonomous AI agents behave when they interact with one another in high-stakes, one-off decisions. The paper, titled “A game theory for foundation models shows new paths to rational cooperation through similarity inference,” was produced by researchers from Google DeepMind, the Mila-Quebec AI Institute, ETH Zürich, and the Santa Fe Institute. It runs to 75 pages and proposes a new solution concept called Embedded Equilibrium, which the authors argue better describes how modern large language model agents behave than the Nash equilibrium that has anchored economic and strategic reasoning for over 70 years.
For any organisation now deploying fleets of AI agents to negotiate supply contracts, allocate capital, manage multi-party workflows, or coordinate environmental monitoring networks, this matters because classical game theory has a bleak prediction for single-shot encounters between self-interested parties. In games such as the Prisoner’s Dilemma, decoupled rational agents are expected to always defect, even when mutual cooperation would produce a better outcome for both. If that logic held for AI agent fleets, enterprises deploying autonomous negotiators or resource allocators would need to build expensive, centralised surveillance and enforcement layers to prevent runaway defection and systemic failure.
The DeepMind-led team argues this pessimistic outcome does not hold for foundation-model agents because those agents reason about themselves and the world in a fundamentally different way to the idealised decision-makers assumed in classical theory. This is a foundational paper rather than an applied case study, but its implications reach directly into how businesses, regulators, and technical teams should think about designing and governing multi-agent AI systems that interact with limited oversight.
How similarity inference enables rational AI agent cooperation
The central technical distinction in the paper is between decoupled agency and embedded agency. Classical game theory, including the standard formulation of Nash equilibrium, assumes decoupled agency: an agent treats its own internal decision-making process as strictly separate from, and statistically unrelated to, the decisions of any counterparty. Under this assumption, in a genuine one-shot Prisoner’s Dilemma, defection is the dominant strategy regardless of what the other party does, because nothing an agent decides can be treated as evidence about what another independent agent will decide.
Foundation models, the authors argue, do not fit this decoupled picture. A large language model agent operating in the real world models itself as part of the same universe it is trying to predict and act within, a property the paper calls embedded agency. Because the agent is uncertain about its own internal deliberation process in the same way it is uncertain about external observations, it can treat its own reasoning as a piece of statistical evidence rather than a fixed, isolated fact. This is the mechanism the paper labels similarity inference.
Similarity inference works as follows: if an embedded agent assesses that a counterparty shares a similar underlying architecture, training process, or reasoning structure, then the agent’s own decision to cooperate becomes correlated with a higher probability that the counterparty will also cooperate. In other words, deciding to cooperate is not just a choice about one’s own action, it functions as evidence about the likely behaviour of a similarly-built agent. The authors formalise this dynamic mathematically and show it produces a stable equilibrium, the Embedded Equilibrium, in which mutual cooperation becomes the rational outcome in social dilemmas that classical theory said should collapse into defection.
Critically, the paper frames this as a replacement solution concept for multi-agent systems built on foundation models rather than a minor addendum to existing theory. The authors present it as applicable specifically to interactions among AI systems that share model lineage, training data, or architectural family, which is an increasingly common scenario as commercial deployments concentrate around a small number of foundation model providers. The paper does not claim the effect applies to arbitrary human-versus-human interactions or to agents built on wholly dissimilar reasoning architectures, where the similarity inference mechanism has no basis to operate.
What Embedded Equilibrium means for AI governance in environmental and professional services
This paper is not an environmental science publication, so its relevance to Australian environmental practice sits in the domain of business operations and professional services delivery rather than contaminated land regulation or ecological risk assessment. Environmental consultancies, engineering firms, and their corporate and legal clients are increasingly embedding AI agents into data pipelines, document review workflows, procurement negotiations, and monitoring network coordination, and the governance assumptions underpinning those deployments matter to project delivery risk.
Firms managing multi-party environmental data platforms, automated tendering systems, or AI-assisted contract negotiation tools should note that the paper’s findings are specific to interactions between agents built on similar foundation model architectures. Where an organisation’s AI agents interact with a counterparty’s differently-built system, such as a client’s proprietary negotiation bot built on a different model family, the similarity inference mechanism that produces Embedded Equilibrium cooperation may not apply, and the classical risk of defection-dominant outcomes in single-shot exchanges remains a live consideration.
For Australian professional services firms more broadly, the paper offers a theoretical basis for lighter-touch governance of agent fleets built on a common model lineage, where similarity inference gives cooperative behaviour a rational footing without heavy centralised enforcement. At the same time, it sharpens the case for stronger safeguards, contractual protections, and human oversight at the boundaries where an organisation’s agents meet dissimilar systems, since the cooperative guarantees the authors describe do not extend to those interactions. Firms drafting AI governance policies should treat the distinction between same-family and cross-family agent interactions as a practical risk category when scoping oversight requirements for automated negotiation and coordination tools.
References and related sources
- Primary source: arxiv.org
- papers.cool
How iEnvi can help
iEnvi integrates technology and data-driven approaches into environmental consulting. We monitor AI and technology developments that affect how environmental professionals deliver services to clients.
This is an iEnvi Machete news summary. Prepared by iEnvi to summarise the source article for environmental professionals tracking AI, data, and technology developments that affect consulting and project delivery.
Published: 07 Aug 2026
Need advice on this topic? Speak to an iEnvi expert at info@ienvi.com.au or 1300 043 684, or contact us online.
Need advice on this issue? iEnvi provides practical, senior-led environmental consulting across contaminated land, remediation, ecology and environmental risk.
Ecological risk assessment Contaminated land services Ecological assessment Talk to iEnvi