Agent Auditability
Auditable Agents: A Framework for Accountable AI Agent Systems
Agent auditability is the enabling system property through which an agent system produces, preserves, and exposes enough trustworthy evidence to make accountability possible.
Accepted to the ACM AI Leadership Summit 2026, Atlanta, as a condensed 4-page version. An earlier version was presented non-archivally at the ACL 2026 KnowFM Workshop.
LLM agents call tools, query databases, delegate tasks, and trigger external side effects. Once an agent system can act in the world, the question is no longer only whether harmful actions can be prevented; it is whether those actions remain answerable after deployment.
Auditable Agents is a position paper accepted to the ACM AI Leadership Summit 2026, with an earlier version presented non-archivally at the ACL 2026 KnowFM workshop. It defines what auditability means for agent systems, introduces five jointly necessary dimensions, identifies three mechanism classes operating at successive temporal points, and provides layered empirical evidence that the auditability gap is real, closable, and practically addressable.
This work sits at the auditing of action point in the lab taxonomy. It covers the auditability framework and Auditability Card as methodology. It also covers the ecosystem scan and runtime enforcement evidence for concrete agent failure modes and defenses.
Paper (arXiv) | Paper (PDF) | agent-audit | Aegis
Figure: A defensible audit verdict requires five jointly necessary dimensions (center), rooted in Evidence Integrity. Three mechanism classes (bottom) realize these dimensions at successive temporal points. No single class suffices.
The Five Dimensions of Agent Auditability
The framework treats these five conditions as jointly necessary for a defensible post-deployment audit verdict.
- Action Recoverability
- The record captures policy-relevant actions and enough fields to reconstruct what happened.
- Lifecycle Coverage
- The record covers the execution phases and context, including retries, fallbacks, approvals, and delegation handoffs.
- Policy Checkability
- The record contains the fields needed to mechanically determine whether stated structural policies were followed.
- Responsibility Attribution
- The record supports recovery of the responsible component or responsibility chain behind each action.
- Evidence Integrity
- The record has verifiable protections against silent modification, allowing auditors to determine whether the evidence can be trusted.
The framework did not emerge in isolation. It grew out of building tools that address specific auditability gaps, and from studying failure modes that make auditability necessary. Each tool below instantiates one mechanism class; each finding is drawn from the paper.
Detect · Pre-deployment 617 findings
agent-audit is the underlying scanning tool (arXiv). In this paper, we used it to scan six prominent open-source agent projects and surfaced 617 security findings (details in the paper), showing that basic prerequisites for Action Recoverability are widely unmet before agents even run.
Enforce · Runtime 8.3 ms overhead
Aegis is a pre-execution firewall that sits between agents and tools, enforcing policies and generating tamper-evident audit trails (arXiv). Every intercepted tool call gets a signed, hash-chained record covering Action Recoverability, Policy Checkability, and Evidence Integrity in a single pass, at 8.3 ms median overhead.
anywhere-agents
is a practitioner-facing analog of pre-execution mediation for coding workflows: a lightweight
PreToolUse guard that stops destructive Git and GitHub commands
(push --force, reset --hard, gh pr merge) before they run
on the developer's own machine, applying the same mediation pattern at a different scale.
Recover · Post-hoc ~0.95 accuracy
Implicit Execution Tracing embeds agent-specific signals into token distributions, enabling post-hoc recovery of which agent produced which output even when identity metadata and orchestration logs have been stripped. Token attribution accuracy reaches ~0.95 across topologies, directly addressing Responsibility Attribution under degraded conditions.
Why auditability is needed · Failure modes
Cross-user contamination: shared-state agents silently propagate information across user boundaries without any attacker. Without Action Recoverability and Lifecycle Coverage, these events are invisible after the fact.
The Autonomy Tax: defense training degrades agent task performance. This tension motivates why Enforce mechanisms must be lightweight; 8.3 ms overhead is practical precisely because it does not impose a capability tax.
Agent Auditability FAQ
What Is Agent Auditability?
Agent auditability is the enabling system property through which an agent system produces, preserves, and exposes enough trustworthy evidence to make accountability possible.
How Does Agent Auditability Differ From AI Auditing?
Agent auditability is an enabling property of an agent system. AI auditing is the process that uses available evidence to reconstruct behavior, check policy, and assign responsibility.
Why Is Auditability Necessary for Accountability?
Accountability requires an auditor to determine compliance and assign responsibility. Those determinations depend on trustworthy evidence across all five dimensions of agent auditability.
What Are the Five Dimensions of Agent Auditability?
The five dimensions are action recoverability, lifecycle coverage, policy checkability, responsibility attribution, and evidence integrity. Together they support a defensible audit verdict.
The Auditability Card
A compact reporting artifact, analogous to model cards, that forces disclosure of auditability properties. A system can answer these questions well or badly, but it should not be allowed to answer them ambiguously.
Q1 through Q5 map to the five auditability dimensions; Q6 stress-tests what happens when logging assumptions break. The Aegis column is an illustrative partial card, not a canonical answer.