Identity and Accountability in the Age of AI Agents
by Nancy Wang
February 24, 2026 - 7 min
Related Categories
AI agents have crossed the line from assisting humans to acting on their behalf. In production environments, that shift changes the security model.
Security leaders say that the main challenge isn’t getting an agent to do something useful. It’s knowing, at runtime, what actions are permitted, which identity it’s operating under, and how to reconstruct what happened afterward. I recently contributed to the 2026: The End of Vibe Adoption whitepaper with researchers from Stanford’s Trustworthy AI Research Lab, led by Professor Sanmi Koyejo, in collaboration with the AIUC-1 Consortium and enterprise security leaders including Jen Easterly (RSAC), Omar Khawaja (Databricks), Mandy Andress (Elastic), Chris Monson (Atlassian), Phil Venables (Google Cloud), Chris Sandulow (Confluent), Gagandeep Singh (Salesforce) and others across industry and government. The research and field experience converge on a simple point: before regulators or insurers can address AI-related harm, enterprises need technical clarity around runtime identity, responsibility, and auditability before deploying agents in critical systems. The gap is not governance intent. It is execution control.
Three challenges shaping 2026
In the whitepaper, we outline three defining challenges for 2026:
The agent challenge
The visibility challenge
The trust challenge
These challenges are interconnected. Each one ultimately stresses the same foundation: identity, access, and accountability must operate at the same speed as autonomous systems.
The agent challenge is about execution. AI has moved from assistant to autonomous actor. Agents don’t just recommend actions; they invoke APIs, retrieve data, create records, modify configurations, and orchestrate workflows across multiple systems.
If that execution is backed by long-lived or over-permissioned credentials, risk scales just as quickly as capability. Traditional access control models were designed for users operating at human speed. However, agents do not operate that way. They execute actions in milliseconds and rely on non-human identities, often acting across systems where accountability boundaries are unclear. Identity must therefore be dynamic, not static. Each agent action should be tied to a scoped, short-lived identity at runtime—eliminating shared credentials, enforcing least privilege at the tool and data layer, issuing access just-in-time with clear expiration, and capturing credential issuance, tool execution, and data access in a unified audit trail where identity is inseparable from action. In production environments, the dominant failure mode is rarely model hallucination. It is over-privileged agents holding persistent access and executing cross-system actions without tightly defined containment boundaries. When prompt injection or logic flaws occur, the blast radius is determined by the existing permissions.
When harm occurs, attribution gets murky
Both the research and practitioner discussions highlight the challenge of attribution.
When a human makes an error, we can usually trace their access and decision path. When a deterministic system fails, logs often tell a linear story.
With agents, harm may result from prompt manipulation, model updates, tool misconfigurations, over-permissioned non-human identities, or subtle changes in system prompts. Increasingly, the research shows that attacks unfold as sequential task-level chains, where no single step appears malicious in isolation. A conversation may gradually lead an agent to retrieve sensitive data or to invoke a tool outside its intended scope. Without structured logging of prompts, retrieval steps, tool parameters, and runtime identity, reconstructing causality becomes extremely difficult. Traditional controls assume static assets and clear ownership. Agent systems are dynamic and distributed across complex supply chains. As Omar Khawaja, VP & Field CISO at Databricks, noted, “AI components change constantly across the supply chain, but security controls assume static assets, creating blind spots, friction, and no clear accountability when behavior shifts.”
If runtime identity isn’t clearly defined and logged, and if tool usage isn’t auditable, attribution becomes guesswork. When attribution is unclear, containment, regulatory reporting, and insurer conversations become significantly more challenging.
Before we debate liability models for autonomous systems, we need a technical foundation that makes responsibility observable.
Visibility is the prerequisite
Many organizations lack full visibility into which AI systems are operating in their environment, what data those systems access, or how their behavior changes over time.
This extends beyond unsanctioned chatbots. It includes embedded assistants enabled by default in SaaS platforms, internal teams wiring together APIs without formal review, and quiet changes to models or prompts that alter system behavior without passing through traditional change management processes. Visibility, in this context, is not just an inventory exercise. It requires mapping non-human identities to specific agents, tracking what data domains those identities can access, and monitoring how agent workflows traverse systems. It also requires logging prompts, tool calls, and outputs in a structured manner that supports both security analysis and forensic reconstruction.
AI components change frequently. Data refreshes, model swaps, and prompt adjustments can quietly introduce new risk. Without continuous monitoring of identity usage and data flows, ownership becomes diffuse and accountability erodes. As Gagandeep Singh, VP, Global Compliance & Certification at Salesforce, observed, “AI risk increasingly manifests as unowned operational decisions, not deliberate security failures.”
You can’t secure what you can’t see. For agents, visibility requires enforcing identity controls at runtime.
From governance principles to technical controls
High-level AI governance frameworks provide important structure, but we also identified a gap: governance principles alone don’t answer the technical questions CISOs are asking. These questions are specific:
How do we validate tool parameters before execution?
How do we detect cascading failures within multi-agent systems?
How do we enforce least privilege for non-human actors?
How do we log and alert on prompt injection attempts in production?
Technically grounded frameworks translate governance intent into specific, auditable requirements. They require that identity boundaries be enforced programmatically, that data access be scoped by policy, and that agent execution paths be testable under adversarial conditions.
From my vantage point, identity and access controls are essential to make that translation possible.
Designing for containment
A key lesson from both the Stanford research and consortium discussions is that model-level safety alone is not enough. As Professor Sanmi Koyejo explained, “Model-level safety is necessary but not sufficient. Fine-tuning attacks, extraction attacks, and indirect prompt injection all bypass many model guardrails.”
That’s why I approach agent deployment with a simple assumption that something will fail. The question is whether we’ve designed the system to contain it. Containment begins with identity. Every agent action should execute under a clearly scoped, short-lived identity. High-risk operations should be isolated. Access to sensitive or regulated data domains should be segmented to prevent cross-context drift.
Containment also requires adversarial testing. Organizations are increasingly testing agent workflows against prompt injection attempts, tool parameter manipulation, data exfiltration scenarios, and cascading multi-agent failures. These exercises evaluate not only output quality but also the integrity of access controls and the resilience of identity boundaries under stress.
The goal is not to eliminate all failure, but to ensure that when failure occurs, the impact is limited and reconstructable.
A foundation before scale
In nearly every board and executive discussion, the focus has shifted from whether to adopt AI to how to scale it responsibly. Regulators are clarifying expectations. Insurers are beginning to evaluate operational controls. Underwriters increasingly ask whether runtime identity is enforced, whether least privilege is applied to non-human actors, whether adversarial testing is conducted, and whether harmful outcomes can be reconstructed from logs. For example, ElevenLabs recently secured insurance coverage for its customer-facing AI voice agents after completing more than 5,000 adversarial simulations spanning privacy, safety, security, reliability, and accountability domains. Those simulations modeled real-world failure modes, from hallucinations to prompt injection, generating risk evidence that insurers could underwrite.
Insurability, in this context, reflects architectural maturity.
But before regulators, insurers, or boards can meaningfully assess AI exposure, organizations must be able to answer who or what acted, under which identity, with what authorization, accessing which data, at what time, and based on which inputs. That requires extending identity, access control, visibility, and auditability to autonomous systems operating at machine speed.
From my perspective, that’s the real inflection point. The era of unstructured experimentation is ending. The organizations that will succeed will build a technically grounded foundation that extends identity, access control, visibility, and auditability to autonomous systems operating at machine speed.
Agents aren’t just another user. They represent a new class of actors in our environments, and our security architecture must evolve accordingly.
