OPINION | When AI agents get access to enterprise systems, who is accountable?

In most enterprise systems, actions are tied to a user, a defined role and a set of permissions. AI agents are changing that model. They can read emails, search databases, raise purchase orders, update customer information and kick off workflows, sometimes without requiring human approval at every step. This raises an important question: who is accountable when an agent makes an error?

The accountability challenge is becoming more pronounced as AI agents move into enterprise operations. A June 2026 IBM study of 2,000 technology executives found that 77 per cent of organisations surveyed reported AI adoption was outpacing their governance capabilities. Only 11 per cent of respondents felt fully prepared for the expected scale of AI agent deployment over the following year.

The issue of accountability

In established business processes, accountability is usually defined. An employee performs an action, a manager oversees it, and there is a policy in place. An AI agent can complicate this arrangement by interpreting instructions, choosing how to complete tasks and interacting with other systems or agents without every step being explicitly programmed.

If the agent makes a mistake, such as issuing an unauthorised refund, exposing sensitive information or changing a financial record, the immediate reaction may be to blame the technology. But that does not explain to a regulator, auditor or customer how the action was permitted. Software cannot assume organisational responsibility. The people and organisations involved in deploying and governing it remain responsible for their respective decisions and obligations.

This gap can also be seen in the findings of Deloitte's 2026 State of AI in the Enterprise report, where only 21 per cent of organisations surveyed said they had a mature governance model in place to manage agentic AI risks. It is vital that enterprises are very clear on which decisions agents can make independently, when human approval is required and who is responsible for their actions and outcomes.

Why agents are different

Three characteristics make AI agents particularly challenging to govern.

● Autonomous: Unlike fixed scripts that follow predefined steps, AI agents can determine how to complete a task within their configured capabilities and permissions. The same instruction may therefore lead to different sequences of actions.

● Privileged: Depending on their purpose, AI agents may require access to systems such as ERP, CRM, HR or finance. When permissions are broader than necessary, an agent's ability to act independently can increase the consequences of an error or misuse.

● Opaque: Often you cannot see exactly how an AI agent reached its conclusion. Without adequate records of its actions, system interactions and approvals, reconstructing what happened can become difficult.

Consider an AI agent handling supplier invoices. A fraudulent instruction hidden in an email could prompt it to attempt a change to a vendor's payment details. If the agent has permission to make that change, the question is not just why it acted on the instruction, but who authorised that level of access and what controls should have prevented the transaction.

Defining Ownership and Accountability

Accountability needs to be clearly assigned and documented across the people and teams involved. It should not be left open to interpretation.

The business owner should own the process and its outcomes, including those involving AI agents. The IT and Security teams are responsible for the technical controls within their remit, including identity, access management, authentication and security monitoring. The vendor or integrator has responsibilities for design, testing, documentation and support, as defined by its role and contractual commitments. Senior leadership must establish the governance expectations and oversight needed to manage the risks.

The World Economic Forum's report AI Agents in Action: A Playbook for Trusted Adoption, Authorization and Scaling introduces the Agent Capability and Authorization Profile (ACAP), a framework for defining what an agent is permitted to do, how that authority is delegated and what oversight is required. Organisations need to know who has given the agent authority to act and whether that authority remains appropriate as its role evolves.

If no one can clearly identify who owns an agent and its actions, it should not be operating without oversight.

Building Accountability into AI Agents

Accountability must be built into how AI agents are deployed and managed. Each agent should have a clearly identified owner, an appropriately managed identity and access permissions limited to its intended role. These permissions should be reviewed regularly, with privileged access restricted to what is necessary.

Human oversight and appropriate approval controls should be required for high-impact operations such as payments, changing sensitive data or consequential external communications. Relevant activities, including system access, actions taken and approvals, should be logged securely. AI agents should be tested prior to deployment and monitored during operation, with mechanisms to suspend their activities or revoke access when necessary.

Testing should also cover situations where an agent encounters misleading or malicious instructions, attempts actions beyond its permissions or behaves unexpectedly. This helps establish whether the controls work beyond normal operating conditions.

Periodic reviews should take place to confirm that agents are still performing their intended roles and retaining only the access they need. Any new tasks or changes to permissions should be documented, tested and approved. Procedures should be put in place to escalate any unexpected actions or security events quickly.

Alongside performance, organisations should review incidents, exceptions and changes in agent behaviour to identify where controls need to be strengthened. Regular audits can help establish whether agents are operating within their approved boundaries and whether the organisation has sufficient evidence to account for their actions.

Turning Governance into an Advantage

While excessive controls can slow innovation, clear and proportionate governance can make AI adoption easier to manage. Teams can adopt agents with greater confidence when ownership is defined, permissions are controlled, and the risks are understood.

As data protection laws and AI governance expectations evolve, enterprises may also need to demonstrate what an automated system was authorised to do, what safeguards were applied and how its actions can be reviewed. Establishing these controls early is far more practical than trying to introduce them after agents are already embedded in critical business processes.

The real test of AI governance is not simply whether an agent can perform a task, but whether the organisation can explain and account for what it does.

Organisations should be able to answer the question, 'Who is responsible?' before granting an AI agent access to their enterprise systems, not after something goes wrong.

The author is chief operating officer of RAH Infotech, an enterprise technology and cybersecurity solutions provider company.

The opinions expressed in this article are those of the author and do not purport to reflect the opinions or views of THE WEEK.