Cohere and US senate move to rein in AI agent risks.

AI vendors are also moving to tackle the practical risks of autonomous software. On 5 October, Toronto-based **Cohere** launched North 2, an upgraded version of its enterprise AI agent platform that “puts AI agents on a budget and gives them a memory” ([1]). The release adds controls that allow companies to set user-specific quotas, rate limits and organisation-wide spending caps for each AI agent ([2]). North 2 also includes persistent memory, enabling agents to maintain context across sessions and draw on shared knowledge libraries ([3]) ([4]). These features directly address two of the biggest hurdles in scaling up AI-driven workflows: unpredictable costs and errors from an agent’s lack of business context ([5]) ([6]).

Cohere’s enhancements reflect a broader industry push to make autonomous agents more reliable for day-to-day business use. Many organisations remain wary of giving AI free rein because of past incidents of runaway prompts and “confidently wrong” outputs ([7]). By allowing human teams to pre-define the rules and memory an agent works with, vendors hope to increase trust in AI co-workers. Effective cost controls and context management can prevent mistakes, like the *"budget-breaking bill"* or nonsense answers that some early agent deployments have suffered ([8]) ([9]). For leaders, the message is that technical safeguards are improving - but selecting AI tools that support transparency and limit unintended actions is critical to harnessing agents safely.

Even with these challenges, businesses that implement AI agents in well-defined processes are starting to see concrete benefits. A June survey of more than 500 enterprise technology leaders by **Anthropic** and research firm *Material* found that **80% of organisations already using AI agents reported measurable economic returns** from those investments ([1]). In practice, this means faster operations and productivity gains. For example, a cybersecurity company cut its average threat analysis time from five hours to just seven minutes by delegating it to an AI agent - and the AI’s findings still matched human experts’ conclusions 95% of the time ([2]). Legal professionals using the *CoCounsel* AI assistant (built on **Anthropic’s Claude**) can now search 150 years of case law in minutes, tapping insight from the equivalent of thousands of expert opinions on demand ([3]).

Importantly, evidence suggests that the biggest performance gains go to those who fully embrace autonomous workflows. According to a global study by **PwC**, nearly three-quarters of all the economic value created by AI so far is being captured by just the top 20% of companies ([4]) - those most aggressive in deploying AI beyond pilot projects. These **AI leaders** are almost twice as likely as others to use agents for multi-step tasks or *“self-optimising”* processes within set guardrails ([5]) ([6]). They also invest more in *“trust at scale”* measures like responsible AI frameworks and cross-functional governance boards ([7]). The early returns are encouraging: done right, AI co-workers can accelerate workflows, improve accuracy and free employees for higher-value work.

However, the majority of firms are not yet ready to let AI agents work fully autonomously. **VentureBeat** research published on 5 October highlights gaps in enterprise oversight: only **21% of companies have a way to automatically enforce spending limits on their AI agents** - the rest rely on after-the-fact monitoring or no tracking at all ([1]). Similarly, **68% of enterprises recently reported that a *“confidently wrong”* answer from an AI assistant was caused by missing or bad business context ([2])**. These statistics underline why many organisations still restrict AI to narrow tasks or require a human manager to handle exceptions. Without strong cost controls, AI workflows can generate unwelcome surprises on the cloud bill; without the right data and context, even the most advanced agents are prone to make mistakes with full conviction.

Business transformation leaders are also grappling with talent and change management challenges. Many firms now plan to invest more in training staff on AI than on the technology itself ([3]). In areas like finance, **94% of organisations intend to keep increasing their AI spending despite uncertain short-term returns** ([4]), but only a small fraction have mature governance models for autonomous agents ([5]). The lesson is that while AI agents can shoulder routine work, companies must develop the internal skills, controls and accountability structures to direct and oversee these new *"digital workers"*. As regulators worldwide signal higher expectations for AI safety and transparency, investing in these capabilities is becoming just as critical as the technology’s raw power.