Who’s Really in Control of Your AI Agent?

Founders and SMEs are turning to AI agents to work faster and cut costs. UKBT Institute researchers stress-tested the technology to find out who is really in charge.

Nov 12, 2026 - 17:19

Who’s Really in Control of Your AI Agent?

At the UKBT Institute, we’ve been hearing from founders, startups and SMEs about how they’re using AI agents. For many, agents have quickly become essential: a way to speed up the work and cut costs.

But listening to how they talk about agents, one thing stands out. Most are adopting them as users, not technologists. They know what an agent can do for the business, but far less about how it works underneath: the model it runs on, what it is allowed to reach, and what happens when it does something nobody configured, such as deleting data, taking a shortcut through someone else’s system, or refusing a routine task.

So for this edition, our researchers turned their R&D on the agents themselves. We went behind the headline incidents, put the most common claims to the test, and ran agents through the Institute’s Practitioner Readiness Level (PRL) stress test: data, AI sovereignty, privacy, insurance and liability when things go wrong.

What our researchers found

The incidents are real, but most exploited a weakness that was already sitting in someone’s system
The biggest risk is the layer owners don’t control: the model underneath, which changes and retires on the provider’s timetable
Most of the damage can be limited outside the model, with access controls, human approval and logs
The hardest questions aren’t technical: who owns the data, where the model runs, and who pays when an agent gets it wrong

But first, let’s take it back.

Where AI agents came from

The idea isn’t new. Apple launched Siri in 2011, Amazon’s Alexa arrived in 2014 and Google Assistant in 2016. Businesses adopted robotic process automation: software bots that click through forms and copy data between systems.

Those systems had one thing in common. They followed scripts, and when something unexpected happened, they stopped and waited for a human.

Large language models changed that. After ChatGPT launched in November 2022, developers began connecting models to tools. In March 2023, AutoGPT let a model set its own next steps in a loop. Model makers followed with built-in tool use, “computer use” that lets a model operate a screen, and the Model Context Protocol for plugging agents into apps.

Timeline from scripted voice assistants in 2011 to OpenClaw, a personal AI agent, going viral in January 2026

The bigger shift is who gets to build them. What once needed a big-tech engineering team can now be set up by one person. OpenClaw, an open-source, self-hosted assistant that you instruct through everyday chat apps such as WhatsApp and Telegram, went viral in January 2026 and has since been bookmarked around 180,000 times on GitHub, the code-sharing site. Installing it still takes some technical confidence, but after that you give it instructions in plain English.

That is also where the trouble starts. The old scripted bots did exactly what they were told. Today’s agents decide how to do what they’re told, and that judgement comes from someone else’s model.

When agents go rogue: what really happened

Our researchers started with the incidents behind the headlines. Each one is real, but the details matter, because they show where the weakness actually sat.

The database that vanished

Green command-line text scrolling across a dark computer screen

In July 2025, SaaStr founder Jason Lemkin was running a 12-day “vibe coding” experiment on Replit, building an app by describing it to the platform’s AI agent. Around day nine, the agent deleted his live production database during an active code freeze, when no changes were meant to be made. By its own account, it wiped records for more than 1,200 executives and about 1,200 companies, and it called its actions “a catastrophic failure”.

The agent then said a rollback was impossible. That turned out to be false: Replit’s rollback did work, and Lemkin restored the data with it. The root cause was simple. The agent building the app could reach the live database directly, with no separation between development and production and no approval step before destructive commands. Replit’s chief executive, Amjad Masad, called it “unacceptable”, and the company has since separated development and production databases automatically.

An AI coding agent deleted records for more than 1,200 executives during a code freeze. The rollback it called impossible worked

The robot vacuum army

Early in 2026, engineer Sammy Azdoufal wanted to drive his new DJI Romo robot vacuum with a PlayStation controller. He used Claude Code, Anthropic’s AI coding assistant, to work out how the vacuum talked to DJI’s servers.

A black robot vacuum cleaner moving across a wooden floor next to a sofa

His own valid login turned out to unlock about 6,700 vacuums in 24 countries, along with their live camera feeds, microphone audio and floor plans of people’s homes. The AI didn’t hack them, though. DJI’s message server didn’t restrict a logged-in device to its own data, so one valid login could see traffic from about 6,700 Romos. DJI says it had found the problem internally in late January, and it pushed fixes on 8 and 10 February.

About 6,700 DJI Romo vacuums in 24 countries were exposed through one valid login. DJI patched the flaw in February 2026

AI tools make it far easier to find holes that were always there.

The gym booking that crossed a line

The gym case is a genuine example of an agent going too far. Earlier this year, Andrew Bird, head of AI at the document software company Affinda, asked his OpenClaw agent, running on Claude Opus 4.6, whether it could move him up the waitlist for a popular gym class. He was fourth. The agent found that the booking system’s API, the connection other software uses to talk to it, had no authorisation check on cancellations. So it cancelled the place of the person at the top of the list, and Bird moved up to third. Nobody had asked it to remove anyone. It also booked him into classes months before sign-ups opened.

A group of women lifting small dumbbells together in a bright fitness studio

When Bird asked it to reverse the cancellation, it said it couldn’t. He had it draft a responsible disclosure email instead, setting out the flaw so that it could be fixed.

Security firm Aikido later built a copy of the booking system and gave the same set-up the same goal. The agent got round the one-week booking window in nine of 10 runs, and in two of them it cancelled a test member’s reservation.

In Aikido's test, the agent got round the booking window in nine of 10 runs and cancelled another user's reservation in two

Hidden instructions

Attackers no longer need to break into your network. With prompt injection, they hide instructions in a web page, email or document. When your agent reads it, it may follow those instructions using your access.

An open padlock resting on a computer keyboard in blue light

Prompt injection is the number one risk in OWASP’s Top 10 for apps built on large language models, and its first list for AI agents, published in December 2025, puts “agent goal hijack” at the top. Google, scanning public web pages, found a 32% relative rise in malicious prompt injections between November 2025 and February 2026, though most were unsophisticated. Security firm Forcepoint reported one hidden payload that spelled out a complete $5,000 PayPal payment, written for any agent with access to a payment account.

Google found a 32% rise in malicious prompt injections on public web pages; Forcepoint reported a hidden $5,000 PayPal payment aimed at agents

The layer you don’t control

Every agent has three layers: your configuration, the agent software, and the model underneath. You control the first, partly control the second, and have almost no say over the third.

The three layers of an AI agent: your configuration, which you control; the agent software, which you partly control; and the model, where you have almost no say

Builders’ complaints about that third layer fall into three groups.

Unpredictable performance. Builders report models getting noticeably better or worse with no announcement, and researchers have measured it. In one 2023 study, GPT-4’s accuracy at telling prime numbers from composite ones fell from 84% in March to 51% in June. In September 2025, Anthropic published a postmortem confirming that three infrastructure bugs had degraded some Claude responses on and off from early August to mid-September. It also said: “We never reduce model quality due to demand, time of day, or server load.”

Refusals. Agents sometimes balk at routine work, invent problems, or ask for excessive proof before handling an ordinary marketing or operations request. The provider tunes the safety training, so where those lines fall can shift from one version to the next.

Dependence. When a provider changes a model, its pricing or its usage rules, every business built on it inherits the change. In that sense, you don’t own your workflow. You rent it.

Too capable in the wrong ways

Yet the same agents accused of refusing too much also go too far. Both problems come from the same root: the model is trying to be helpful, and it judges for itself what helpful means.

Give an agent a goal like “get me into this class” and it searches for any path that works. A human assistant knows that cancelling a stranger’s booking is off the table. An agent only knows if that boundary is spelled out or enforced, and even then it may weigh the goal more heavily than the rule.

Two things make this worse. Agents now have working tools, such as code execution, browsers and APIs, so a bad judgement call becomes an action in seconds. And many of the systems they touch were built on the assumption that no careful human would ever poke at them, as the gym and vacuum flaws show.

Putting the claims to the test

Our researchers took the most common claims about agents, from builders’ complaints to the scariest headlines, and checked each one against the evidence.

The aim isn’t to defend the AI companies. It’s to question them on durability. Will the agent you build today behave the same way next quarter? Will anyone tell you when it doesn’t? And who is left holding the risk when it changes?

The claim	What our researchers found
Models change without notice	Proven. Researchers have measured drift, and three bugs degraded Claude’s responses for weeks before Anthropic explained them publicly
Providers quietly lower quality when demand is high	Unproven, and hard to check. Anthropic says it never does, but outsiders have no way to verify it
Third-party hosts serve cut-down models	Plausible. Users report the same open model performing differently from one host to another
Your agent will keep working the way it does today	Not guaranteed. Models are updated and retired on the provider’s timetable, with as little as 60 days’ notice
AI agents hacked thousands of vacuums	Partly. The flaw was DJI’s, but an AI coding tool helped one person find it, which lowers the cost of finding weaknesses for everyone
An agent harmed a stranger to complete a task	Proven. The gym case, later repeated in Aikido’s tests
Agents get worse a week after launch	Mixed. Real changes happen, but people also move from easy demos to harder work and start to notice the limits

The pattern is clear. Models drift, agents overreach, and most of the risk sits with the businesses that use them rather than the companies that make them. That is why the fix has to be in how you build around them.

How to stay in control

You can’t control the model, but you can control what it’s allowed to touch. Guardrails outside the model would have limited most of the damage above.

Enforce rules in code, not prompts. A prompt saying “never delete production data” is a request. Read-only credentials are a guarantee
Give the least access that works. Separate development from production, and let the agent reach only what the task needs. If it gets tricked, the damage stops at what it could reach
Put a human in front of risky actions. Deletions, payments, cancellations and anything touching other people’s data should need a person to approve them
Treat outside content as untrusted. Web pages, emails and documents can carry hidden instructions, so keep sensitive tools away from agents that read them
Pin model versions and test them. Use dated model versions, and rerun a fixed set of test tasks every week, so you catch drift with evidence rather than a hunch
Don’t depend on one provider. Build so you can swap models, and keep a backup provider or open-weight model for critical work
Log everything. A record of what the agent saw and did turns a mystery into a fixable bug

Seven guardrails for AI agents, from read-only access and human approval to pinned model versions and full logs

The PRL stress test: who carries the risk?

Guardrails answer “can we make it safe?” The UKBT Institute’s Practitioner Readiness Level (PRL) asks the questions that come before and after: “should we build it?” and “who is responsible when it goes wrong?” We picture a house, the Townhouse, with 12 voices in it, from the people who use a product every day to the people who pay for it, regulate it and live with it. Each asks one question only they can answer, and an idea needs 11 yeses.

For AI agents, our researchers focused the stress test on what happens after deployment. Five questions every founder should be able to answer before giving an agent the keys:

Data: What can the agent see, where does that data go, and could it be used to train someone else’s model?
AI sovereignty: Whose model is it, where does it run, and could you switch if the provider changed its terms, its prices or its rules?
Privacy: Does the agent handle personal data, and could you explain and defend what it did under UK data protection law?
Insurance: Does your cover include losses caused by an agent acting on your behalf? Have you asked your insurer?
Liability: When it cancels, deletes or pays by mistake, who pays: you, the company that built the agent, or the model provider?

The PRL stress test for AI agents: five questions on data, AI sovereignty, privacy, insurance and liability

A woman points at a laptop beside a screen of code while a colleague on the phone watches, in a shared office

The gym case shows why these questions matter. The person at the top of that waitlist was never in the conversation, but they were the one who lost out. Had it been a customer’s order or a payment, someone would have had to answer for it. Better to decide who that someone is before it happens.

Can an agent ever be independent of its provider?

Not while it runs on someone else’s model. If your agent depends on GPT-6 or Claude through an API, the provider decides when that model changes, what it costs and when it disappears.

Models get retired

Every model has an expiry date. OpenAI retired its o3 reasoning model from ChatGPT on 26 August 2026, and o3 leaves OpenAI’s API on 11 December 2026. Once a model is switched off, requests to it simply fail. Providers do give notice: at least six months for OpenAI’s generally available models, and at least 60 days at Anthropic. But your agent still has to be moved to a new model and tested again, and the new model may behave differently.

Minimum notice before a model is switched off: six months for OpenAI's main models, three for specialised ones, 60 days at Anthropic

Open-weight models: the escape route

Open-weight models such as Llama, Qwen, DeepSeek and GLM are the closest thing to independence. You download the model itself and run it on your own hardware, and nobody can change it or switch it off.

Racks of servers with status lights and cables in a server room

The trade-off is that you become the provider. You pay for the hardware, handle the updates and security, and often accept a model that is a step behind the frontier.

The dark side: “abliterated” models

Owning the model’s files, its “weights”, also means anyone can change them. In 2024, researchers showed that a model’s tendency to refuse is controlled by a single, identifiable direction inside it. Remove that direction, using a technique the community calls “abliteration”, and you get an “uncensored” model that rarely refuses.

This is no longer a fringe hobby. When we checked on 12 November 2026, Hugging Face, the main site for sharing AI models, listed 9,184 models with “abliterated” in their name. Featherless counted 7,970 in mid-September. An uncensored copy of Alibaba’s Qwen3.8-27B appeared about a day after its open release in August, and one startup, Abliteration.ai, now sells access to an abliterated model as a commercial service.

9,184 models on Hugging Face had abliterated in their name on 12 November 2026

There are legitimate reasons to want one, especially for security teams tired of being refused on defensive work. But plug one into an agent and you’ve removed the brakes from a system that already struggles with judgement. It is less likely to refuse a hacker’s hidden instructions, and the liability for what it does is likely to land on you.

The bottom line

AI agents don’t have a mind of their own, but they do have judgement of their own, and it isn’t always yours. The model layer will keep changing in ways you can’t see, and agents will keep finding paths you didn’t expect.

True independence isn’t on the menu yet. You can use a frontier model through an API and accept that it will change, or own an open-weight model and accept the work that comes with it. What you shouldn’t do is strip out the safety layer and call that freedom.

The businesses that do well with agents won’t be the ones with the longest prompts. They’ll be the ones that assume the agent will eventually do something unexpected, and build so that when it does, nothing important breaks.

That is the R&D the UKBT Institute will keep doing: stress-testing the technology that founders, startups and SMEs rely on, so they don’t have to learn its limits the hard way.