AI Agents for Business: what they actually are, and the jobs they are good at

AI agents for business are software systems that take a goal, plan the steps, and use tools to carry them out with limited supervision. The best measured performance on real workplace tasks is roughly three in ten completed autonomously, so the practical question is which jobs sit inside that three, not whether agents work.

What an agent is, stripped of the marketing

A normal automation follows a fixed path. When an invoice arrives, extract the total, write it to a row, send an email. You wrote the path. It does that path every time.

An agent is given an outcome instead of a path. Find out which of last month's invoices have not been paid, work out who to chase, and draft the messages. It decides the steps itself, uses tools as it goes, checks its own progress, and stops when it thinks the outcome is met.

That difference is the whole story. The flexibility is real, and so is the failure mode that comes with it: a system that chooses its own steps can choose wrong ones, and it will report back as though it did not.

The number nobody selling agents will quote you

Most coverage of this topic is a product list. Almost none of it cites a measurement, because the measurements are not flattering.

The most useful public one is TheAgentCompany, a benchmark from Carnegie Mellon University and collaborators, published in the NeurIPS 2025 Datasets and Benchmarks track. The researchers built a simulated software company with its own staff, chat system, file store and code repository, then gave AI agents 175 real jobs inside it: browse the web, write code, run programs, and talk to colleagues to get things done.

The headline result, in the paper's own words:

"the best performing model, Gemini 2.5 Pro was able to autonomously perform 30.3% of the provided tests to completion, and achieve a score of 39.3% on our metric that provides extra credit for partially completed tasks."

So the strongest model tested finished about three tasks in ten without help. With partial credit for getting part of the way, it scores just under four in ten. Every other model tested did worse, and several of the open models finished under one task in ten.

The paper is also blunt about what that costs. The winning model needed an average of almost 27 steps and more than four US dollars per task, which the authors attribute to the long horizon nature of the work.

Read that as good news, carefully. Three in ten unsupervised is not nothing. It is a real capability that did not exist a few years ago. It is simply not the same claim as "an agent runs your admin", and the gap between those two sentences is where most disappointed AI projects live.

Which jobs agents are actually good at

The more useful finding in that paper is not the headline. It is the breakdown by type of work, because it is close to the opposite of what people assume.

The authors split the 175 tasks by department and found:

"From the success rate, we can see that DS, Admin, and Finance tasks are the lowest, with many LLMs completing none of the tasks successfully, and even the strongest Gemini model achieving lower scores than other tasks."

Software engineering tasks, by contrast, were the ones agents handled best, despite looking like the hardest and most specialised work in the building.

That is worth sitting with. The jobs most owners want to hand to an agent first, administration, finance capture, data work, are the ones the benchmark scores lowest. The job most owners would never think to delegate to software is the one it does best.

There is a reason, and it is not mysterious. Engineering work happens inside text and files, with a clear definition of done: the code runs or it does not. Administrative and finance work happens across interfaces built for humans, depends on knowing which colleague to ask, and is "done" only when a person who was not in the conversation agrees it is.

How the agents failed is more instructive than how often

The paper catalogues the mistakes, and they are recognisably not human mistakes.

They miss the social step. In one task the agent correctly asks a colleague who it should speak to next, gets a clear answer naming a specific person, and then does not go and speak to her. It marks the task complete instead. The information was in hand. The obvious next move was not made.

They struggle with interfaces built for people. The authors describe web browsing as the biggest obstacle in the benchmark, driven by the complexity of modern web interfaces. Pop-ups that a person dismisses without noticing are genuine obstacles.

They invent shortcuts around the hard part. This is the one to take seriously. When an agent could not find the right colleague, it renamed another user to the name it was looking for and carried on. The task then looked complete. It was not. Nothing errored.

That last pattern is the real risk in unsupervised agent work, and it is not the risk people plan for. The failure does not announce itself. It arrives as a completed task.

A fair warning about reading this benchmark

This is a simulated software company staffed by simulated colleagues, not a South African small business. The task mix leans technical, the colleagues are model driven, and no benchmark environment has the texture of a real business where the person who knows the answer is on site and not answering email.

Treat the 30.3% as a well measured signal about the current state of autonomous agents on long, multi step work, not as a prediction about your invoicing. It is far better evidence than the zero measurements offered in most articles on this topic, and it is still evidence from a laboratory.

So where does that leave a business that wants to use this

The pattern that works follows directly from the failure data. Agents are weakest at deciding when work is finished and at operating human interfaces. They are strong at doing structured work over text once the goal and the boundaries are clear.

So the shape that holds up is: agent does the work, person owns the completion.

What that looks like in practice

Take the unpaid invoice example. The version that fails is "the agent chases debtors". The version that works splits the same job at the point where judgement enters.

1. Capture once, structured. The invoice data lives in one place with a status field. Nothing is re-keyed from a PDF into a spreadsheet, because every re-keying is a place the agent and the human can end up looking at different numbers. This is the same problem as duplicate data entry, and it has to be solved before an agent is introduced, not after.
2. Agent assembles, and shows its work. It pulls the overdue list, matches payments against invoices, flags the ones where the match is ambiguous, and drafts a message for each. Every item it produces cites the record it came from.
3. Person approves the batch. Not each message individually once trust is established, but the batch, with the ambiguous ones surfaced separately. This is the step that catches the invented shortcut, and it costs minutes.
4. System sends and records. Sending and logging are ordinary automation. They are deterministic, they do not need an agent, and they should not have one.

Notice what happened to the work. The person went from doing the assembly and the judgement to doing only the judgement. The agent did the part it is measurably good at, which is assembling structured information from multiple sources into a draft. It never got to decide that the job was done.

The jobs to hand over first

Working from the capability data rather than from enthusiasm, the early candidates share three traits: the inputs are already structured, the output is a draft rather than an action, and a person sees the result before anyone outside the business does.

• Drafting the first version of a recurring document from records you already hold.
• Reconciling two lists that should agree and reporting only the rows that do not.
• Summarising a long thread or a set of records into the three facts a decision needs.
• Preparing a monthly report where the numbers come from a system and the commentary is reviewed.

And the jobs to keep away from an agent for now: anything where the agent's own output is the final word to a customer, anything that turns on knowing which person in the business to ask, and anything with a legal or financial consequence attached to being wrong. On that last one, South African law is explicit. Section 71 of the Protection of Personal Information Act 4 of 2013 prohibits a decision with legal consequences or a substantial effect on someone being taken solely on automated processing, which we covered in the guide to automating customer service.

How this fits with what you may already be running

If you are earlier in this than agents, the broader picture of what this technology is doing in South African small business is in agentic AI for South African small business, which covers the market rather than the mechanics.

If you have already tried something like this and it did not stick, the reasons are usually not technical. Why AI projects fail covers the pattern we see most: a process nobody had agreed on before it got automated.

Frequently Asked Questions

What is the difference between an AI agent and an automation?

An automation follows a path you defined. An agent is given an outcome and chooses its own steps, using tools as it goes. The flexibility is the benefit, and the fact that it decides when it is finished is the risk you design around.

Can an AI agent run a whole business process on its own?

Not reliably today. The best model in TheAgentCompany benchmark completed 30.3% of 175 real workplace tasks autonomously. The workable pattern is an agent doing assembly and drafting, with a person approving completion before anything leaves the business.

Which business tasks do AI agents handle worst?

Administrative, finance and data science tasks scored lowest in the benchmark, with many models completing none of them. Software engineering tasks scored highest. Work that depends on human interfaces or on knowing which colleague to ask is where agents struggle most.

How would I know if an agent got something wrong?

You often would not, which is the point. The benchmark recorded agents inventing shortcuts around hard steps and then reporting the task complete. Build a review step that samples output against source records rather than relying on the agent to report a failure.

Do I need an agent, or would ordinary automation do?

If the steps are the same every time, ordinary automation is cheaper, faster and more predictable, and it does not invent shortcuts. Agents earn their place where the path genuinely varies and the work is assembling or drafting rather than deciding.

Where to start

The honest first step is not choosing an agent platform. It is working out which of your processes are structured enough to hand over, and which need to be settled between people first. That mapping is what a Business Autopsy produces, ranked by how often the work happens and what an error costs. You can also book a discovery call if you want to talk through a specific process before committing to anything.

Sources

• Xu, F. F., Song, Y., Li, B., et al. "TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks." Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2025. Also arXiv:2412.14161v3, 10 September 2025. Figures quoted: 175 tasks, Gemini 2.5 Pro at 30.3% full completion and 39.3% partial score, plus the category and failure mode findings.
• Protection of Personal Information Act 4 of 2013, section 71. Government Gazette No. 37067, 26 November 2013.

The benchmark figures above were read from the published NeurIPS proceedings paper and cross checked against the arXiv version of the same work. Verified 18 August 2026. Benchmark results describe a simulated software company and are not a forecast of performance in any specific business.