A review of 106 experiments found that human–AI teams often failed to beat whichever performed better alone.
“A human will review it” sounds like a sensible answer to concerns about AI errors. It leaves an important question unanswered: does that review actually improve the decisions? A person can catch a mistake, but they can also accept a bad recommendation or replace a correct one with their own error. A large research review compared human–AI teams with people and AI systems working separately. On average, the teams did not outperform the better of the two working alone. That is a stricter comparison than asking whether AI helps a person who would otherwise have no assistance, and the difference matters.
If a workplace introduces an AI tool, comparing the new process only with the old human process may miss part of the picture. The new combination might be better than the person alone while still being worse than the AI alone on the measured task. Review can remain necessary for accountability and context, but its value should be tested rather than assumed.
Michelle Vaccaro and colleagues reviewed 106 experiments, producing 370 effect-size estimates. Their 2024 Nature Human Behaviour paper compared people working with AI against both people and AI working independently. On average, the combined arrangements performed worse than the better independent option, although they could still improve on people working alone.
The pattern differed by task. Decision-making tasks showed losses relative to the stronger independent option. Content-creation tasks looked more promising, but that subgroup did not establish a statistically clear gain above zero. It would therefore be too strong to conclude that combining people and AI reliably improves creative work.
The original review covered studies from early 2020 to mid-2023 and several kinds of AI, rather than only current chatbots. It provides evidence about the arrangements tested during that period, not a permanent ranking of humans and machines.
Consider an invented test of 100 questions. A person answers 60 correctly, an AI system answers 85, and their combined process answers 75. The system has helped the person get 15 more answers right. Yet the combined process has also produced ten fewer correct answers than the system alone.
Both comparisons are accurate, but they support different claims. If the question is whether the tool helps a person, compare the combination with the person. If the claim is that the team is the most accurate arrangement, compare it with both independent alternatives.
This distinction matters when evaluating a new workflow. Reporting only the improvement over a weak starting point can make the combined process look better than it is. Choosing the comparison in advance also makes it harder to select whichever result looks most favourable afterward.
Imagine an AI system flagging damaged parts in factory photographs. A worker asked to approve its decisions needs a clear view of the original image and enough time to inspect it. Showing only a score and an approval button gives the worker little opportunity to discover why a recommendation is wrong.
The same issue arises when someone reviews a generated report. Access to the original sources is necessary for checking factual claims, and the review needs enough time to use them. A person cannot reliably verify a growing number of claims simply because the workflow assigns them the title of reviewer.
The review step should therefore identify the kinds of errors a person can catch and provide the information needed to catch them. Otherwise, adding a signature at the end may change who is responsible without improving the work.
For a simple newsletter, AI might help group notes into sections. A human might check dates, names, and promises. The job is small enough that each role is visible. For a spreadsheet, the tool might suggest a formula. The person can test it on a few rows with known answers. That is a concrete check. Asking whether the sheet looks sensible is weaker. These are workflow suggestions, not effects established by the review. The idea is to make the collaboration testable. You should be able to point to the step the tool improved and the mistake the person can catch.
If neither role is clear, the process may just move uncertainty from one place to another.
A team may recall one hidden error AI found. It may forget ten small errors AI added. Keep a record of both. For repeated work, sample completed tasks and check them against a reliable answer or rule. Count errors that the tool introduced, errors the person introduced, and errors either side caught. Use the same standard across cases. Do not compare a careful team with a rushed person and call the result a fair test of AI. Time, task difficulty, and access to information all matter. A local check will not settle the question for every workplace. It can still tell you if your own process works well.
A useful evaluation compares three arrangements on suitable cases: the person alone, the tool alone, and the combined process. Record the errors as well as the successes, and examine what reviewers actually changed. That shows whether people are catching mistakes or introducing new ones. Accuracy is not the only reason to keep a person responsible for a decision. Context, appeal, and accountability may require human involvement even when a system scores well on a test. The review's lesson is narrower: adding a reviewer is not, by itself, evidence that the final answer will be better.
