Separate unclear business decisions from technical limits before choosing what an agent should handle.
What you'll learn
- Test supported inputs, missing context, changes and reply branches.
- Use a clear brief as an acceptance specification.
- Verify actual actions and safe human handover.
- Choose a smaller task or another tool where it fits better.
AI agents can fail because the task is unclear, the needed information is missing or the implementation cannot carry it out. Separate those causes before choosing a fix. A clear procedure helps define success; the connected agent still has to demonstrate it.
Why do vague tasks make AI agent failures harder to diagnose?
A vague task leaves the team without a shared expected result. Define the action, inputs and exceptions so failures can be investigated. Even a precise task can exceed the agent’s capabilities or fail through its connections, permissions or handling of information.
Suppose the task is to draft a follow-up after a chosen five-business-day wait. Specify the approved original send, recipient, later replies, open status, contact restrictions and reviewer. That makes a candidate workflow testable; it does not make it reliable merely by description.
"Handle customer communication" is simple enough to feel like it should be straightforward. It fails in production because "handle customer communication" is not a task. It is a broad category with different inputs, outputs and exceptions that need separate decisions.
Underspecification is one possible cause. The agent may misinterpret clear instructions or fail to use available information too. Test the actual outcome and inspect the inputs, actions and failures rather than treating every error as a missing line in the brief.
What are the four failure patterns in production?
Use these four patterns as a checklist of task and workflow gaps. They are illustrative categories, not a measured ranking or an exhaustive account of agent failure. Investigate the actual cause, including technical failures, when a result is wrong.
Ambiguous inputs — The agent receives something it wasn't designed for. A customer writes in a language the agent wasn't briefed for. An order arrives with two line items instead of one. A form is submitted with a required field blank. The input isn't wrong — it just wasn't anticipated. The agent produces output that looks correct but isn't, because the case it encountered was never defined.
Context-dependent judgment — The task requires knowing something the agent was never given. "Follow up with this lead" sounds defined. But the right tone depends on how the lead arrived, how long they've been in the pipeline, and whether they had a difficult exchange last quarter. A colleague may know that history from prior work; confirm rather than assume it is shared. An agent cannot reliably supply a private history it has never received.
Moving scope — The task definition drifts because the business changes. The agent was briefed for how the workflow worked in February. By April, the team added a step, changed a field name, or started routing a new case type. Nobody updated the brief. The agent continues running the old version of the workflow.
Multi-party coordination — The task involves waiting on another person or system, then acting based on the response. "Send a proposal, then follow up if no response" seems simple. But what if the prospect replies with a question instead of a decision? What if they respond to the wrong thread? The agent was briefed for one path. Production contains several.
The examples above are possible scenarios. Use these signals to test whether each applies:
| Failure mode | Root cause | Detection question | Production symptom |
|---|---|---|---|
| Ambiguous inputs | An input type or safe handling rule may be missing | Are supported inputs and the unknown-case route clear? | Anticipated messages handled well; unusual inputs produce outputs that look correct but miss context |
| Context-dependent judgment | Relevant context may be missing or incorrectly used | Does the right response depend on something not stored in the agent's data sources? | Outputs are technically correct but tone, priority, or framing is wrong for the specific case |
| Moving scope | Brief was not updated when the workflow changed | Has any step, field, or case type changed since the brief was written? | Agent runs the old workflow on new inputs; errors appear random because they depend on which new case type arrived |
| Multi-party coordination | Workflow branches based on external party responses | Does any step require waiting for a reply and acting differently based on its content? | A demonstration omits branches that appear when real replies vary |
Which tasks look automatable but regularly break?
Clear requirements make failures easier to test; they do not remove technical limits.
Break broad customer communication into testable tasks, with named actions, sources, approval and human handover.
Inbox management, scheduling, summaries and lead replies need more detail than their short names suggest. Check the decisions inside each task before deciding what the agent should handle.
"Manage the inbox" — Every message is different. An agent can handle a specific message type — refund requests, delivery questions, account changes — when that type is isolated and specified. The whole inbox is not a task. It is a category.
"Schedule meetings" — Looks mechanical. Contains preference logic. What if two time slots are available but one falls right before a client call the agent has no visibility into? What if the other party prefers mornings and is in a different time zone? Ask the task owner which preferences matter and where they are available. Test whether the implementation actually follows them; do not rely on either a person or agent to guess missing preferences.
"Summarize this week's activity" — What counts as this week? Which activities matter? For which audience? A founder summarizing for themselves includes different items than a summary going to an investor. The agent needs a defined scope and a fixed template — not a general instruction.
"Monitor and respond to leads" — Monitoring is automatable. The response depends on how the lead arrived, what they said, and what stage they are at. Define the response branches, information and approval requirements. Test that a changed client status or reply changes the action correctly.
What are the workarounds for each failure pattern?
For each pattern, define the expected behaviour and investigate what prevents it. The repair may involve business rules, information, connections, permissions or the model. A clearer brief is useful, but its safeguards must work in the actual implementation.
How do you fix ambiguous inputs and context-dependent judgment?
Ambiguous inputs and context-dependent judgment are fixed by defining every case type, then giving the agent the context it needs as structured data it can read.
For ambiguous inputs: Enumerate every case type the agent will handle and every case type it will not. Write an explicit rule for "unrecognised input" — what the agent does when it receives something outside its defined cases. Test unfamiliar inputs before live use and keep a route for new ones afterwards. The agent needs a defined action for them before go-live, not a discovered one after. Consider a narrower scope with a tested handover for unsupported cases. Measure accepted results, review effort and consequential errors before deciding whether that boundary helps. A larger automated share alone does not determine usefulness.
For context-dependent judgment: Extract the context the agent needs as structured data and give the agent read access to it. If the right follow-up depends on how long a lead has been in the pipeline, store that as a CRM field. If tone depends on a previous interaction outcome, log it as a deal note. The agent applies context it can read — it cannot infer history it was never given. For relationship-sensitive communication, a human approval step on the agent's draft preserves the relationship layer while keeping the drafting benefit. If using OpenClaw, verify that the actual send operation waits for the intended approval. A general command-approval setting does not by itself control every outward message.
How do you fix moving scope and multi-party coordination?
Moving scope and multi-party coordination are fixed with a review cadence set before go-live and a workflow split at the reply boundary.
For moving scope: Schedule a brief review cadence before the agent goes live. Choose a review cadence for the task’s change rate and consequences; review relevant changes before they take effect. When a step is added, a tool changes, or a new case type is introduced, the agent owner updates the brief before the change reaches production. Workflow changes that skip this step appear in the agent's outputs as unexplained errors — because the agent is running instructions for a workflow that no longer matches production.
For multi-party coordination: Split the workflow at the reply boundary. The agent handles the outbound action: sending the proposal, the follow-up, the confirmation. A separate rule monitors for replies and routes them. Expected replies — acceptance, booking, approval — continue the workflow. Unexpected replies — questions, objections, out-of-scope requests — go to a human queue. Test that ambiguous replies stop and reach the right person. A routing rule still needs to recognise the boundary and handle failed handover.
How do you separate missing documentation from technical limits?
Write down the expected result, then test whether the implementation can produce it under the required conditions. A new colleague’s difficulty following the procedure can reveal missing context. Successful human rehearsal does not establish that the agent has the same capabilities, sources or access.
Anthropic’s December 2024 engineering guide describes agents’ added cost and potential for errors to compound as they act. It recommends testing and safeguards.[1] Treat clear documentation as a test specification, then examine actual actions and outcomes rather than assuming that a described process is technically solved.
| What to inspect | Undocumented process | Complex but documented process |
|---|---|---|
| Can an unfamiliar colleague understand the decisions? | Explanation or observation may expose missing rules | Rehearse cases to find remaining gaps |
| Is the trigger consistent? | Identify what event starts the task | Test that the actual event starts the correct task |
| How are edge cases handled? | Agree actions and receiving owners | Verify defined branches and failed handovers |
| What does implementation require? | Resolve requirements alongside access investigation | Check capabilities, connections and permissions against the requirements |
| What determines the build plan? | Unresolved decisions and technical dependencies | Demonstrated reusable work, remaining dependencies and tests |
Document the trigger, steps, decision points, output and edge cases. Those are five useful headings, not a limit on length or complexity. Include rare consequential cases as well as normal exceptions and make the unknown-case route explicit. Link to detailed rules when a short page would conceal essential conditions.
Use a colleague’s walkthrough to improve the agent brief. Then test the connected workflow with permitted representative cases and meaningful exceptions. If the colleague needs an unstated fact, document it; if the agent fails despite a clear rule, investigate the technical cause and maintain the safe limit.
How do you use these limits before you commit?
Use four screening questions to identify the evidence and controls the task needs. The workflow rehearsal can expose missing decisions before building; implementation tests reveal whether the planned behaviour works.
Which inputs are supported? Name the sources, meaningful variations and handling for missing or unrecognised information. Do not use the length of a description as an input-coverage test.
How is the result accepted? State expected outcomes and needed review. Some checks can be automatic; others require a person. Human review can be a valid part of the task, not evidence that the task is necessarily underspecified.
What happens to exceptions? Record occurrence, consequences and staff handling effort. Investigate changes against volume and case mix rather than applying a universal weekly exception cap.
Who receives a case the agent cannot resolve? Specify the safe state, receiving person, context and response when handover fails. Test the route; do not assume an instruction guarantees that every uncertain case will be recognised.
Use the answers to choose a limited trial, a different tool or a human-led task. Missing definitions can be repaired, but a clearly described task may still be unsuitable for the available agent or too costly to operate safely.
What should you use when an agent is not the right tool?
Not every workflow that fails the screening questions should be abandoned. Some belong with a different tool or a narrower scope.
| Scenario | Why the agent fails | What to use instead |
|---|---|---|
| Variable inputs with no definable pattern | Scope cannot be bounded — every message is different | Human reviewer + narrow agent for a defined message subset |
| Judgment that requires undocumented context | Context not available as structured data | Human with agent-drafted content for review and approval |
| Workflow changes faster than its rules and tests are maintained | Current rules, tests or ownership are missing | Rule-based automation or human-led checklist |
| Multi-party workflow with branching replies | Agent cannot handle reply variation without explicit routing | Agent for outbound + human escalation protocol for unexpected replies |
| Relationship-sensitive communication | Tone risk exceeds the drafting benefit | Agent drafts, human approves before send |
| Consequential regulatory or contractual decisions | Required authority and accountability need case-specific assessment | Qualified human decision with a limited, tested support task where appropriate |
Ask for the smallest useful, verifiable outcome. Anthropic’s guide distinguishes predictable predefined workflows from agents that choose their steps.[1] Use a fixed rule where the route is known; consider an agent only where its flexibility helps and its outcomes can be checked. Both need suitable data, permissions and maintenance.
Frequently asked questions
What are AI agents actually bad at? Agents can struggle with ambiguous tasks, missing context, changing workflows and complicated coordination. Clear requirements help you test the result, but technical failures can remain. Inspect actual outcomes, permissions, connections and safe handover for consequential cases.
What is underspecification in an AI agent? The brief leaves important requirements, input boundaries, exceptions or handover decisions unclear. Clarify them with the task owner and use them as acceptance criteria. The agent may still fail with clear instructions, so investigate the observed cause.
Why can't AI agents handle open-ended instructions like managing an inbox? That phrase leaves message types, actions, priorities and exception decisions unclear. Define the supported subset and required review, then test actual results. A narrow description makes the task testable rather than guaranteeing reliable execution.
What should every AI agent brief include? Specify inputs, expected outcomes, permitted actions, required approvals, exceptions and a named handover. Agree how results are accepted and what happens when the agent cannot complete the task. Test those arrangements in the connected system.
Notes
- Building effective agents. December19,2024. Accessed October 15, 2026.
