Summary

For business owners commissioning a custom AI agent, this guide follows the work from brief to operating handover. It explains what to specify, test and accept at each stage.

What you'll learn

  • What the workflow brief must decide before development
  • How to distinguish granted permissions from message filters
  • Which decisions, approvals and outputs customization changes
  • How to test normal cases, unusual inputs and actual tool outcomes
  • How to define acceptance measures, ownership and post-launch changes

Say a services firm wants an agent to handle client intake emails: read the message, check the CRM, draft a response and put the draft in a review queue. During testing, a renewal request from an existing client is treated as a new enquiry. The brief never explained where those categories separate.

The repair starts with a business decision. Agree which requests belong in the workflow, which record settles the client's status and when the agent must ask a person. Then test the implementation against that decision. Better wording alone cannot supply a policy your team has not agreed.

Custom AI agent development connects that brief to your working tools, tests the result and hands over a system someone can operate. Use the five stages below to review a proposed build. They describe deliverables and decisions, not a universal calendar: access, test results and the consequences of an error determine what is ready to launch.

1

Write the workflow

Document the trigger, decisions, sources, outputs and human review points. Resolve disputed cases with the process owner before treating them as build instructions.
2

Map the integrations

List what each connected tool can read and change. Agree credentials, permissions, exclusions and the response to a failed connection.
3

Write and test the prompts

Use representative real inputs and expected results. Keep a separate set for testing changes so a good result on familiar examples is not the only evidence.
4

Test against edge cases

Try missing information, conflicting records, unfamiliar requests and interrupted actions. Record the expected stop or escalation as well as successful completion.
5

Launch with defined success criteria

Agree acceptable results, launch blockers, review responsibilities and a tested pause. Hand over the records and access needed to operate the workflow.

Ask for an estimate tied to these deliverables. A connection that already exists, an unresolved access request and a legacy tool needing custom work represent different dependencies. The table gives you a way to challenge the estimate without assuming every project has the same duration.

StageDeliverable before moving onWhat can extend it
Stage 1: Write the workflowAgreed scope, decisions and examplesAn undocumented or contested process; unavailable process owner
Stage 2: Map integrationsDemonstrated access and action boundariesRestricted tools, access approval or missing test accounts
Stage 3: Write and test promptsRecorded results against representative casesInput variation, unclear source priority or inconsistent outputs
Stage 4: Test edge casesEvidence for stops, recovery and unusual casesConsequential failures, retries or untested connected workflows
Stage 5: Launch preparationAcceptance decision and operating handoverUnresolved blockers, review capacity or unclear support ownership
TotalA scoped plan with dependencies and acceptance evidenceChanges to the agreed task or unresolved work at any stage

Stage 1: What belongs in a custom AI agent brief?

A custom agent brief states what starts the job, which information it uses, what it may decide and where the result goes. It also states when the job stops or needs a person. Give the builder representative cases and an agreed answer for disputed ones; process documentation and technical feasibility need to meet in the same specification.

Before building, give the process owner a normal request and an ambiguous one. Agree the expected result for both. If the team cannot settle the decision, route that case to a person instead of asking the agent to invent the policy.

A complete workflow document answers five questions for every step:

  1. What triggers this step? Name the event and where it arrives: an intake email, an approved form or a changed customer record. Define what is outside the intake route.
  2. What data does the agent need? Name the fields and systems, and decide which source wins when records disagree. A customer name alone may not identify the right account.
  3. What decision does the agent make? Describe the allowed paths, the conditions for each and the unresolved cases that return to a person.
  4. What is the output? Specify its format and destination: a Gmail draft, a Slack message or a HubSpot record change. State whether the result is merely prepared or actually released.
  5. When does a human need to review? Identify the action held for approval, who reviews it and what happens when nobody responds.

For the intake example, write down how new enquiries, renewals and complaints differ. If an email combines two requests, specify whether the agent creates separate review items or returns the whole message. Include attachments, missing customer details and an outdated CRM record in the examples.

Do not leave acceptance as “the agent understands our business.” An agreed case should include the input, the records available at that point, the permitted result and any prohibited action. That lets you review a failure without arguing afterward about what the agent was supposed to do.

Five stages of a custom agent build: write the workflow, map connections, write and test prompts, test unusual cases, then agree success and launch
The workflow brief supplies the decisions; each later stage needs its own evidence before launch.

Stage 2: Which connections and permissions does the agent need?

Map every connection by the data it needs and the actions it can perform. A promise that the agent only processes a selected folder or label should be supported by both the workflow and the actual access granted. Ask the builder to show the boundary, including what happens when the agent requests something outside it.

Gmail illustrates the distinction. As of October 2026, Google's gmail.readonly permission covers email messages and settings; it is not a permission restricted to one label.[1] Gmail's message-listing method can filter the messages returned by label.[2] That filter selects the work to process, but does not by itself reduce the permission granted to the connection.

For a business owner, the question is practical: could the same connection retrieve mail outside the intended intake route? Ask for a demonstration. Where the required permission is broader than the business task, agree a narrower account or another enforceable control, and test it. Do not treat an instruction saying “ignore other mail” as the whole access design.

For each tool, record who owns the connection, who can renew or revoke it and where a failed connection appears. Staff departure or revoked access can interrupt a workflow; arrange the handover so it does not depend on an absent colleague's personal account. Keep credentials out of the prose instructions given to the agent.

Also record failure handling. If the CRM is unavailable when an intake email arrives, should the email wait, go to a person or produce a draft that explicitly leaves the status unresolved? The agent must not silently treat a failed lookup as evidence that the customer is new.

Test action boundaries separately from reading. A review queue is useful only if the message cannot also be sent through another route without the agreed approval. Try a rejected approval, an expired review item and an unavailable reviewer. Confirm the business tool's state afterward.

Stage 3: How do you customize and test the instructions?

Write instructions against representative work, with expected results agreed by the process owner. Use real cases where suitable, handling customer information under your data rules, and add clearly labelled examples for gaps or unusual conditions. Keep some cases separate for testing later revisions rather than judging every change on examples used to write it.

There is no single sample size that makes a workflow ready. Cover the important variations: customer type, message format, missing fields, language, attachments and the consequences of choosing the wrong record. Add cases when a new decision or failure appears. Record which important inputs are still untested.

The brief defines the job. Testing shows whether the build follows it.

For each failure, distinguish the business rule from the implementation. If two interpretations are both reasonable, the process owner needs to settle the rule or require review. If the correct record was available but ignored, investigate retrieval and instructions. If the right result was prepared but the wrong record changed, investigate the action that made the change.

Keep expected results specific enough to grade. For the intake example, correct classification alone is insufficient: check the selected customer, the draft's facts, the review destination and the absence of an unauthorized send. Stylistic edits and wrong customer facts need different labels in the test record.

Repeat important cases rather than assuming one successful attempt settles them. Anthropic's January 2026 evaluation guide describes multiple trials because model outputs vary between runs.[3] Your team should agree which cases need repeated testing and what failures block release. Preserve the failing case after repair and rerun previously successful cases when relevant instructions, sources or connections change.

Stage 4: How do you test unusual inputs and failed actions?

Test the workflow's response to incomplete, conflicting and unsupported work, alongside its normal cases. Decide the acceptable outcome first: ask for missing information, stop an action or pass the case to a person. Finding a gap before launch is useful; it does not guarantee that every future input is covered.

Normal tested inputs surrounded by unusual cases: partial input, conflicting data, new format, ambiguous category and missing field; another case may appear after launch
Include unusual inputs in testing and retain a response for cases that first appear after launch.

Start with four groups. Partial inputs: remove a field the instructions expect and inspect whether the agent asks or guesses. Ambiguous inputs: combine an enquiry and complaint, or give two plausible customer matches. Conflicting inputs: make the email and CRM disagree and check the agreed source priority. Volume and timing: interrupt processing, delay a tool response or repeat a request while an earlier attempt is pending.

For each case, inspect the connected tool as well as the visible answer. Anthropic's evaluation guide distinguishes the recorded conversation from the final state: an agent can claim a flight is booked while the reservation is absent.[3] For intake, check whether the intended draft exists and whether a duplicate, unintended send or wrong customer update also occurred.

Test recovery without live customer consequences. If a write succeeded but the confirmation was lost, retrying could repeat the action. Ask the builder to demonstrate how the workflow identifies a completed action before trying again, and what it does when the result remains unknown.

Add a request that asks the agent to bypass its instructions or release a message before approval. Include misleading instructions inside incoming material. The expected result should follow the workflow's permissions and approval decision rather than treating the customer's message as authorization to change them.

Record each finding, its consequence and the release decision. A harmless formatting defect and an unauthorized customer message are different issues. Repair launch blockers and retest them; assign lower-consequence issues only when the process owner has accepted their remaining effect. A date in the project plan does not resolve an unsafe action.

Stage 5: What makes the agent ready to launch and hand over?

Launch when the agreed tests, permission boundaries and operating responsibilities support the proposed scope. Acceptance should state both useful results and unacceptable failures. Hand over the workflow brief, test records, access ownership, review queue and stop procedure so the receiving team can operate what was built.

Use three measures with explicit definitions and denominators:

Task completion rate: completed tasks divided by eligible tasks in the measured set. Count only results that meet the agreed quality and action requirements. A finished draft or lack of escalation does not by itself mean the result was correct. Record excluded work separately so narrowing the scope cannot silently inflate the figure.

Escalation rate: cases sent to a person divided by eligible cases, with the reason recorded. A required approval is different from an unexpected failure. Decide which handoffs are part of normal operation and whether your team has enough capacity to handle them; reducing escalation is not useful if it removes a necessary safeguard.

Error and edit rates: incorrect results divided by reviewed results, and edited results divided by reviewed results. Keep them separate: a reviewer may change wording on a correct draft, while an unreviewed incorrect result never appears in the edit count. Record how the review sample was chosen and include consequential action failures.

Agree thresholds for this workflow with the responsible owner. Estimate the human work using the trial's actual review and repair time, not a generic completion percentage. State what triggers a pause and who can restart the job. For ongoing costs, see what a custom agent actually costs. For the definition and when a custom build fits, see what is a custom agent.

The handover should let the receiving team demonstrate a rejected action, a failed connection, an unresolved request and a successful ordinary case. Agree support coverage, who approves changes and where test evidence lives. An agreed handover can close the build; ongoing support needs its own responsibilities and scope.

What actually gets customized in a custom AI agent?

AI agent customization adapts the workflow to your decisions, tools and usable outputs. Ask the proposal to describe those changes concretely. The choice of AI matters for performance and cost, but a product name alone does not explain how the finished agent will fit your business.

Four layers connect the brief to the build:

  • The integration surface. Which systems the agent reads or changes, the records it can reach and the permissions granted to each connection — stage 2.
  • The decision rules. When it acts, asks, stops or escalates, including the source priority and excluded work — stages 1 and 3.
  • The approval logic. Which actions wait for a person and how that decision is enforced, rejected or timed out — stages 2 and 4.
  • The output contract. The format, facts, tone and destination that make the result usable, with criteria for checking it — stages 3 and 5.

These are areas to specify, not an exhaustive list of possible development work. A build may also need changes to source records, retrieval or the AI it uses. Ask which changes the quote includes and which require a separate decision.

Connection count and approval boundaries affect the work, but so do difficult decisions and the consequences of a mistake. Compare proposals for the same scope. A simple-looking intake task can need substantial testing when it changes customer records; a complex analysis that remains an internal draft can have a different action boundary.

What if the agent produces wrong outputs after launch?

Preserve the wrong output and affected records, contain further harmful actions, then compare the failing case with the accepted workflow. Changed requirements, connections and scope are useful places to investigate, alongside model behaviour, source quality and implementation faults. Symptoms guide the investigation; they do not establish the cause by themselves.

Changed instructions or requirements. Your team may now use different categories, fields or output rules. Compare current examples with the agreed brief. If the business rule changed, approve it before changing the agent. Test the revision against both the new cases and previously successful ones.

Changed connections. A tool, field, credential or response may have changed. Check actual connection errors, source records and action outcomes, then the provider's change information. Empty output is not a sufficient diagnosis: the source itself could be empty, or the workflow could have selected the wrong record. Retest the affected read and write paths after repair.

Expanded scope. New request types or formats may be arriving through the existing route. Inspect the escalation queue and decide whether the job should include them. If it should, update the brief, permissions and tests together. Otherwise, document the exclusion and return those cases to a person.

Keep the original failing case in the test set and record the repair. Decide what customer or record correction is needed before treating a successful rerun as the end of the incident. Bring forward review when repeated failures or their consequences warrant it, rather than waiting for a fixed calendar interval.

For ongoing maintenance cost and what it covers, see custom agent cost. For the decision of whether to build custom or use an existing platform, see custom vs. off-the-shelf.

Frequently asked questions

How long does it take to build a custom AI agent? Agree a timeline for the specific brief, connections, tests and acceptance process. Access approval, unresolved business decisions and failed tests can extend it. Ask which dependencies the estimate assumes and what evidence is required to move between stages.

What is the most common reason custom agent builds fail? Unclear process rules are one cause to investigate, alongside source quality, model behaviour, permissions and tool failures. Define expected outcomes and inspect the actual failing case rather than assuming all builds fail for the same reason.

How many integrations should a first custom agent build have? Use the smallest set needed to complete the agreed workflow. Each connection needs demonstrated access, ownership and failure handling. A necessary CRM lookup should not be omitted to meet an arbitrary connection count.

What does testing against edge cases mean in practice? Try missing fields, ambiguous requests, conflicting records and interrupted or repeated actions. Agree the expected stop, response or human handoff first. Check actual tool outcomes and retain failing cases for regression testing.

What do you do when a custom agent produces wrong outputs after launch? Contain further harmful actions, preserve the output and inspect the failing case against the accepted brief. Investigate changed requirements, sources, permissions, connections and model behaviour. Repair affected work, test the fix and previously successful cases, then agree whether to resume.

Notes

  1. Google, Choose Gmail API scopes. 10 September 2026; accessed 14 October 2026.
  2. Google, Method: users.messages.list. 15 April 2026; accessed 14 October 2026.
  3. Anthropic, Demystifying evals for AI agents. 9 January 2026; accessed 14 October 2026.