Test with simulated callers

Testing by hand finds the first problems. Simulated callers find the rest: an AI plays a caller with a goal and a personality, talks to your agent, and a judge scores the call against the checks you wrote. Save the callers you care about as scenarios and re-run them after every change.

You need a workspace role of Builder or higher. Viewers can see results but not run tests.

1. Run a one-off simulated call

1. Open the agent, then Test under Ship, and the Simulated caller tab.

2. Under The caller, keep A one-off caller and describe them:

   • Who they are: for example Arjun, 62, polite, speaks slowly and mixes Hindi and English. Be specific: who they are, how they talk, what they want.
   • What they want: for example Book a cleaning next Tuesday morning, then change it to Wednesday.
   • What to check (one per line): for example Asks one question at a time and Reads the booking back before saving. Leave it empty to just check the caller got what they wanted.
   • Longest call (caller turns): stops a call that goes round in circles.

3. Under How the caller talks, choose:

   • Text: the real agent, fast and cheap. The same agent as a real call (prompt, tools, bookings, pathway), with text instead of audio. Use it for most tests.
   • Voice check: a real call. A real call with audio. Use it for what only audio shows: reply time, turn-taking, pronunciation and mishearing.

4. Under Test against, keep Draft, or pick a published version.

5. Click Start the call (or Run voice check).

Watch the conversation live. The status moves from Waiting to On the call, then Scoring when the caller hangs up, and ends as Passed or Failed. A run that couldn’t finish shows Couldn’t run, with the reason.

2. Read the result

The judge gives a score out of 100 and goes through each check: passed or failed, why, and quotes from the call as evidence. A run passes only when every check passes.

Besides your checks, the judge always looks for two serious mistakes:

• Wrong dates. A date that doesn’t match its weekday, a wrong “tomorrow” or “this Saturday”, or a date or time different from what the caller asked for.
• Made-up actions. The agent says it booked, scheduled, cancelled, sent or saved something, or that a slot is free or taken, without a tool actually doing it. These fail the call.

For a failed call, open it and use What went wrong? for suggested fixes. See Test an agent in your browser.

Here’s a voice check with an agent made from the Appointment booking template. The caller is the AI; it books a check-up and the agent’s booking tool saves it:

Voice check: book a check-upEnglish · simulated caller · 1 min 21 s

3. Save scenarios and re-run them

A scenario is a saved caller with its checks. Build a small set that covers the calls that matter: the normal booking, a caller who changes their mind, one who gives unclear answers, one who goes off-topic, one who asks for a person.

1. Open the Scenarios tab.

2. Click Suggest with AI to get Suggested scenarios written from your agent’s instructions. Pick the ones to keep and click Add selected. Or click New scenario and write one: a name, Who the caller is, What they want, and What to check (up to 8).

3. Under Run against, pick the draft or a version, and click Run all.

Re-run your scenarios after every change, before you publish. The Compare versions tab shows the pass rate per version and for the draft, and which scenarios passed before but fail now.

Checks the judge can verify

Write checks about what happens in the call: Reads the date back before booking, Never gives medical advice, Offers a callback when it can’t help. Avoid checks the transcript can’t show, like Sounds friendly.

Note

Simulated calls count towards your workspace’s usage limits. If you reach one, the message says when it resets, with a way to contact 360.