How can we help you today?

More information:

Guide - Using AI Telephone Bots

AI Telephone Bots can run as voice agents or advanced IVR flows. They can gather data, call external systems, and perform call-control actions.

Contents

Basic Configuration

AI Bots are configured under console -> Stuff -> Add -> AI Bot.

A valid config must include a root description.

description: >
  You are an AI telephone bot for Widgets Ltd.
  Welcome the caller and collect their name.

initial: Thank you for calling Widgets Ltd, please tell me your name.

initial is optional. If set, it is spoken as the first assistant message for that scope (root/context/step).

Models and Language

If model is omitted (or unknown), the bot defaults to mistral-large-2512.

Currently supported model keys:

  • gpt-3.5-turbo
  • gpt-4o
  • gpt-4o-mini
  • gpt-4.1
  • gpt-4.1-mini
  • gpt-4.1-nano
  • mistral-large-2512
  • mistral-small-2506 (retired by Mistral 31st July 2026 — use mistral-small-2603)
  • mistral-small-2603

temperature is clamped to 0..2.

Optional language should be an ISO 639 code and is used by speech-to-text hints.

Data Location

Some organisations need to say where a call may be processed. The optional region key restricts every AI model the bot can reach to providers processing in the regions you list.

description: >
  You are an AI telephone bot for Widgets Ltd.

region: [ uk, eu ]

model: mistral-large-2512      # eu
stt:
  engine: voxtral              # eu
tts:
  engine: polly                # uk

It takes a single region or an array, from uk, eu and us. Leave it out entirely for no restriction.

Everything the bot can reach is checked: the main model, guards at bot, context and step level, the voicemail and language-gate classifiers, webhook translation, and the text-to-speech and speech-to-text engines including any per-language overrides.

Where each provider processes:

uk eu us no guarantee
Model mistral-* gpt-*
Speech-to-text voxtral openai-whisper cloudflare
Text-to-speech polly mistral inworld

What happens if a bot breaches its region

The bot is rejected before the call is answered. The caller is passed on to whatever the rule's next step is — a person, or voicemail — so they are never left on a dead call, and the reason is recorded in the call history.

Nothing is quietly swapped for a compliant alternative. A substituted model would mean the bot that ran was not the one that was signed off, and the call record has to stay truthful about what processed the call.

Three things to know

Regions mean exactly what they say. eu does not satisfy uk. If you accept EU processing for a UK service that is a decision for you to make and state, by writing region: [ uk, eu ].

Defaults are checked the same as models you name. The default text-to-speech engine is polly (uk), so region: eu on its own will fail — you also need tts.engine: mistral. Likewise the language gate's default speech-to-text engine is cloudflare, so a language gate plus any region needs languagegate.stt.engine set to an in-region engine.

Cloudflare has no residency guarantee. It is anycast, so we cannot say which country the processing happened in. It satisfies no restriction at all.

Every AI bot call records where its models ran, whether or not a region was set, so you can evidence it after the fact.

Text-to-Speech

Text-to-speech engine and voice can be configured:

tts:
  engine: polly # or mistral, or inworld
  voice: Amy    # Polly voice ID, Mistral voice slug, or Inworld voice name

Supported engines:

  • polly (default) — AWS Polly voices
  • mistral — Mistral voices (plain text only, no SSML)
  • inworld — Inworld voices using the inworld-tts-2-flash model (plain text only, no SSML)

Polly voices

Any AWS Polly voice ID can be used (e.g. Amy, Brian, Emma, Joanna, Matthew). The default is Amy. See the AWS Polly voice list for all available voices.

Mistral voices

The full list of supported Mistral voice slugs:

Voice Variants
nick —
alison alison_excited
en_paul en_paul_neutral, en_paul_happy, en_paul_sad, en_paul_excited, en_paul_confident, en_paul_cheerful, en_paul_frustrated, en_paul_angry
gb_oliver gb_oliver_neutral, gb_oliver_excited, gb_oliver_curious, gb_oliver_confident, gb_oliver_cheerful, gb_oliver_sad, gb_oliver_angry
gb_jane gb_jane_neutral, gb_jane_confident, gb_jane_curious, gb_jane_sad, gb_jane_confused, gb_jane_frustrated, gb_jane_jealousy, gb_jane_sarcasm, gb_jane_shameful
fr_marie fr_marie_neutral, fr_marie_happy, fr_marie_excited, fr_marie_curious, fr_marie_sad, fr_marie_angry

Inworld voices

Inworld synthesis uses the inworld-tts-2-flash model, generated at a 16kHz sample rate and resampled internally for telephony. The voice ID is the Inworld voice name (e.g. Ashley, Dennis, Sarah); the default is Ashley. Browse the full voice library in the Inworld TTS documentation.

tts:
  engine: inworld
  voice: Ashley

Per-language voices

When a bot handles more than one language, tts.voices can override the voice (or the whole engine) per language tag. The flat engine/voice settings remain the default for any language without an override:

tts:
  engine: polly
  voice: Amy
  voices:
    fr: { engine: mistral, voice: fr_marie_neutral }
    cy: Gwyneth # string shorthand - same engine, different voice

Language Gate

An optional multilingual gate can run at the start of the call: it plays a greeting in each offered language, listens for the caller's choice, and sets the call's language for the rest of the conversation (TTS voice and STT hints follow it automatically).

languagegate:
  languages: [ en, fr, ar-LB, uk ] # offered languages (supported: en, fr, ar-LB, uk, cy, it)
  default: en # used when no answer can be resolved
  greeting:
    text: # spoken per language, in that language's voice
      en: For English, say English.
      fr: Pour le français, dites français.
  reprompt: # optional short retry prompt, spoken instead of the full greeting
    text:
      en: Sorry - which language?

The greeting plays while the gate listens underneath. The caller starting to speak pauses the greeting (instant feedback); a cough or background noise resumes it from where it paused; a real answer stops it and selects the language. An LLM classifier interprets natural answers — the language's name in any language ("Arabic", "عربي", "arabi"), simply speaking in one of the offered languages, or positional references ("the first one").

Options:

  • greeting.audio / reprompt.audio — pre-rendered wav instead of per-language text
  • decider: keyword — deterministic matching only (no LLM cost); provide matchers: { en: [ english, yes ], ... }
  • classifier.model — model override for the LLM decider (defaults to mistral-small-2506)
  • stt.repeat — how many times the greeting repeats before falling back to default (default 2)
  • stt.silence — trailing listening window in seconds (default 5)
  • power/duration knobs as in Speech Input (stt.startabovepower etc.)
  • stt.engine — speech-to-text engine for the gate turn (defaults to cloudflare). If the bot sets a region you must name an in-region engine here, because cloudflare carries no residency guarantee.

Per-language TTS voices are configured on the scenario tts block:

tts:
  engine: polly
  voice: Amy
  voices: # per-language overrides (string voice, or an object to override engine too)
    fr: Lea
    ar-LB: { engine: polly, voice: Zayd }

Per-language STT overrides use stt.perlanguage keyed the same way.

Speech Input and Guarding

Speech-to-text can be configured with:

stt:
  engine: voxtral # or openai-whisper, or cloudflare
  mode: concurrent # recommended - natural turn-taking (see below)

Natural turn-taking (concurrent mode)

With mode: concurrent the bot listens continuously — while its reply plays and while it is thinking — instead of speaking and listening in strict turns:

  • The caller can interrupt. The reply pauses the moment the caller starts speaking (instant feedback that they were heard). Real speech cuts the reply off and is answered; the bot never talks over the caller.
  • Coughs and background noise cost nothing. If a capture turns out not to be speech, the reply resumes from exactly where it paused — not from the start.
  • Slow speakers are not cut off. The model can judge that the caller paused mid-sentence or mid-number (an address or reference given in parts) and wait silently for them to finish, answering the completed thought once.
  • Speech while the bot is thinking is kept. If the caller adds something during the model's response latency ("oh — and use my mobile number"), the bot answers the addendum instead of the superseded question.

The last two behaviours are driven by two internal tools (ignore_input and await_more_input) that are made available to the model automatically on concurrent scenarios — no configuration is needed.

Tuning (all optional, sensible defaults):

stt:
  engine: voxtral
  mode: concurrent
  startabovepower: 250 # speech-start threshold; lower for quiet callers, raise if line echo falsely pauses the reply
  poweraveragepackets: 10 # smoothing window (packets of 20ms) for the speech-start detector
  finishbelowpower: 80 # end-of-utterance threshold
  minduration: 1000 # minimum capture length in ms
  maxduration: 10000 # maximum capture length in ms
  silence: 10 # listening window in seconds after the reply finishes
  maxturns: 8 # noise re-captures per turn before giving up

Legacy barge-in (sequential mode)

Without mode: concurrent the bot speaks its reply in full and then listens. Barge-in can still be enabled on this sequential path:

stt:
  engine: voxtral
  interrupt: true # enable barge-in
  bargeinpower: 10 # power threshold for barge-in detection
  bargeinpoweraveragepackets: 5 # packets to average for power detection

When interrupt is true, speech during prompt playback interrupts the prompt and starts recording immediately. bargeinpower and bargeinpoweraveragepackets control the sensitivity. Note that this cuts the prompt on any energy (a cough kills the reply permanently) — prefer mode: concurrent, which pauses instead and can resume.

For stricter turn-taking, use guard.in at root, context, or step scope (same precedence as other scoped settings: step -> context -> root).

guard.in supports:

  • description: instruction text for the guard classifier (JEXL templates supported)
  • model: optional model override for guard classification (defaults to mistral-small-2603)
  • allow: array of free-text allow rules (JEXL templates supported)
  • correct: boolean to enable speech-to-text correction on caller input
  • action: what to do when input is classified bad

Example:

guard:
  in:
    description: >
      Check caller input against the current prompt.
      Be lenient to natural phrasing.
    model: mistral-small-latest
    allow:
      - Caller answers the prompt
      - Caller asks to speak with a person
    action:
      reprompt: Sorry, I did not catch that. ${{ prompt }}

Speech-to-text correction

correct: true enables automatic correction of speech-to-text errors and regional dialect artifacts. The guard classifier already receives the prompt and the caller's transcript, so it can infer what the caller likely meant.

This is useful when callers have strong regional accents. For example, a caller from Liverpool asked "are you the patient or the carer?" might produce a transcript of "cara" — with correct: true, the guard will correct this to "carer" before it reaches the main AI.

correct can be used on its own (without description or allow) for correction-only, or combined with guard for both validation and correction:

# correction only
guard:
  in:
    correct: true

# guard + correction
guard:
  in:
    description: Check caller input is relevant to the question.
    correct: true
    allow:
      - Caller answers the prompt
    action:
      reprompt: Sorry, I did not catch that. ${{ prompt }}

Like other guard settings, correct follows step -> context -> root precedence. A step can set correct: false to disable correction inherited from a higher scope.

Runtime behavior

  • The spoken prompt is always what is passed to STT.
  • If guard.in is not configured, input is accepted normally.
  • If guard.in is configured, a second AI classification pass is run on each captured user input.
  • When correct: true, the classifier may also return a corrected version of the transcript which replaces the original before it reaches the main AI.
  • Guard classifier failures are fail-closed (treated as bad input).
  • On bad input, guard.in.action runs (reprompt, finish, hangup, or annotate).

Annotate instead of reject

A strict guard can frustrate callers — if the guard rejects unclear input, the caller simply recycles through the same prompt. action.annotate offers a softer alternative: instead of rejecting, the caller's transcript is passed through to the main LLM wrapped in a tag, so the main AI can decide how to handle it (e.g. ask the caller to spell their name).

guard:
  in:
    description: Check caller input is a plausible name.
    action:
      annotate: "[guard: possible STT error - proceed with caution] {{input}}"

annotate accepts:

  • true — use the default template ([guard: possible STT error — proceed with caution] {{input}})
  • a string — a custom template where {{input}} is replaced with the caller's raw transcript

When using the longform, action.tool: annotate can be combined with bad and empty templates:

guard:
  in:
    action:
      tool: annotate
      bad: "[guard: unclear answer] {{input}}"
      empty: "[quiet murmuring, not understood]"

When using annotate, add guidance to the main AI's system prompt telling it how to handle bracketed tags — e.g. "Input wrapped in [...] is metadata from the STT layer; do not read it aloud. If you see [guard: ...], ask the caller to clarify or spell their answer."

Full example: passing the error to the main LLM

Here a patient is asked for their surname. STT often mangles names, so rather than rejecting and making the caller repeat themselves, the guard annotates the transcript and hands it to the main LLM, which already has instructions to ask the caller to spell the name.

description: >
  You are a receptionist taking a patient callback request. Your job is to
  collect the caller's surname and phone number, then confirm.

  When the caller's input arrives wrapped in square brackets (e.g.
  "[guard: ...] smith"), treat the bracketed portion as a private note
  from the speech-to-text layer — DO NOT read it back to the caller.

  If you see "[guard: possible STT error ...]", the transcript is
  uncertain. Ask the caller politely to spell their answer letter by
  letter instead of repeating the question verbatim.

  If you see "[quiet murmuring, not understood]", the caller said nothing
  intelligible. Check they can hear you, then repeat the question once.

contexts:
  collectname:
    description: Ask the caller for their surname.
    guard:
      in:
        description: Check the caller gave a plausible surname.
        allow:
          - Caller said a recognisable name
          - Caller is spelling a name letter by letter
        action:
          tool: annotate
          bad: "[guard: possible STT error - surname unclear] {{input}}"
          empty: "[quiet murmuring, not understood]"
    steps:
      - description: Ask for surname, confirm once you have it, then move on.

Example conversation:

Turn Content
Assistant "What is your surname please?"
Caller (STT) "ffff"
Guard classifies as bad → wraps input
Main LLM sees [guard: possible STT error - surname unclear] ffff
Assistant "Sorry, I didn't quite catch that — could you spell your surname for me, letter by letter?"
Caller (STT) "s m i t h"
Guard accepts (matches "spelling a name" rule)
Assistant "Thank you — so that's Smith, is that right?"

Notice the main LLM picks a more helpful reply than the generic guard reprompt would, because it saw why the guard was suspicious and adapted its strategy.

When to use annotate vs reprompt

  • Use reprompt when the rejection is deterministic and you want tight control of the error wording (e.g. "I can only accept yes or no.").
  • Use annotate when the main LLM has richer context and can recover more gracefully — names, free-form symptoms, addresses, anything where "please spell it" or "please describe it differently" is a better recovery than repeating the same question.
  • Combine them by scope: set a strict reprompt on a yes/no step, and a looser annotate at the context level for open-ended questions.

Tools and Permissions

Built-in tools:

  • send_sms
  • send_sms_caller
  • jump_extension
  • forward_call
  • hangup
  • finish

Example:

tools:
  send_sms:
    destinations:
      - 447700900123
  jump_extension:
    extensions:
      - "1000"
      - "1001"
  hangup: true
  finish: true

Permission resolution order is:

  1. step-level tools
  2. context-level tools
  3. root-level tools

A tool explicitly set to false at a narrower scope is denied even if enabled elsewhere.

hangup and finish can be either:

  • true (AI provides final message)
  • object with final (fixed message enforced by config)

Example fixed final:

tools:
  hangup:
    final: Thank you for calling. Goodbye.

Variables and Session Values

Template format is ${{ ... }}.

Runtime vars available under var:

  • var.now
  • var.uuid
  • var.callerid

Date/time helper functions are available in JEXL compute/template expressions:

  • now()
  • now("YYYY-MM-DD HH:mm:ss")
  • now("YYYY-MM-DD HH:mm:ss", "UTC")
  • formatdatetime(input, "YYYY-MM-DD")
  • formatdatetime() (defaults to current date/time)
  • yearssince(input)

Useful formatdatetime tokens:

  • YYYY, YY
  • MM, M
  • DD, D
  • HH, H
  • mm, m
  • ss, s
  • MMM, MMMM
  • ddd, dddd

Examples:

session:
  today_utc:
    compute: now("YYYY-MM-DD", "UTC")
  timestamp:
    compute: formatdatetime()
  patient_dob_display:
    compute: formatdatetime(session.dob, "ddd, DD MMM YYYY")

Session values are under session.

You can pre-populate session values in config, including templates and compute expressions:

session:
  enquirer_telephone: ${{var.callerid}}
  callerid_len:
    compute: var.callerid.length

Within logic, last contains the most recent webhook result (e.g. last.success). Where the webhook declares a returns shape, last.data holds the validated values — including any kept from the AI with expose: false — and last.reason says why a response was rejected.

Contexts, Steps, and Actions

Use start to set the first context.

start: intake

contexts:
  intake:
    description: Collect caller details.

Context Switching

Context switching is controlled by allowed context lists:

  • contexts.<name>.contexts
  • steps[].contexts

When switching, session values defined in collect are saved.

Steps

Steps are ordered and run one at a time.

contexts:
  intake:
    description: Ask one question at a time.
    steps:
      - initial: What is your first and last name?
        collect:
          first_name:
            description: Caller first name
          last_name:
            description: Caller last name
      - description: What is your date of birth?
        collect:
          dob:
            description: Date of birth in YYYY-MM-DD

Important:

  • A string step (e.g. - "ask name") is treated as a step description.
  • It is not converted to initial.

when is supported on contexts and steps.

goto is supported on steps for context jumps:

- goto: another_context

Entry Actions

Contexts and steps can define action blocks that run immediately on entry (before normal AI turn):

action:
  webhook: submit_case

or

action:
  tool: hangup
  final: We have what we need. Goodbye.

Supported action targets:

  • tool: hangup
  • tool: finish
  • webhook: <name>

Webhooks

Webhooks are function tools the AI can call.

Required webhook keys:

  • description
  • url
  • fields

Example:

webhooks:
  submit_case:
    description: Send case to CRM
    url: https://example.com/api/cases
    method: POST
    content_type: application/json

    expect:
      status: 200
      content_type: application/json

    headers:
      Authorization: Bearer ${{secret.crm_token}}

    fields:
      callerid:
        type: string
        value: ${{var.callerid}}
      dob:
        type: string
        description: Date of birth in YYYY-MM-DD
      age:
        compute: yearssince(session.dob)

Notes:

  • Default method is POST.
  • Default request content_type is application/json.
  • Supported request bodies: JSON and application/x-www-form-urlencoded.
  • For JSON payloads, path can map nested objects.
  • required: false omits missing/empty values.
  • expect.status takes a single status code or a list of them, and must match one of them when provided.
  • Without expect, success defaults to HTTP 200 or 202.
  • expect answers whether the call worked. What of the response the AI is allowed to see is a separate question — see Controlling what the AI sees.

Controlling What the AI Sees

Whatever a webhook returns is handed to the AI as the result of the tool call, where it sits in the conversation alongside everything else the bot has been told. Without returns, a JSON response is passed on whole — every field the endpoint sends, however large, and whatever text it contains.

returns declares the shape you expect back. Only declared, validated values reach the AI.

webhooks:
  bookappointment:
    description: Book an appointment
    url: https://example.com/api/book

    fields:                        # what goes out
      nhsnumber:
        type: string
        description: The caller's NHS number

    expect:                        # did the call work?
      status: [ 200, 201 ]
      content_type: application/json

    returns:                       # what may the AI see?
      path: data.booking           # where to read from in the response
      fields:
        reference:
          type: string
          pattern: "^[A-Z]{2}[0-9]{4}$"
        clinician:
          type: string
          truncate: 60
          required: false
        patientid:
          type: string
          expose: false            # kept for your logic, never shown to the AI

The AI is told {"reference":"AB1234","clinician":"Dr Who"} and nothing else. Leave returns out and nothing changes from before.

Two optional settings decide what happens to everything else:

Situation Setting Default Alternatives
Data you did not declare on_unknown strip — dropped fail, allow
Declared data that is missing, the wrong type, or breaks a rule on_invalid fail — the call fails strip

The defaults suit a live call. An endpoint that starts sending an extra field should not break anything, so data you did not ask for is dropped. Data you did ask for and did not get is a broken agreement, so the call fails — and the AI is told no more than it is told about any other failed webhook.

Rules fail; cuts do not:

  • max_length, max_items, pattern, enum, minimum and maximum are rules. Breaking one goes through on_invalid.
  • truncate on a string and take on an array always cut, and never fail. take is the usual way to limit how much of a list reaches the AI.
  • A field with a default never fails either — it falls back to the default.

Other keys:

  • returns: false — nothing from the response ever reaches the AI. Right for a webhook that only needs to fire.
  • expose: false — capture a value for your own logic (last.data.patientid) without putting it in the prompt. An identifier can go back out to another webhook without the AI ever seeing it.
  • max_bytes — a cap on the whole payload handed to the AI, 4096 by default. A response can satisfy every field you declared and still be far more than belongs in a prompt.
  • path — read from part of the response, either at the top or on an individual field.
  • Types are string, number, integer, boolean, object (with fields) and array (with items). Nothing is converted: a field declared string that arrives as a number breaks the declaration and is handled by on_invalid.
  • A plain text response is declared as a value rather than as fields: returns: { type: string, truncate: 200 }.

Text is always stripped of invisible characters before the AI sees it. The strongest control is still pattern or enum: a value restricted to a reference format cannot carry an instruction to the bot.

Lists and nested objects work the same way:

webhooks:
  findslots:
    description: Find available appointment times
    url: https://example.com/api/slots
    method: GET

    fields:
      day:
        type: string
        description: The day to look at, YYYY-MM-DD

    returns:
      type: array
      path: slots
      take: 5                      # only the first five reach the AI
      items:
        type: object
        fields:
          id:
            type: string
          time:
            type: string

Webhook Scope by Context/Step

Webhook exposure can be restricted by:

  • context-level webhooks
  • step-level webhooks (array or object allow/deny)

This lets you grant webhook access only where needed.

Mailto Webhooks

url: mailto:someone@example.com is supported.

For mailto webhooks, you can define subject and body as template strings or compute objects.

webhooks:
  submitprescription:
    description: Email prescription request
    url: mailto:ops@example.com
    subject: New prescription request
    body: |
      Caller: ${{ session.first_name }} ${{ session.last_name }}
      Telephone: ${{ session.enquirer_telephone }}
      Recording: https://www.babblevoice.com/a/callexplorer?u=${{ var.uuid }}
      Transcript:
      ${{ fields.data }}
    fields:
      data:
        compute: >
          messages|chattext({ caller: session.first_name, assistant: "Bot" })

Rich formatting with markdown

Set format: markdown to send the email as multipart/alternative. The body is rendered as Markdown into HTML for clients that support it, and the raw Markdown is kept as the plain-text fallback. Use **...** for bold, *...* for italic, # Heading, - for lists, etc. The default (format: text) is unchanged.

webhooks:
  submitprescription:
    description: Email prescription request
    url: mailto:ops@example.com
    subject: New prescription request
    format: markdown
    body: |
      **Caller:** ${{ session.first_name }} ${{ session.last_name }}
      **Telephone:** ${{ session.enquirer_telephone }}
      **Recording:** [listen](https://www.babblevoice.com/a/callexplorer?u=${{ var.uuid }})

      **Transcript:**

      ${{ fields.data }}
    fields:
      data:
        compute: >
          messages|chattext({ caller: session.first_name, assistant: "Bot" })

RAG Search Tool

You can expose a search tool that queries MiniRAG.

tools:
  search:
    - url: kb://prescriptions
      purpose: NHS prescription policy documents

The AI then receives a search(url, query) function constrained to configured URLs.

Execution Notes

  • Root system prompt is always based on root description.
  • Active context description is appended when in a context.
  • Active step description is appended when in steps.
  • Initial prompt precedence is:
  • current step initial
  • current context initial
  • root initial (only when not in a context)
  • On context switch or step completion, message history is sliced to the new base index to keep prompts focused.
  • AI tool loops and entry-action loops are capped to avoid runaway behavior.
  • Guard input retries are limited per prompt to avoid infinite reprompt loops.
  • Three consecutive no-input turns (a silent line) end the conversation rather than replaying the prompt forever.
  • On concurrent scenarios the internal turn-taking tools (ignore_input, await_more_input) are budgeted per caller turn so the model cannot wait indefinitely.
  • A bot whose models breach its region is rejected before the call is answered; the caller passes on to the rule's next step. See Data Location.

Practical Recommendation

Keep permissions tight:

  • expose only required tools
  • expose only required webhooks per context/step
  • prefer deterministic action + when for critical workflow transitions