Two new Conversation Intelligence operators entered public beta on November 17. Twilio also moved five Conversation Relay voice features to general availability the same day.
Twilio published two product announcements on November 17 aimed at companies moving AI agents out of pilots and into production. The first puts two Conversation Intelligence operators into public beta. Task Completion and Hallucination Detection review an AI agent’s conversations with customers after those conversations end. The second announcement makes five Conversation Relay voice features generally available and adds an API for running operators on demand.
Both build on the Conversations layer Twilio introduced at its SIGNAL conference in May, which The Agile Brand Guide covered at the time.
For marketing and customer experience leaders, the supervision operators are the larger news. Brands that put AI agents on service and sales lines have had few ways to check, at volume, whether those agents told customers the right thing. Twilio is now offering a prebuilt check on that question for conversations that run on its platform.
How the supervision operators work
Task Completion assigns each conversation one of four outcomes: resolved, unresolved, escalated, or declined by the agent. It adds a one-sentence reason that points to the moment in the conversation behind the label.
Hallucination Detection compares what the agent said against the knowledge articles the agent was given and against the rest of the conversation. When it finds a conflict, it returns the agent’s statement alongside what the source says. Twilio’s documentation adds that if a team has not selected a knowledge base in Enterprise Knowledge, the operator checks only for contradictions inside the conversation itself.
Twilio says teams need both signals because an agent can finish a task and still give a customer wrong information. Its example is a chat about returning a damaged order. Task Completion marks the conversation resolved because the customer’s question about the refund window got an answer. Hallucination Detection flags that the refund window the agent quoted was incorrect.
Both operators run once a conversation ends and analyze it in full. Each uses a large language model as a judge, and that model is separate from the one behind the agent, so the agent is not grading its own work.
Results go to a customer’s webhook and to Conversation Insights, an API that aggregates operator results across conversations. Twilio says teams can use it to count how many conversations ended in each outcome during a given week, or to see which kinds of hallucination came up most in a month. The data can feed a company’s own dashboards or be exported to existing analytics tools. The documentation names Datadog and Snowflake as destinations.
Twilio contrasts this approach with AI observability tools that look inside the agent at traces, tool calls and latency. Those tools require teams to instrument each agent and write their own evaluations, the company says. Its operators start from the conversation record instead and need no changes to agent code. Twilio says they work whether an agent runs on one model, several models or an external agent platform.
That flexibility comes with a condition. The conversations have to run on Twilio, because Twilio’s record of the conversation is what the operators analyze.
Limits Twilio states
Twilio is direct about what the operators can and cannot do.
The judge model can be wrong. Each result includes a reason or the flagged statement so a person can check it, and Twilio recommends using the results to find conversations that deserve review, without treating them as a final ruling.
The operators are in public beta and open to all customers. Twilio’s documentation says public beta products are not covered by its support terms or service level agreement, and that details may change.
The documentation also says the two operators are not PCI compliant and are not a HIPAA Eligible Service, and that customers should keep them out of workflows subject to those rules.
Results are not available in real time. Twilio’s blog post says they arrive about 15 minutes after a conversation ends. The documentation says 15 to 20 minutes.
Conversations products appear only in the new Twilio Console, so accounts that have not migrated from the legacy Console will not see them.
Pricing follows Twilio’s existing rates for operators it authors. The announcement gives no figures.
Voice AI updates
The second announcement focuses on Conversation Relay, Twilio’s product for connecting voice calls to AI agents. Twilio argues that model quality alone does not produce a dependable voice agent. Callers pause mid-sentence and interrupt. Speech providers slow down or hit rate limits. Audio quality drops, and callers switch languages or use specialized terms.
Five features addressing those conditions are now generally available:
- Smart Turn Detection helps an agent tell a caller who has finished speaking from one who is pausing.
- Automated STT Failover switches to a backup speech-to-text provider when the primary one has latency, rate-limit or stream problems, without custom failover code.
- Dynamic Configuration Parameters let teams change speech hints, languages and voice settings during a call without restarting the conversation.
- HD Audio provides higher-fidelity voice input to improve speech recognition.
- Advanced Conversational Voice adds support for newer voice models. Twilio names ElevenLabs v4 Turbo and says it covers more than 90 languages.
Twilio also made the Conversation Intelligence On-Demand Execution API generally available. It lets teams run language operators when they choose, in addition to automated triggers.
Agent Connect and Twilio’s positioning
The same post highlights Twilio Agent Connect, the open-source toolkit the company introduced at SIGNAL in May for connecting agents built on outside frameworks to Twilio’s voice and messaging channels. Twilio lists integrations with Amazon Bedrock AgentCore, Google Cloud’s Agent Development Kit, Microsoft Foundry and OpenAI’s speech-to-speech models, along with support for custom agent runtimes. SDKs are available for Python and TypeScript.
Under this model the customer’s framework handles prompts, tools and business logic. Twilio handles the channels underneath.
Taken together, the two posts show where Twilio wants to sit. The company is not offering its own agent or model. It is offering the communications layer agents run on and a way to check what those agents said to customers. Twilio presents that as a way for companies to avoid being tied to one AI vendor.
What marketing and CX leaders should weigh
Several practical questions follow from the announcements.
Coverage is one. The operators see only conversations that run on Twilio. An organization with AI agents on several platforms would get a partial view from this tool and would need to decide how to supervise the rest.
Knowledge base upkeep is another. Hallucination Detection checks the agent against the articles a company has loaded. If a policy document is out of date, the operator would measure the agent against the outdated version. Teams that own customer-facing policies and product content may want a defined process for keeping that source current.
Timing matters for how the results get used. They arrive after a conversation closes, which suits quality review and trend analysis. They do not stop an incorrect answer from reaching the customer in the moment.
Regulated teams should note the PCI and HIPAA exclusions during the beta. Financial services and healthcare organizations will want to ask Twilio when that might change.
Cost and accuracy are open. Neither post gives pricing figures, customer results or accuracy benchmarks for the judge model. Buyers evaluating the operators will likely want to run them against a sample of conversations they have already reviewed by hand.
Twilio says this release lays the groundwork for a broader supervision layer. It has not published a roadmap or timeline.
Sources: Twilio blog: From Voice AI Demo to Production; Twilio blog: Twilio Expands Conversations Layer for AI Agent Supervision; Twilio docs: Supervise AI agents; Twilio docs: Conversation Relay; Twilio blog: SIGNAL 2026 product announcements; The Agile Brand Guide: Twilio SIGNAL 2026 coverage








