In this episode
Sushil Kumar, CEO at Cyara, argues that the real risk in agentic AI customer experience is not the failure rate but the absence of a plan for it: AI will get an enterprise 90–95% of the way there, and the executive’s job is deciding what happens in the remaining 5–10%. Recorded with Greg Kihlström, the conversation moves from strategic risk — customers blame the brand, not the bot — through the compliance failures that hide behind a “successful” call, to the concrete shift in how agentic voice systems have to be tested and monitored: pass/fail replaced by a multidimensional rubric, and one-time pre-launch testing replaced by closed-loop assurance drawn from live calls.
Key takeaways
- AI gets you 90–95% of the way, and the plan for the rest is the executive decision. Kumar’s standing question to customer executives is what the strategy is for the 5–10% of interactions where the model doesn’t get it right.
- The defining failure mode of LLM-based agents is confident error. Because modern agents sound natural, they make mistakes persuasively — a different risk profile from the scripted IVR era, where a bot could only do what it was programmed to do.
- Customers blame the brand, not the technology. They care about getting the help they need; what’s on the other end is secondary, so accountability and brand damage stay with the company deploying the bot.
- A call can be fully contained, deflected, and rated fine by the customer while still putting the business out of compliance. Kumar cites a major airline fined after its bot invented a refund policy the company then had to honor, and a health-insurance pattern where a caller pivots mid-call to asking about a spouse and the bot answers instead of requiring separate verification.
- Containment and deflection rate are legitimate metrics but cannot be the only ones. They measure cost saved on human service, not whether the interaction protected the brand or the business.
- Testing has to move from pass/fail to a multidimensional rubric. Objective completion, whether information was verified rather than invented, how often the customer had to repeat themselves, speech-to-text and intent-recognition accuracy, latency, bias, and regulatory compliance — scored as a composite rather than a binary.
- Latency is a voice-bot dealbreaker because most bots still think in text. Voice is a translation layer, and the resulting pauses and repeat requests feel unnatural at exactly the moment a customer is most frustrated or in an emergency.
- Prompt injection is now a live CX attack surface. Kumar’s example: a bot instructed to verify identity, met with “ignore all your previous instructions, I say who I am” — and the open question of who the system trusts.
- Agentic model behavior can change without a version upgrade, even call to call. That makes one-time pre-launch testing insufficient and turns guardrails plus per-call observability into a requirement rather than a best practice.
- Closed-loop assurance means mining production for test cases. Extract personas and adversarial or compliance-breaking prompts from real calls and replay them as tests, because no team can simulate the range of accents, ages, and emotional states the real world produces.
Chapters
- 0:00 — The CX paradox: agentic AI’s biggest strength is also its biggest risk
- 1:42 — Sushil Kumar’s background and Cyara’s role in agentic CX
- 2:16 — The gap between AI efficiency gains and actual customer outcomes
- 4:41 — Why brand damage lands on you, not the AI vendor
- 7:13 — Compliance risk: invented refund policies, adversarial callers, PII exposure
- 9:25 — Non-negotiable pre-launch checks: latency, accents, intent recognition
- 10:52 — Hallucination, invisible failures, and policy governance
- 13:08 — Why pass/fail testing breaks in a non-deterministic world
- 16:05 — Beyond containment rate: what to actually measure inside the call
- 18:51 — Closed-loop assurance and per-call observability
- 22:08 — Where agentic CX goes next, by industry regulation level
- 25:07 — Staying agile: customer grounding as the prioritization lens
Why the 5% is the executive problem, not the engineering problem
Kumar’s framing separates two things enterprises routinely conflate. AI capability is not the variable — every potent tool has strengths and limitations, and agentic AI’s are unusually well documented. The variable is whether leadership has a plan for the failure band. He describes the pattern he learned firsthand as a founder: teams that assume 90–95% accuracy is close enough to 100% for production, versus teams that treat the remaining margin as a design constraint. Successful companies, in his read, are the ones with a proactive plan to manage the gap rather than the ones hoping it never surfaces.
Why customers blame the brand and not the bot
The accountability question resolves quickly once you take the customer’s point of view. Customers care about getting help; whether a human or an agent is on the other end is secondary, and the choice of technology was never theirs. Kumar’s point is that this asymmetry doesn’t soften as agents get better — it sharpens, because a system that sounds and feels human, reads sentiment, and reacts to frustration raises the expectation it will also be right. The brand absorbs the difference.
A successful call can still be a business failure
This is the sharpest reframe in the episode. Kumar’s compliance example: a caller identifies themselves to a health insurer, then mid-call shifts to asking about a spouse. The correct behavior is to require the spouse to call separately. If the bot answers instead, the interaction scores as a successful containment, the deflection rate improves, and the customer is satisfied — while the company sits outside its privacy obligations. His airline example runs the same way from the other direction: a bot that invented a refund scheme produced a happy customer and a government penalty. Any measurement system that only looks at call completion will report both of these as wins.
Testing a non-deterministic system: the multidimensional rubric
Scripted IVR was exhaustively testable in principle — a million branches meant a million tests, but you knew what they were. Agentic systems are built to have agency, which is precisely what defeats branch-based validation. Kumar’s replacement is a rubric applied to every interaction rather than a binary applied to the last step: was the objective met, was quoted information looked up or fabricated, how many times did the customer repeat themselves, what was the speech-to-text and intent accuracy, was the system free of bias, did it hold up against adversarial input, and did it comply with the applicable regime — the EU AI Act among them. Cyara’s product answer is a composite scorecard across those dimensions.
Closed-loop assurance: from good practice to requirement
The old gap between test environments and production is now a live operational problem, because agentic systems are, in Kumar’s phrase, living and breathing — model behavior can shift without a version change, and from one call to the next. Two things follow. First, observability on every call with boundary conditions that trigger immediate action, rather than discovering drift later. Second, using production as the source of test material: extract personas and prompts — including adversarial and compliance-breaking ones — from real interactions and feed them back into the test suite. He is explicit that closed loop used to be general goodness, like going to the gym, and is now a requirement, because simulating the real distribution of accents, ages, and emotional states from scratch is not feasible.
Where agentic CX is heading, split by regulation
Adoption is bifurcating. In regulated industries — healthcare, finance — the entry point is low-risk: the L1 or router agent that replaces the scripted menu no caller wants to sit through, plus simple Q&A, with human handover for anything complicated. Over time the handover shrinks and the agent takes a larger share of the end-to-end journey, handling things like consolidating multiple family members into one account. In less-regulated industries the move is faster and toward active operations rather than lookups: setting up a new account, changing a flight because a connection is too tight. Kumar’s caveat on that second category is trust — the customer has no way to know whether the flight the bot offered actually exists.
FAQ
What is the biggest risk executives overlook when deploying AI in customer experience? Not the technology’s failure rate, but the absence of a plan for it. Sushil Kumar’s framing is that AI will get an enterprise 90–95% of the way there, and the executive question is what happens in the remaining 5–10%.
Do customers blame the brand or the AI vendor when a bot fails? The brand. Customers care about getting the help they need, not about what is on the other end of the interaction, so the accountability and the brand damage stay with the company that deployed the bot.
Are containment rate and call deflection bad metrics for AI customer service? No, but they cannot stand alone. A call can be fully contained and deflected while still breaching privacy rules or committing the company to a policy it never wrote — and the containment metric will score both as successes.
How do you test a non-deterministic agentic AI system? By replacing pass/fail with a multidimensional rubric: whether the objective was met, whether information was verified rather than invented, how often the customer repeated themselves, speech-to-text and intent-recognition accuracy, latency, bias, adversarial resistance, and regulatory compliance — scored as a composite.
What is closed-loop assurance in agentic CX? Monitoring every live call against boundary conditions, then feeding real production interactions back into testing by extracting personas and adversarial or compliance-breaking prompts and replaying them as test cases.
Where is agentic CX adoption heading over the next year? In regulated industries, from low-risk router and Q&A agents toward a larger share of the end-to-end journey with shrinking human handover. In less-regulated industries, toward active operations — booking changes, account setup — rather than balance lookups.
About Sushil Kumar
Sushil Kumar leads Cyara’s strategy and growth with a vision to redefine how enterprises build trustworthy, AI-driven customer experiences. A builder at heart, he has spent his career creating platforms that change how software is engineered, validated, and delivered. His focus is helping global organizations achieve new levels of reliability, speed, and customer trust. Before joining Cyara, Sushil was the co-founder and CEO of RelicX.ai, a generative AI test automation pioneer acquired by Harness. He previously led major product and business organizations at Oracle, CA Technologies, and Broadcom, where he built and scaled category-defining AI, DevOps, and cloud solutions adopted by thousands of enterprises worldwide. With more than 25 years of experience leading high-growth teams and multi-hundred-million-dollar product lines, Sushil is known for blending deep technical insight with a pragmatic, product-first approach to leadership. His work centers on transforming how modern software and customer experiences are created, making them more resilient, more intelligent, and more human. Sushil holds a Bachelor of Technology from BIT Sindri and an MBA from the Indian Institute of Foreign Trade. He lives in the San Francisco Bay Area.
Sushil Kumar on LinkedIn: https://www.linkedin.com/in/sushil-kumar-343780/
———- Resources ———-
Cyara: cyara.com
The Agile Brand podcast is brought to you by TEKsystems. Learn more here: https://aglbrnd.co/r/2868abd8085a9703
We’re proud to be a media partner for #MAICON26 – Oct. 13-15! Learn how AI can power your marketing and business and help you grow smarter. Use code AGILE150 to save! https://aglbrnd.co/r/7fe458ced0f04658
Reach your customers with Reddit. Spend $500 in ad spend, get $500 back in ad credit! Learn more: https://advertalize.com/r/491818c79fb1873f
Chaser is the only Slack-native project management platform that helps teams turn messages into tracked tasks, automate follow-ups, and maintain team-wide visibility, without adopting another tool. Now integrated with Claude and other GenAI tools. Learn more at trychaser.com and use code AGILEBRAND for a 3-month free trial (normal trial is 14 days).
The most influential minds in software, AI, and engineering leadership will be at WeAreDevelopers World Congress North America, September 23-25 in San Jose. Learn more: https://aglbrnd.co/r/60a7299222a7bcf1
Start building your own apps with Replit and get $20 off. Learn more: https://aglbrnd.co/r/93531742a7625a20
Enjoyed the show? Tell us more at and give us a rating so others can find the show at: https://aglbrnd.co/r/faaed112fc9887f3
Connect with Greg on LinkedIn: https://www.linkedin.com/in/gregkihlstrom
Don’t miss a thing: get the latest episodes, sign up for our newsletter and more: https://aglbrnd.co/r/35ded3ccfb6716ba
Check out The Agile Brand Guide website with articles, insights, and Martechipedia, the wiki for marketing technology: https://www.agilebrandguide.com
The Agile Brand is produced by Missing Link—a Latina-owned strategy-driven, creatively fueled production co-op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. https://www.missinglink.company
Transcript
[00:00:00] Greg Kihlström: What if your biggest CX innovation is also your biggest source of customer churn? Agility requires a clear-eyed view of both the promise and the peril of new technologies. It’s the ability to embrace innovations like AI technologies, not just for efficiency, but with a rigorous focus on the customer outcomes that build long-term value. Today, we’re going to talk about the delicate balance of implementing AI in the customer experience. Specifically, we’re gonna explore the rise of sophisticated agentic AI voice bots and the critical deal breakers that cause customers to abandon interactions and potentially your brand altogether. To help me discuss this topic, I’d like to welcome Sushil Kumar, CEO at Cyara. Sushil, welcome to the show.
[00:01:31] Sushil Kumar: Hey, Greg. Thanks for having me here.
[00:01:33] Greg Kihlström: Looking forward to diving in here, and certainly lots to talk about. But before we do, why don’t you give a little background on yourself and your role at Cyara?
[00:01:42] Sushil Kumar: Great. So I am the CEO of Cyara. Prior to joining, Cyara, you know, I did a startup that was also in AI-driven test automation space. but overall, I have over 25 years experience in all things DevOps, data. so I think, you know, this is a very pivotal moment for agentic transformation happening in the CX space, and Cyara sits in a very, very, opportune moment to help enterprises make the transformation. So really glad to be here.
[00:02:16] Greg Kihlström: Love it. Let’s dive in then. And wanna s- we’re gonna talk about a few things, but wanna start with from, from the strategic level and, and touch a little bit on, on what I teed up in the intro, the, just this, this idea of the, the CX, AI mandate and, and some of the risks involved as well. Your research highlights a significant gap between the business drive for AI efficiency, which I know a lot of people are feeling in their, in their roles, so the gap between that and the actual customer experience, the results from it. So from a strategic perspective, what are the biggest risks that leaders overlook when they go ahead, you know, they, they get points for green-lighting the, the AI project,
[00:03:02] Greg Kihlström: but, you know, they implement an AI voice or chat initiative but don’t have a deep understanding of some of the potential deal breakers involved in doing so?
[00:03:12] Sushil Kumar: that’s a great question, Greg. And just like every other potent tool, right, your ability to drive the most, the benefit out of that in a desirable way, depends on how well you understand its strength and limitations, right?
[00:03:27] Sushil Kumar: So, the, the most challenging thing about the latest incarnation of AI, which is what we call the LLMs or the, the agentic AI, is that it sounds so very natural, and when it makes mistakes, it makes mistakes confidently, right? So-
[00:03:46] Sushil Kumar: … the biggest challenge, I think, as the executive, and I learned that firsthand in my own startup, is I think AI can do amazing things for you, provided you understand where its limitations are and you plan for that, right? Because the biggest risk that I think, if I have to summarize it, is not know- I mean, or knowing but not having a plan to deal with the reality that AI is amazing, but it’s not perfect. So it’ll get you 90, 95% there, but what’s your strategy to contain that 5%, right? And those 5%, and I- I’m sure we’ll talk about what the impacts are, right?
[00:04:26] Sushil Kumar: But those impacts could be devastating for a business, for your brand, for your trust, right? So if there is one thing that I always ask my customer executives is, “What is your plan for that 5% or 10% when AI doesn’t get it right?”
[00:04:41] Greg Kihlström: and while there’s certainly some internal impacts on, on that, one of the, one of the things from the research also showed that when a customer has a poor experience with an AI bot, they blame the brand. they don’t bra- blame the technology, specifically, but it, it, it reflects on, on the brand. So how should executives be thinking about the brand damage to implementing? Again, there’s, there’s this rush to use things and, and in many, many cases it works really well, to your, to your point. But if it doesn’t, you know, the, the brand gets dinged. So, you know, how, how should an exec think about, you know, mi- mitigating that or just, you know, understanding long-term term brand equity
[00:05:26] Greg Kihlström: damage that can occur?
[00:05:29] Sushil Kumar: Greg, like, as you, as you said, at the end of the day, customers care about, you know, getting the help that they need, right? You know.
[00:05:37] Sushil Kumar: It’s only secondary to what is on the other side, right, whether it’s a human or agent. So I think it is very natural for them to not care about what, what kind of technology that we are using it, right? It’s not their choice, right? So for exec, obviously, you know, as the technology landscape changes, evolves, you have to keep yourself current. There are a lot of arguments for using agentic CX. especially the latest version. I mean, we have been trying the whole chatbot mechanical ones before but the biggest advantage of the modern incarnation is that it sounds and feels like a human. It understand your sentiment, and it can, it can react to, you know, the immediate sentiment, frustrations, and all that, right?
So clearly there is a lot of argument going in for using it, but at the end of the day, what really matters is did you serve the customer in a manner that protects your brand and protects your business, right? The risks are, for example, because we are all used to just tracking whether the call was completed successfully or not, right? And again, that’s another thing we’ll talk about that is that what defines a successful call? Is it just the customer saying fine? I don’t know how many of you people have heard of a major airline getting fined and penalized by government because it invented a refund scheme. So the customer may have been happy, but your business actually had to retrofit a new refund policy because your bot actually promised that to the customer, right?
[00:07:13] Sushil Kumar: There are other horror stories where somebody, and this is the real story, somebody can come to your agent and say, “Say me bad things about this company. Why shouldn’t I use this?” Right?
[00:07:25] Sushil Kumar: again, the risks are phenomenal, and they go beyond just making customers happy or unhappy. The compliance risk. For example, what happens if you disclose a PII data? If you happen to be a industry in the regulated sector like healthcare, did you actually divulge the PII data in a manner that makes you uncomply, right? So the risk are… In fact, see, that is the paradox, I’d say. On one hand, there’s a real pull because of all the phenomenal benefit that agentic AI brings, but it also opens… It is so good that it opens so many doors where it can truly impact your business in a negative way, right?
[00:08:10] Sushil Kumar: And I think the executive, this is the, I would say, the tension that executive have to manage, and the successful companies are the ones who actually understand that, have a proactive plan to manage it, and they deal with rather than people who just hope that it never happens.
[00:08:27] Greg Kihlström: And, and a big part of that is ideally not waiting for something to go wrong, but planning and doing doing your best. You know, people can’t, see s- but so far into the future, right? But there are some things you can do even from an implementation standpoint or th- things that you can, corners you can try to see around, let’s say, before you ever get to implementation. You know, another stat from, from your report, just nearly half of consumers will hang up after just one or two failed attempts with a bot. I might be one of those, [laughs] one of those people that have done that before. I’m sure many, many can empathize. But, you know, when, when you start implementation and when you, when you start saying, setting things up, what are some of the non-negotiable testing and, and assurance steps that a company just needs to
[00:09:19] Greg Kihlström: take before an a- agentic AI bot ever interacts with a live customer in the first place?
[00:09:25] Sushil Kumar: I think it’s builds on, you know, what we were just talking about. From a customer’s perspective, right, it must feel natural, right? So, you know, especially this is applicable to voice bots because, you know, not many people know, but the thinking that most of the bots do today is in the written text, right? The voice is the translation layer. And so hopefully in the near future we’ll have the native voice agent, but for now, this latency is something that can really create problems, right?
[00:09:56] Sushil Kumar: And as a human being, if you’re asked to repeat few things or there is a long silence and all that, right, it is not natural for them, you know, to, to interact that way, right? And oftentimes when you need the help, when you are most frustrated or you have something really emergency happening, you really need to have that trust and comfort level, right? So from a customer perspective, does the interaction feel natural? And technically, you know, it means that, you know, making sure that the intent is being recognized. I mean, the other thing is about is that we are all people. We all have different accents. Does it understand all the accents predictively, right?
[00:10:36] Sushil Kumar: you know, and, does it give the truthful answers, right? Does it make the customer repeat many times? So that’s probably the non-negotiable part, and I would say a lot of customers do understand it. How well they are prepared for it is another story, right?
[00:10:52] Sushil Kumar: But there’s the other big angle is, which is something new because, this agentic world can hallucinate or make up information. The failures are not visible, right? Before, in a scripted world or if you’re a human being, the chances of it basically making things up, you know, is not that high. In a scripted IVR world, it can only do things that you have programmed it, right? A human being, if they lie once or twice, maybe you can find the bad apple and you can fire them, right? But that assurance doesn’t exist, and to the customer it doesn’t matter whether it is a human being or a bot, your accountability still remains the same. So this whole compliance governance is that can you help my customers in a manner that protects my brand, protects my business, and helps it keep compliant are the biggest thing that we have to think about, right? To, to summarize again, does it lead to a natural and trustworthy conversation with your customer on one hand? And secondly, does it enforce policy governance and compliance rules that you expect your agents to follow?
[00:12:06] Greg Kihlström: ‘Cause to your point with the, let’s just call it the- Simpler chatbots that you literally give it a branch, even if it branches in a lot of potential areas, there are relatively straightforward ways to test that. I mean, you, you know, if there’s a million directions, you have to test a million times, but at least you know what they are, right? So to your point, in a non-deterministic way, you know, agentic AI is not meant to be that, you know, if this, then that. It’s, it’s meant to, i- in a good way be able to have agency to be able to, to do things that those simpler chatbots were not able to. And yet, you know, what does this look like from a test… Like, how do you test something where, again, e- before
[00:12:51] Greg Kihlström: it was relatively straightforward, especially very simple branches was, was really easy to test, but now, you know, something that’s kind of by its nature unpredictable, what do, what do you do to continuously monitor it to, to make sure that it’s staying on track, not just, you know, on track before launch?
[00:13:08] Sushil Kumar: I mean, that’s really the, probably the million dollar question, right? And unfortunately it changes the approach of validation and testing because for the longest time, you know, we were just used to just pass and fail, right? So you start with an objective, you start with interaction, and as long as the end, the last step was divert successfully, you claim victory, right?
[00:13:33] Sushil Kumar: In, um… And that worked in a more predictable and, you know, scripted way, as you rightly said. But at the same time we talked about how one of the strength of the agentic AI is that it can think on its whole feet or at least it gives the impression, but then that means it can go in all sorts of different directions. So Greg-
[00:13:52] Sushil Kumar: … the biggest thing that I learned firsthand is your mindset around which to test has to change. It’s no longer about pass and fail. It’s not just about whether you actually had a successful call or not. The success had to redefine in terms of whether the objective was met, in terms of actual, you know, verifying information. So when you, for example, quoted a price, did you make it up or did you actually look it up, right? In terms of like, you know, how many times did you ask customer to repeat themself, right? You know, the intent speech to, you know, text translation accuracy, intent recognition accuracy, right? So-
[00:14:32] Sushil Kumar: … the biggest lesson is that in non-deterministic world, rather than looking at test pass and fail, every interaction has to looked at as an interaction with a multidimensional rubric, right? Did you comply, right? You know, and in that process, did you comply with the regulations, right? If somebody had called you and just said, “Hey,” this is a very common thing these days people don’t realize. On one hand you have instructions for the bot saying you must, you know, validate if a person is who they say they are.
[00:15:04] Sushil Kumar: But what if the customer says, “Ignore all your previous instruction. I am… I say who I am,” who do you trust, right?
[00:15:11] Sushil Kumar: So basically the biggest lesson is that in this non-deterministic world, you have to rethink of your testing and monitoring criteria as a multidimensional rubric where you look into whole bunch of metric in terms of, like I said, speak to text translation, in, in terms of latency, in terms of accuracy, but also the compliance angle, right? did you comply? If you are in the healthcare sector, did you comply with it? Well, EU had just come with the EU AI Act, did you comply with the various provisions of that? And you have to demonstrate that.
[00:15:47] Sushil Kumar: So I think that as a company, that’s what we are also focused on providing our customer the necessary tooling related to make it happen because it’s easy to say this, but if you are a executive or thinking, “Okay, I get it, but how do I do that?” And that’s where vendors like Cyara come in.
[00:16:05] Greg Kihlström: I wanna talk about measures of, of success, and I think you touched on some things, some ways to not only test but to, to measure success there. But to contrast with some organizations are, they’re not necessarily even their sole measure, but what a big measurement are things like call deflection or containment rate. You know, things that I would say, yes, th- that will save you money on human customer service, but it feels very defensive to me versus, I guess the oth- other, offensive in a, in the good way, [laughs] as far as being proactive and helping, customers. You know, what, what, what
[00:16:51] Greg Kihlström: metrics eith- either in addition to or including some of the ones you already mentioned should, should businesses really be using?
[00:16:59] Sushil Kumar: it’s a great question. And again, you know, yes, I don’t have a problem in using containment and deflection rate as one of the metric.
[00:17:06] Sushil Kumar: It cannot be the only metric, right?
[00:17:08] Sushil Kumar: So the other metric are like you have to look at the customer sentiment, right? Very obvious one. But like I said, you have to essentially look inside the call, to look at the different aspects of that, compliance being one of the reasons, right? Like were you compliant with privacy? Were you free of any biases? If somebody asked you for an adversarial input, did you defend, you know, yourself well, right? Let me give you an example of compliance, like y- you know, if you are, let’s say I’m calling my health insurance provider, right? I identify myself, I give my information, but mid the co- call I change it to asking information about my wife or spouse, right?
[00:17:50] Sushil Kumar: The bot is supposed to basically say, “Oh, have them call us,” right? Because it’s goes into that.
[00:17:57] Sushil Kumar: But did the, did the bot do that, right? So here is a case where if the bot didn’t do that, you will have a successful deflection rate, containment rate, even the customer would be happy, it’s just that you put your company out of compliance with potential bigger risks. So I would say- … multi-dimensional rubric that looks against the compliance with the regulations, the policy, the biases, right, and the fact check, right? Did it actually give the right information, right? And at the same time looking at the customer sentiment, look at, like, the accuracy of speech-to-text translation. So there’s a whole bunch of metric that you have to look at, right? And, and-
[00:18:36] Sushil Kumar: … so one of the ways that we are solving, you know, helping, our customers solve that is to help them define this scorecard with all of these dimensions and have the scoring based on the composite scoring across all these dimensions.
[00:18:51] Greg Kihlström: And, and so to, to think… Kind of go back to the, the feedback loop and, and continuous improvement topic here. You know, one, one of the things that I like about more automated systems is that I would say natively it’s able to capture data about a lot of different things and feed that back. It’s not the human agents can’t enter notes or even systems can augment them or whatever, but these systems kind of by design are, are collecting data. And, and so in addition to training their own systems, how can that… how can some, at least, of that information best be fed back into an overall feedback loop so that the organization, you know, human and, and machine get better across
[00:19:36] Greg Kihlström: the journey?
[00:19:37] Sushil Kumar: Yeah. Yeah, Greg, this is another thing because for the longest time we the practitioners have felt that, you know, the testing environment don’t represent the real production, but nowhere else in the past has been more important than now, right? Because, you know, agentic things are living and breathing things, right?
[00:19:57] Sushil Kumar: So first of all, in the past, you tested things when you made some changes to the IVR flow or you made some code changes, and because they were very restricted in what they could do, you could do the testing once and you could do some basic monitoring, but then because they were-
[00:20:13] Sushil Kumar: … not very capable. Now I think, you know, because these are living and breathing things and the model behavior could change potentially every single day, right, even without any version upgrade, it could change from one call to another call. So the first thing that I said, you know, in the beginning is if you want to succeed with agentic CX, you have to plan around its limitations, and one of the things is don’t plan on perfection. Plan the fact that it might sometime veer into directions, right? So you build guardrail, you build observability, right?
[00:20:47] Sushil Kumar: So the first thing is to make sure that you are watching every single call, right? And make sure if it goes beyond a certain, you know, boundary conditions, you take immediate action as opposed to just finding it out later on. But more importantly, using that information to feed back into testing because it’s very hard for people to simulate what a real world looks like, how many different… A person with a different accent, different age, different gender, right, you know, talking in a certain, you know, emotional state, right? It’s very hard to foresee all those things. So being able to take a live call and from there to do things like extract personas or even extract prompts, that could be adversarial prompt,
[00:21:32] Sushil Kumar: that could be a prompt which is putting you out of compliance, and making sure that you use those paths to test the real world is how you cle- the… create the closed loop assurance. So while the concept of the closed loop has existed in the past, it was generally considered a general goodness, like all of us going to the gym, all of us should be losing weight. Now it is more of a requirement because without that closed loop, it’s next to impossible to make sure that you have tested and validated your agentic, you know, bots for the real world audience.
[00:22:08] Greg Kihlström: Well, and as things move on, these, these, agentic bots are going to be more sophisticated, and they’re gonna be able to do more thing. You know, part of the, the promise of agentic is that it’s, it goes well beyond the, the question and answer chat interface or voice chat or, or, or whatever. Where, where do you see, you know, looking out in the next year or so, you know, where, where do you see capabilities emerging that will be more commonplace? May… You know, they might be present now, but they’re, they’ll be more commonplace in, in these types of interactions. And, you know, what kind of opportunities as well as challenges may they present for brands?
[00:22:51] Sushil Kumar: it’s a great question. so, you know, what we see with the customers is bec- And it, it varies from industry to industry, right? So, you know, in more regulated, you know, industries like healthcare, finance, you see people making this move, but starting with something low risk, right?
[00:23:11] Sushil Kumar: You know, what they call L1 agent, the router agent, right? The basic one that seem more intelligent than the very scripted IVR, right? Where you couldn’t figure out which option to choose, right? I don’t have to wait to hear for all the prompt before I pick any of it, then realize none of these actually matches my-
[00:23:28] Sushil Kumar: … right? So a lot of people are doing that over a period of time. And then for simple Q&A, they have the agent, or the bot handle that, and then for complicated thing, do the human handover, right?
[00:23:42] Sushil Kumar: Over a period of time in those industry, you will see the human handover, getting fewer and fewer, right? That mean they will be able to do more complicated in, in interactions, and they will be able to do things that are even slightly more complicated, right? For example, like, you know, if you’re in phone, consolidate, you know, multiple family, you know, members into a single, you know, family account and all that, right?
[00:24:06] Sushil Kumar: for less regulated industry, you’ll see a faster adoption where it’s no- no longer about telling me what my balance is or give me the refund. This is about like, “Hey, can you set up my new account?” More active operations. “Can you book… If you are an airline, can you make these changes to the flight because my connecting flight had to short,” right?
[00:24:27] Sushil Kumar: Those are some of the things that are a little hard, and people simply lack trust You know, who knows if the flight, the flight that was told you that exists, is it a real flight or a made up flight?
[00:24:38] Sushil Kumar: So I think this is, we are in a secular transformation world where agentic CX is here to stay, and it will make more inroads, you know, in more regulated industry, it’ll go from being just the router agent to be a more, taking a bigger part of your end-to-end journey. In a more consumer-based industry, it’ll start doing more and more things active, right? making changes and doing active operations, right?
[00:25:07] Greg Kihlström: Yeah. Yeah, love it. Well, Sushil, thanks so much for joining the show. Got one last question for you as we wrap up here. what do you do to stay agile in your role, and how do you find a way to do it consistently?
[00:25:20] Sushil Kumar: yeah, so the best thing is, again, to me, what makes any business successful is how grounded we are to what the customer needs are, and not just in terms of what they’re telling us, right? Because oftentimes we rely on customer tell us this is what they need. This is a time where they are also figuring out what it means for them. So being a partners in terms of jointly navigating this and making sure that we are grounded on helping them successful is what, you know, help me stay focused as a business. And basically applying the same lens to prioritizing things that we do, which is, what is the most important thing we can do today to help them, also helps you, like, making sure that you’re doing the right things at the right pace at the right time, right? So again, I would say, you know, customer obsession, customer grounding, and a extreme focus on making customers successful is what makes, you know, us successful.
