This article was based on the interview with NiCE CTO Kevin Lee on moving from AI pilot to production by Greg Kihlström, AI and MarTech keynote speaker for The Agile Brand with Greg Kihlström podcast. Listen to the original episode here:
The mandate from the C-suite is clear, and it has been for some time: “Do something with AI.” For enterprise marketing leaders, this directive has become a constant hum in the background of every strategic discussion. The most common response, and a logical one at that, is to spin up a pilot. It’s a contained, manageable way to explore the technology, demonstrate potential, and generate a quick win. The pilot is launched, the demo is impressive, the initial metrics look promising, and for a moment, it feels like progress. But then, something happens. Or more accurately, nothing happens. The pilot stalls, the momentum dissipates, and the promising new capability quietly retreats to a forgotten corner of a roadmap.
This scenario, which plays out in countless organizations, is what we might call the “million-dollar pilot trap.” The real risk in an enterprise AI program isn’t that the pilot fails; it’s that it succeeds, and the organization is still unable to absorb it into production at scale. Agility isn’t measured by how quickly you can launch a proof of concept, but by how effectively the organization can operationalize what works. This requires a fundamental shift in thinking—from short-term experimentation to long-term enterprise integration. In a recent conversation, Kevin Lee, CTO of the customer experience platform NiCE, shared his perspective on what separates the AI initiatives that achieve escape velocity from those that remain stuck in the pilot phase. His insights offer a clear-eyed roadmap for leaders looking to move beyond the demo and create real, sustainable business value.
The Fallacy of Low-Hanging Fruit
In the rush to show results, the natural inclination is to identify the simplest, most straightforward use case. We’ve all been conditioned to look for the “low-hanging fruit.” These are often the transactional, high-volume, low-complexity queries like “Where is my order?” or “What’s my account balance?” They seem like a perfect starting point for automation. However, as Lee points out, this approach is often a strategic misstep that can doom a program before it even has a chance to prove its worth. The ease of implementation is mistaken for business impact.
“What we’re seeing a lot of is organizations will pick the proverbial low-hanging fruit. Turns out the low-hanging fruit doesn’t always yield high ROI. And it, it’s not necessarily the wor- the juice is not worth the squeeze, as the saying goes… utilizing their data, the historical interactions that these brands have, mining that to understand- Which interactions are actually worth automating? Which ones will pay back the returns on that pilot and the resources that were invested?”
Lee’s point cuts to the core of the pilot trap. An AI initiative that successfully automates a low-value interaction may prove the technology works, but it fails to build a compelling business case for further investment. The focus, he argues, must shift from “what can we automate?” to “what should we automate to drive meaningful outcomes?” This requires a more disciplined, data-driven approach. Instead of relying on executive intuition about what seems easy, leaders should be mining their vast stores of historical interaction data. This analysis reveals the more complex, nuanced, and often emotionally charged conversations that create the most friction for customers and the highest operational costs for the business. While these use cases are more challenging to automate, they are the ones that deliver the significant ROI needed to justify a production-scale rollout. This is a crucial distinction: a pilot should not just be a technical test; it must be a value proposition.
Proportional Governance and the Black Box Problem
Once a high-value use case is identified, the next major hurdle is governance. This is often where innovative projects meet the unyielding realities of the enterprise, particularly in highly regulated industries like financial services or healthcare. The temptation for some is to treat governance as a box-ticking exercise to be dealt with after the “real” work is done. For an AI agent interacting with customers, however, this is a recipe for disaster. A cool demo that occasionally hallucinates is an interesting novelty; a production system that does the same is a brand-damaging, compliance-violating liability.
“You cannot have that black box when you ask 10 questions and you get 12 different answers. Right? It can’t be a magic eight ball. I ask 10 questions, I wanna know the exact answer that I’m gonna get back every single time… a single hallucination is unacceptable for highly regulated organizations, and that’s why we take those extra precautions and steps.”
Lee highlights a critical requirement for enterprise AI: predictability and reliability. The “magic” of generative AI is compelling, but it cannot come at the expense of determinism where it matters most. For regulated interactions, the system must provide the right answer, every single time. This means that from day one of the pilot, leaders must be thinking about guardrails, compliance, and the ability to audit and explain an agent’s behavior. It also means choosing the right tool for the job. A simple, deterministic AI or a standard NLU model is often the correct, and far more cost-effective, solution for straightforward queries. The expensive, token-burning frontier models should be reserved for the complex, multi-threaded conversations that genuinely require their reasoning capabilities. This isn’t about stifling innovation with bureaucracy; it’s about building a foundation of trust and safety that allows innovation to scale responsibly. A pilot that ignores these enterprise-grade realities is not a pilot at all; it’s a science fair project.
The Compounding Effects of a Platform Approach
The final piece of the puzzle, and perhaps the most strategically significant, is moving from a fragmented collection of point solutions to a unified platform. In the dynamic world of MarTech and AI, it’s tempting to chase the “best-in-breed” tool for every specific need. This can lead to a patchwork of technologies that are powerful in isolation but create an integration and maintenance nightmare at scale. The most mature organizations, Lee observes, have learned this lesson and are deliberately choosing partners who can provide a comprehensive, enterprise-scale platform. This decision is what ultimately unlocks the velocity needed to move from a single successful pilot to a transformative, organization-wide capability.
“The moment you say, ‘Okay, pilot to production,’ I’m getting 8%, 10% maybe of volume inside of chat. Where is the other volume coming from? Enterprise-scale voice… to be able to now turn on that same experience inside a voice at scale, that’s where the compounding effects of having a single platform are- become very acutely evident… you can start to see the compounding effects of having that, that platform that lets an organization focus on the value and not on the architecture.”
This is the scaling mechanism. A successful chatbot pilot is a great start, but it often addresses a small fraction of total customer interaction volume. The majority is still happening over the phone. With a point-solution approach, scaling the chatbot’s logic to the voice channel means starting over: finding a new vendor, building new integrations, wrestling with latency, and navigating a host of technical complexities that distract from the core business goal. On a unified platform, it becomes a configuration change. The same knowledge base, the same integrations into backend systems, and the same governance rules can be seamlessly extended from a digital channel to a voice channel. Furthermore, that same intelligence can then be used to augment human agents, providing them with real-time guidance and taking the cognitive load off their shoulders. This creates a compounding effect, where each new capability builds on the last, accelerating the delivery of value across the entire customer experience. It allows leaders to stop focusing on plumbing and start focusing on performance.
From Pilot to Production
The journey from an AI pilot to a fully integrated, production-scale system is less about technical hurdles and more about strategic foresight. It requires a deliberate shift away from the quick, easy wins that feel like progress but ultimately lead nowhere. The leaders who are successfully navigating this transition are those who are asking the right questions from the outset. They are using data to identify use cases that deliver real business impact, not just operational volume. They are embedding governance and compliance into their designs from day one, recognizing that trust is the price of admission in the enterprise.
Most importantly, they are making strategic platform choices that provide the foundation for speed and scale. They understand that true agility lies not in a collection of disparate tools, but in a unified architecture that allows the organization to focus on value, not on the underlying architecture. The mandate to “do AI” isn’t going away. For marketing leaders, the challenge—and the opportunity—is to answer that call not with a dazzling demo that fades away, but with a resilient, scalable capability that fundamentally improves how their brand connects with its customers. That is how you escape the trap and move into production.
