It’s hard to escape talk of AI these days, but not all AI is the same, and not all of it is safe for large organizations to use.
Today we’re diving into the evolving world of generative AI for the enterprise with Sumeet Agrawal, VP of Product Management at Informatica. We’ll discuss strategies and considerations for building robust, enterprise-grade generative AI applications.
About Sumeet Agrawal
Sumeet Kumar Agrawal is a VP of Product Management at Informatica. Based in the Bay Area, Sumeet has 15+ years of data engineering and product management experience, driving innovative products within the cloud technology sector. He leads the Cloud AI/GenAI, Analytics & Data warehouse & Data lake, and iPaaS product portfolio at Informatica. It consists of multiple product lines such as Cloud Data Engineering, Streaming, big data, NoSQL, Serverless, Cloud Mass Ingestion(including CDC), Serverless & AI/ML, GenAI, API, and App Integration initiatives. He has a lot of experience working with many cloud ecosystem vendors like AWS, GCP, Azure, Snowflake, Databricks, etc. Apart from this, Sumeet has been frequently recognized as a strong communicator who has successfully worked with people from broad socio-economic backgrounds, and diverse cultures, building strong and fruitful organizational teams. He sits on the advisory board of many startups.
Resources
Informatica website: https://www.informatica.com
Register for the Medallia CX Day webinar: Building Loyalty: How Top Brands Create Forever Customers with CX – https://bit.ly/3M7dkQM
Connect with Greg on LinkedIn: https://www.linkedin.com/in/gregkihlstrom
Don’t miss a thing: get the latest episodes, sign up for our newsletter and more: https://www.theagilebrand.show
Attend the Mid-Atlantic MarCom Summit, the region’s largest marketing communications conference. Register with the code “Agile” and get 15% off.
Register now for HumanX 2025. This AI-focused event which brings some of the most forward-thinking minds in technology together. Register now with the code “HX25p_tab” for $250 off the regular price.
Check out The Agile Brand Guide website with articles, insights, and Martechipedia, the wiki for marketing technology: https://www.agilebrandguide.com
The Agile Brand podcast is brought to you by TEKsystems. Learn more here: https://www.teksystems.com/versionnextnow
The Agile Brand is produced by Missing Link—a Latina-owned strategy-driven, creatively fueled production co-op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. https://www.missinglink.company
Learn more about your ad choices. Visit megaphone.fm/adchoices
Transcript
Greg Kihlstrom:
Today, we’re diving into the evolving world of generative AI for the enterprise with Sumeet Agrawal, VP of Product Management at Informatica. We’re going to discuss strategies and considerations for building robust enterprise-grade generative AI applications. Sumeet, welcome to the show. Hey, Greg, thank you so much for having me. Yeah, looking forward to talking about this with you. Definitely a topic on a lot of people’s minds. Like I mentioned, you know, we’ve talked about this from a few different angles on the show already, but it’s, you know, just so quickly evolving that it’s, you know, I think there’s just always, always more to talk about here. Before we dive in, though, why don’t we start with you giving a little background on yourself and your role at Informatica?
Sumeet Agrawal: No, absolutely, Greg, and thanks again for having me here. And my name is Sumeet Agarwal. I lead the product management team at Informatica, and I’m responsible for mostly of their AI analytics, iPaaS product lines. I do a lot of work with Gen AI, and I have a lot of experience working with enterprises on Gen AI, so very excited to join this podcast and share my point of view.
Greg Kihlstrom: Great, great. Yeah, definitely. You’re the right person to be talking about with this topic. So looking forward to it. So why don’t we start with just give us an overview of what you’re seeing, what’s the current state of generative AI and enterprises today?
Sumeet Agrawal: Absolutely. I talk to a lot of enterprises and most of these enterprises are either implementing something with Gen AI, piloting something with Gen AI, or something has already been in production. In fact, the number which was just 2% or 3% earlier in 2023, Now that number has gone to 55%. When I say 55%, so these are the JDI projects which are either in pilot or in production. And that is just the start of last year. And if I fast forward, there is a big prediction that by 2026, 80% of enterprises have either implemented Gen AI, or they will be in production, or they will be at least piloting with Gen AI. So Gen AI is real. It is something top of mind for many executives. In fact, it’s a board-level initiative. And recently, there was a survey done, and there were around 89% of execs have ranked AI and Gen AI as one of the top priority for their organization. So it’s a big, big thing in many big enterprises. We’ve also seen enterprises either trying to enhance their existing use case for better productivity with Gen AI, and many of them are also exploring new use cases. So it’s pretty real.
Greg Kihlstrom: Yeah, yeah. And yeah, 80% adoption, that’s, that’s pretty prevalent. And it certainly tracks with, you know, everything I’m hearing, I know, and I’m sure, you know, what, what’s what others are hearing as well. You wrote a blog post recently that talks about the necessity of moving beyond traditional data science tools. I mean, you know, let’s just say pre chat GPT, because I know everyone’s familiar with that, you know, this realm was pretty much relegated to data scientists. And you know, AI was kind of synonymous with, with the data scientists and data science tools out there and stuff like that. But you know, you’re, you wrote a post recently talking about, you know, why we need to not, not scrap them altogether, but move beyond solely using data science tools when developing generative AI apps. What’s the reason for this? What are some limitations and why look at this in this way?
Sumeet Agrawal: That’s a good question, Greg. In fact, when I last checked, I saw that there are more than 17,000 Gen AI companies or startups which has come up just since last December of 2023. So you can imagine the kind of innovation and the kind of vendors which are coming in into this space. It’s changing every day. As we speak, it’s changing. The new best practices are coming up. And it’s extremely hard for enterprises to keep track of what’s going on in the market. And my main reason of saying that you need to look beyond these traditional data science tool is because, especially when you’re deploying any technology in enterprises, it’s not just about deploying the tech, you also need to deploy it in production with the enterprise grade standard, right? You need the same standard like operationalization, security, governance, you need to be wary of cost, you need to know how easily it can scale, how it can be deployed in a different runtime environment. So all these things are sometimes an afterthought in many of these newer companies or many of these initial, I would say, early science tools. And that’s where we have seen many of the organizations struggle. They start with something which is shiny, and they take these newer tools, and very soon they realize that they need all these functionality. And that’s where it becomes hard for them to come out of that and really put that in production. So in my opinion, these are some good tools for prototyping and experimenting. But what you need is not a science tool. You need something which can be deployed in production. So just be wary of that.
Greg Kihlstrom: Yeah. And building on that, you know, you, as you mentioned, you work with a lot of enterprise orgs and, you know, where, where do you see the biggest obstacles in, you know, is it, is it related to kind of the tooling and, you know, but where, you know, where, where are you seeing some of the biggest obstacles in kind of adopting Gen AI?
Sumeet Agrawal: Yeah, I think it’s definitely tooling is one of the big aspects, but there are other issues, right? Like, for example, if you are building a Gen AI projects within the enterprises, the first thing I see most of the enterprises struggle is the skill set gap. As I was saying, as I was sharing you before, right, there are 17,200 startups that just came out. How hard it would be for enterprises to to really keep up with all the newer things which comes up in Gen AI, all the new best practices and everything. In fact, McKinsey did a survey recently and they found out that there are around 60 to 80% of enterprises find it extremely difficult to hire the right talent for Gen AI. And even if they are able to hire them, it is extremely difficult for them to retain these talent because these talents are rare and they don’t tend to stick around. So that’s, I would say, the number one obstacle I see with enterprises is the skill set gap and hiring the right talent. The other big issue I see is the data issue, because your GNI is as good as the data you are using it for. And here, especially when you’re building any GNI projects, you’re looking at not just the traditional structural data, you’re looking at unstructured data, you’re looking at data coming from different latency, real-time batch. And especially for CDOs, having the right data strategy is crucial for GNI value, and many of the organizations do struggle to get the right data needed for GNI, whether it’s getting the access of the data or even to prepare your data for GNI is extremely hard. And the third thing I would say is the similar problem we face, right? Like these many of the organizations kind of pick up a smaller tools or a science tool and they start facing issues with enterprise-wide adoption where we see that there is a survey which was done recently which says that 80% of your projects will fail, like won’t go beyond POC, and that is because of the lack of enterprise-wide standards, right? You don’t have a good standard for model building, training, deployment, monitoring. All that is the key reason why you kind of fail the Gen AI project. So these are the three reasons I generally see what companies are struggling with.
Greg Kihlstrom: Yeah. And so, you know, what, what do we do about this then? You know, are there, are there lessons to be learned? I mean, we’ve certainly been through plenty of technology disruptions before, like, you know, even the, the beginnings of, of the internet and, you know, for, for enterprise, big data, cloud, you know, what can, can enterprises learn from some of these things? And, you know, if, if so, you know, what, what are some of the strategies to use to overcome some of these things?
Sumeet Agrawal: Yeah, it’s a great question. In fact, I would say that for big enterprises, first of all, you need to start small. I mean, don’t have a very large goal where you can’t measure that goal. I would say start small. have a very focused goal in your mind with the business outcome in your mind. And you should build a team which is cross-functional team, which can achieve that goal for you easily. And the goal should be such that it is measurable. And especially when you are using Gen AI, the other thing which I missed in the last point is the cost, right? So be wary of the cost. of the point gen AI solutions. Remember some of these large language models runs on GPUs and your cloud budget will shoot very soon. So I think be wary of that. and make sure that you do show the initial success to your stakeholders, and then go on to the other use cases. So, I mean, that’s what we have done when even whenever the new technology came, whether it’s big data cloud, it’s always starting small, showing the progress, showing the success to your business, and then moving on to the other big things.
Greg Kihlstrom: And so, you know, one of the things that I think not only has made generative AI so popular so quickly. But also, I think one of the powerful things about it is that it is, it is kind of a democratizing technology. I mean, all just about anybody can, you know, go on to chat, GBT, or Google Gemini, or, or this or similar platforms and do something with it, right. So, you know, consumers, in other words, can can even do something with these technologies. But as we know, there’s a big difference between what works for consumer or even, you know, small business and what works well in the enterprise. And so, you know, I want to talk with you a little bit more about some of those distinct things that the enterprise really needs and, you know, things like scalability, security, privacy, all those things. You know, I’d like to start with scalability. which is certainly an issue, even if you’re doing a proof of concept or a pilot project or something, it’s, you know, scalability is always going to be an important aspect. So can you elaborate on, you know, what are some of the strategies for ensuring that a Gen AI application can scale effectively within a large organization?
Sumeet Agrawal: Yeah, no, it’s a great question. And that goes back to the point we were making that whenever you deploy a Gen AI solution, don’t think of it as a science tool. Think of this as an enterprise solution, right? And the reason why you have to think about it as an enterprise solution is because scalability is very important, right? And when we talk about scalability, I think the very first thing is the infrastructure and architecture you pick. So you need to be very wary of what kind of infrastructure you pick, which kind of cloud ecosystem you want to pick. And many organizations have a relationship with some of the cloud vendors, whether it’s AWS, Azure, Google. So make sure that you pick the right infrastructure, which gives you the right scale, as well as the discount you are looking for in terms of your cost. Then the other big important thing is that how are you doing your model optimization? I mean, whether how are you using your GPUs, etc. So we bury of that. Deploying it on containers, making it serverless, all these are the important strategies you have to make. Generally, if everybody start deploying it on their own and they don’t redo any kind of reusability, then your cost will definitely go for, you will definitely have a cost overrun issue. So having scalability deployment with containers, dockers, having serverless, as your architecture is an important part of it. Cost management, as I said, is important, like having a good strategies of cost management, which can help your overall on your scalability is important. Governance is important, like who is using it, how many people have access to the GNI solution is important, and how you govern it is extremely important. And last and not the least, I think reusability is extremely important. I think before you need to build a reusability framework at every level so that anything which is built by your team is being reused across the board so that not everybody start building their own language model or any of the modeling aspect. They kind of first see what is available and then start building on it.
Greg Kihlstrom: Yeah, I mean, along those lines, like in your experience, I mean, I think most, even if they’re not a CIO, or, you know, in an engineering team or data team, you know, most people are familiar with the concept of, you know, cost for storage and data and stuff like that. Or is Gen AI adding with GPU usage, all that stuff? Is it adding a another layer of complexity, like are, are, are most people that you’re talking with in the enterprise kind of aware of the, you know, the cost structure or the cost structure differences, you know, there’s a lot of, for instance, like chief marketing officers and, and people listening to this show that again, very familiar with cloud, you know, cloud and, and, and a sense and stuff like that. But is, is it, is it a lot different? Is it a little different? You know, what, what, what are your thoughts there?
Sumeet Agrawal: Yeah, so it is definitely going to add another layer of cost in your overall budget, right? Because remember, first, when you were doing your digital transformation, you were moving from your data centers to cloud. And when you had a good FinOps strategy on how to use containers or how to use your EC2 instances to get your better cost, Now, when you start deploying your Gen AI solutions, now you have to think about how will you deploy these language model or large language model. And some of them tend to be extremely expensive. And like, say, for example, if you go and deploy Azure OpenAI solution in your private interface, it is going to cost you. And then if you try to do like generally, just deploying the vanilla LLMs is not going to help you. Sometimes you need to fine-tune it or on top of your vanilla models with your enterprise data, and that tends to be even more expensive. So yeah, having the right strategy on how to deploy these models, how, when you are doing fine-tuning, how to, what kind of right infrastructure to use, who should have access to, because some of, if you’re using, for example, public Gen AI solutions, then how much query you are making is going to cost you based on that. So who should have access to your Gen AI solutions also makes a huge difference on your cost. So yeah, all these are the different factors which you have to look in when you are deploying your Gen AI solutions, especially in terms of cost.
Greg Kihlstrom: And so, you know, there’s I think most of us have been through the some of the changes with consumer data privacy, you know, starting essentially with GDPR. And I know there’s there’s quite a few regulations here in the United States, as well as elsewhere in the world. It’s it looks like we’re basically going to be going through similar things with with AI and it appears like Europe is kind of leading the charge again here, but elsewhere in the world is going to quickly catch up as well. How can enterprises ensure that the Gen AI applications they’re currently working with are, you know, keeping these things in mind, again, anyone that’s been through the GDPR and CCPA and all those things with consumer data privacy know that these things are often start out a little, let’s call it ill-defined, but get more defined over time. So, you know, what’s your recommendation there and just, you know, building something that’s going to be able to comply with something that is kind of a work in progress right now?
Sumeet Agrawal: This is an extremely important question and an extremely important area for any enterprises to look into, that having a right governance standard is extremely important. In fact, I’ll tell you a story where we have released a feature within Informatica, which helps customers to build their own. I mean, it’s something where you can use a chat prompt to actually build Informatica solutions. And we have seen customers asking a lot of questions around how you train your models, what kind of compliance you have used, and all that is extremely important. So I think I’ll give you tips which I kind of learned from my customers, speaking to my customers. I think the first thing is establish a clear ethical guidelines, right? Like what kind of guidelines you want to align with. You mentioned GDPR, CCPA, all these are the different regulatory compliance, and you need to stay updated with these regulatory appliance. They keep on changing, so make sure that you are keeping up to date on these compliance. You need to also build a robust governance framework to oversee development and deployment of AI applications, which means whichever, however your engineering team or your AI team is, how they’re building these Gen AI solutions. You need to build a framework to see how they are building it, what kind of data they are using for training. And then you need to adopt bias mitigation strategies, right? You need to detect and mitigate biases in AI models. And just to, you need to build a framework which can actually understand that, hey, this data which is getting trained here, it is biased, it doesn’t represent everybody here, and it doesn’t have the, or your algorithm which you’re using is not, it doesn’t have the full fairness. So you need to build that mitigation strategies and also the bias framework, identifying the bias framework. And then also engage in stakeholder consultations. You need to have a diverse set of stakeholders in your AI development process to gain the wide range of perspectives. You cannot have just stakeholders with, for example, only males there. You need to have stakeholders, which is more diverse in different angles. You need to provide training and education to everybody who is using AI, especially ethical AI. You need to do that. And then finally, you also have a monitoring and auditing framework, which can monitor how well are you doing in this area, and especially on the ethical and regulatory areas, and then how you can conduct regular audits to identify and address any issues. So these are some of the strategies many of the organizations are taking to be very fair and to have this kind of AI governance in play.
Greg Kihlstrom: Yeah. Yeah. And, you know, as, as you mentioned, there’s, there’s a lot of organizations that have already, you know, started down this path. There’s, there’s many more that are starting down this path as an organization is looking at, you know, let’s say they have a pilot project or two, or even some, some, you know, fully functioning apps out there. What’s the, you know, there’s always a tendency to, kind of reached for something brand new versus maybe build on the existing, what would your advice be? Obviously knowing that there’s a lot of use cases out there, but how should organizations be thinking about kind of creating net new applications and use cases for Gen AI versus kind of building on existing ones? What should guide them?
Sumeet Agrawal: It’s an important thing, especially when you’re getting started with Gen AI. One of the very big dilemmas many companies have is, shall I enhance what I already have or shall I look for newer use cases? I think my answer is very simple. It all depends on how mature you are as an organization in your Gen AI practice, how good is your team, how aligned your stakeholders are on this. And especially if you are starting new with Gen AI, I think my approach is that I mean, always start with enhancing your existing use cases, because your existing use cases is already being bought by your, I mean, your business has blessed it. If you can make it more efficient and more automated, there’s nothing like it. And it helps you to manage your risk. It helps you to show your easy win to your organizations, control your costs, because many of that framework is already built. And then once you have a solid understanding and operational base with Gen AI, then you can start exploring with the newer use cases to drive innovation and capture additional market opportunities. But if you start everything from brand new, it will be difficult, first of all, for you to get the buying from your business and then to show the value it is adding up, right? So I always tell my customers that take something which you already have and see how you can enhance it and then and move on from there, right? And you will definitely see a big success with that strategy.
Greg Kihlstrom: Yeah, yeah, that makes sense. Just to talk a little bit about, you know, since you have a good purview of not only what’s current state, but as well as what’s ahead. And I know you mentioned some statistics about just how how fast organizations are adopting these things. What do you see as far as what’s ahead in the months and years to come with generative AI? What would your predictions be for the months ahead?
Sumeet Agrawal: Yeah, so I think, in my opinion, Gen AI will be widely adopted across industries. I mean, today many of the industries are either experimenting or piloting with it, but I definitely see that in the next few years, it will be widely adopted across industries, whether you’re in healthcare, fencer, retail, I mean, you will see that it will be widely adopted and it becomes a commodity. I mean, it’s just like, Today, if you go to internet and search, I mean, it is not like a big deal, right? I mean, in the few years, I mean, many decades back, it was such a big deal of using internet. So, in my opinion, it will be widely adopted, it will become a commodity. There will be a lot of advancement in creativity and content creation with Gen AI. I mean, I’ve seen, I mean, in my opinion, many of the I mean, a lot of things have been changed with JNI coming into picture on the content creation and creativity side. And I kind of think of JNI to be something like a human assisted tool, like in human and AI collaboration. will change with Gen-AI and there’ll be more and more. Gen-AI will become like a co-pilot for human, for collaboration. And then as we discussed, right, with all these things, there are two things which also will change is how you do ethical and regulatory frameworks. There’ll be more and more of such things will come up. Every country will have something on their own. And if you’re talking to it, you need to comply to some of these regulatory and governance policy. And since JNAI is always cost sensitive, so there will be definitely a cost angle. I see that more than large language model, language model will also come into picture. I mean, it’s already there. So whether it’s small language model or just the language model, those will also be more prominent in the next few years. So overall, I believe that there’ll be a big adoption of JNI and it will become more and more mainstream in the next few years.
Greg Kihlstrom: Great, great. Yeah, well, we’ll have to have you back on the show to follow up on a few of those things as well. But yeah, no, exciting stuff. Well, thanks so much for joining today. One last question before we wrap up here. You touched on a lot of great stuff already. So even if it’s a point that you’ve already made so far, what is one step that you’d recommend that an enterprise should take to just focus on the right things with Gen AI?
Sumeet Agrawal: Yeah, so I think the one thing if one step I would all I’ll recommend all of you is develop a clear and strategic roadmap for your journey initiative. It’s very easy to get distracted with so many things going on. But in my opinion, start small, take a very business outcome focused use case and see how it can add value to your existing businesses and establish a framework which can measure the success of your use case. And also think about investing in ethical governance. These are all the different things you can do to get started with Gen AI. And once you see the success, then scale beyond your existing use cases. So that would be the one thing I would recommend all the enterprises listening to this podcast to take back on it. And Greg, again, thank you so much for having me on the show. And it was a pleasure talking to you.
