638: What’s the ROI on those Big Game Ads? Featuring Nataly Kelly, CMO at Zappi

Much is made about the creative decisions in ads for the Big Game, but how does all that money, those requisite celebrity cameos, and everything else that goes into these multi-million dollar investments translate into Return on investment?

Today we’re going to talk about what the numbers tell us from all those high-profile ads and who the winners and losers of the Advertising Bowl are in 2025.

To help me discuss this topic, I’d like to welcome Nataly Kelly, CMO at Zappi, who unveiled their annual Super Bowl Ad Success report on Monday. We’re here to talk about the approach, the results, and what those results mean for brands that invested a lot of money – and time – into their campaigns.

About Nataly Kelly

I help companies unlock global growth

For more than two decades, I have helped scale businesses across borders, as an executive at B2B SaaS and MarTech companies.

I’m Chief Marketing Officer at Zappi, a consumer research platform.

I spent nearly 8 years as a Vice President at HubSpot, a multi-billion-dollar public tech company, driving growth on the international side of the business.

Having served as an executive at various tech companies, I’ve led teams spanning many functions, including Marketing, Sales, Product, and International Ops.

I’m an award-winning marketing leader, a former Fulbright scholar, and an ongoing contributor to Harvard Business Review.

I love working with interesting people and removing barriers to access.

Resources

Zappi website: https://www.zappi.io/web/

Connect with Greg on LinkedIn: https://www.linkedin.com/in/gregkihlstrom

Listen to The Agile Brand without the ads. Learn more here: https://bit.ly/3ymf7hd

Don’t miss a thing: get the latest episodes, sign up for our newsletter and more: https://www.theagilebrand.show

Check out The Agile Brand Guide website with articles, insights, and Martechipedia, the wiki for marketing technology: https://www.agilebrandguide.com

The Agile Brand podcast is brought to you by TEKsystems. Learn more here: https://www.teksystems.com/versionnextnow

The Agile Brand is produced by Missing Link—a Latina-owned strategy-driven, creatively fueled production co-op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. https://www.missinglink.company

Transcript

Note: This was AI-generated and only lightly edited

Greg Kihlstrom:
Welcome to Season 7 of The Agile Brand, where we discuss the trends and topics marketing leaders need to know. Stay curious, stay agile, and join the top enterprise brands and martech platforms as we explore marketing technology, AI, e-commerce, and whatever’s next for the omnichannel customer experience. Together, we’ll discover what it takes to create an agile brand built for today and tomorrow, and built for customers, employees, and continued business growth. I’m your host, Greg Kihlstrom, advising Fortune 1000 brands on Martech, AI, and marketing operations. The Agile Brand Podcast is brought to you by TEKsystems, an industry leader in full stack technology services, talent services, and real world application. For more information, go to teksystems.com. To make sure you always get the latest episodes, please hit subscribe on the app you listen to podcasts on and leave us a rating so others can find us as well. And now onto the show. Much is made about the creative decisions and ads for the big game, but how does all of that money, those requisite celebrity cameos and everything else that goes into these multimillion dollar investments translate into return on investment? Today, we’re going to talk about what the numbers tell us from all those high profile ads and who the winners and losers of the Advertising Bowl are in 2025. To help me discuss this topic, I’d like to welcome Nataly Kelly, CMO at Zappi. who unveiled their annual Super Bowl ad success report on Monday. We’re here to talk about the approach, the results, and what those results mean for brands that invested a lot of money and time into their campaigns. Nataly, welcome to the show.

Nataly Kelly: Thank you, Greg, lovely to be here.

Greg Kihlstrom: Yeah, looking forward to talking about this with you. Definitely as someone born very close to Philadelphia, I was happy with the results of the game itself. But, you know, also being in the advertising and marketing world, you know, definitely want to want to hear your thoughts here and the results of the report. Before we dive in, though, why don’t we start with you giving a little background on yourself and what you’re currently doing with Zappy?

Nataly Kelly: Sure. I’m the Chief Marketing Officer at Zappi. We are a tech company, agile market research platform, enabling connected insights for our customers. We partner with 350 plus global brands. That includes PepsiCo, McDonald’s, Heineken, Reckitt, Diageo, spanning more than 50 global markets. So a lot of our customers were actually building Super Bowl ads in Zappy and developing them with our software. So it’s pretty fun to see them go live and see how consumers react then. And we have about 300 employees in more than 13 countries. We’re surpassing 100 million in ARR in 2025. And we most recently raised 200 million, including a $170 million growth capital round from Cinero Equity Partners in 2022.

Greg Kihlstrom: Great, great. So yeah, let’s let’s dive in. So as I mentioned, and we’ll have a link to the the report in the show notes, but you know, the Super Bowl ad success report went live on Monday right after the game. Can you talk about the approach that you take before, during and after the game to do this analysis? You know, what does it look like? And I mean, the report was available Monday. So obviously, some hard work being done Sunday night, Monday morning, I assume. But, you know, what does it look like and who and what technology are involved?

Nataly Kelly: Yeah, we do have a team of people both in New Orleans right before the Super Bowl on the ground to talk about all the ads at a bunch of different events and such. But then live during the Super Bowl, we were also launching the new ads that were released that night. What’s interesting, Greg, is a lot of these ads are actually released before the big game now and it’s becoming even more common and consumers actually say that they like that. You know, our data shows that they like to watch the ads before and even after they air. But for us, we had to basically configure all of those ads in our software platform to enable us to test with 20,000 consumers across those 65 plus ads that aired. during the game. A lot of them had already pre-aired and been pre-released. Sometimes they do teasers, but you can’t really test the teaser because it’s not the full ad. So we can get a sense of what people might think, but until you really see the whole ad, you can’t really understand the impact on sales and brand and all of those things we love to talk about.

Greg Kihlstrom: Yeah. Yeah. And so, you know, you mentioned that A lot of the ads are being made available or at least teasers available ahead of time. So, you know, can you talk a little bit about, you know, just how the process has evolved over the last few years and including things like that? But, you know, how much is the actual game itself involved and how much includes social media and other channels?

Nataly Kelly: Yeah, well, you know, I like to remind folks that even though we like to analyze the ads and talk about the ads, for the marketing teams involved in these, it’s big campaigns, right? So they’re not just thinking about Super Bowl ads, television ads, they’re thinking about Super Bowl advertising strategy. So to your point, social media plays in. There’s live activations in New Orleans, which I had the fortune of seeing up close. You know, there is a parade where a lot of the brands are featuring, you know, different activations within the parade environment. There’s, as soon as you get to the airport in New Orleans, first thing out of the gate, you see different, you know, Super Bowl ads and advertisers all hitting you, even on the escalators. You know, there’s Pepsi branding all the way down. So a lot goes into it, including the tie-ins with the game. And I don’t know if you saw some of the tie-ins that happened, but the Duracell ad with Tom Brady being rebooted, heading into that commercial break. And if you went on Twitter when the Doritos ad was airing on X, you would see to, you know, the Dorito chips flying with like, you know, kind of some cool guerrilla marketing tactics and things the teams are doing. So it’s kind of exciting to see. And one of the things I learned on the ground in New Orleans, talking to a lot of these advertisers, is that it’s not just the Super Bowl for them. And it’s not even just Super Bowl advertising, but it’s football advertising and sports marketing and, you know, the NFL and specific sponsorships, you know, brand advocates with different players of specific teams with local tie-ins. So a lot goes into all of this that we don’t really think about every day if we’re not involved in it on the marketing side.

Greg Kihlstrom: Yeah, and I mean, that makes a lot of sense because I mean, it’s quite a hefty investment for a one-time thing, right? So, you know, it makes a lot of sense for it to be part of a larger, you know, strategic initiative. And that, as you mentioned, the pre-game, whether they’re teasers or rolling out the ads earlier or things like that is certainly a big component here. We’re gonna talk about the post-game analysis in a little bit here, but Zappy did some analysis before the game and had a list of pre-game winners. Could you maybe talk us through one or two of those winners and what went into those results?

Nataly Kelly: Sure. So whenever we’re testing any sort of advertising, we have a tool that we call Amplify. And for each of these ads, we survey with 300 consumers. And they have to self-report that they’re actually indicative of the buyers of the products being advertised. So if it’s a weight loss product, they need to report that they have an interest in weight loss or have purchased such a product before. And then when we look at how they perform within those categories, we’re thinking about three things. We call it the 3R framework. It’s reach, which is in context, how do the ads stand out and drive brand recall? And then resonance, which is the emotional response. You know, does this ad actually make people feel something? And then response is, how does it actually encourage an action or purchase uplift? And we look at that based on pre and post exposure questions, asking about likelihood to consider the brand or purchase the product. So that’s kind of the methodology behind all of this that we use when we test in Zappy.

Greg Kihlstrom: Great, great. Were there any of the pregame winners that stood out from a brand perspective?

Nataly Kelly: Yes. Oh, so many. I mean, the interesting thing is a lot of these Super Bowl ads, you know, with the teasers and everything, we’re really starting to see some of them come out that are so strong ahead of time. One of the top performing ads that we tested that was really shining from kind of the very beginning was this Budweiser ad called First Delivery. It’s got the pony, one of the Clydesdales. It’s a baby Clydesdale and it’s, deemed too small, too little to participate in the Budweiser delivery, but it gets a loose keg. It is trying to prove his worth. And it’s really a sweet, heartwarming ad. It performed well across men, women, you know, Gen Z, millennial, every generation. And that was one of the really standout winners of the night and also in the pre-release ads that we tested. It had a sales impact score of 100, which is the highest that you can get. Wow. Yeah, so pretty impressive results for that one.

Greg Kihlstrom: So let’s, uh, let’s, let’s get to the, the post game report as well. And again, we’ll, we’ll have a link to the full report in the, in the show notes, but you know, who were the winners this year? You know, if you could maybe talk us through the top two.

Nataly Kelly: Sure. So we basically, in our report, rank them by the categories because keep in mind when you’re thinking about which ones are winners, they’re all competing with each other in kind of the same category. So if you’re selling a beverage, you know, you might be competing whether you’re a carbonated beverage or a beer brand or, you know, what are people drinking, you know, that night and also what are they drinking in general. And same thing on snacks. You might have Doritos, which we mentioned, and also Nerds, which was one of the top performers. One of my great surprises was actually the Nerds ad. They had an ad called Unleash Your Senses, and it was so fun. They had Shaboozie, the singer of a bar song, strolling down the street in New Orleans with the Nerds mascot, the giant gummy, which they introduced last year during the Super Bowl. And that was so much fun because it was really showing off the beauty of New Orleans, and it was just a very positive, positive feeling. And it scored really high on emotion. It got a 67 when the norm for that category is 56. And then on love, which was one of the emotions that we measure through emoji reactions that people give throughout while they’re watching the ad, it got 40% when the norm is 28%. So it really did well on distinctiveness and the other measures, but I think the emotional part was really what struck me from that ad. Yeah, yeah.

Greg Kihlstrom: Were there any other surprises that come to mind?

Nataly Kelly: So my number one surprise was the ad from Hims and Hers, which has been very controversial. And a lot of people have said, oh, that was a horrible ad. And other people thought it was a great ad. And whenever you see that kind of bipolar reaction, then you kind of know, oh, they struck a nerve or they did something special with that ad. Well, when we tested it with people most likely to purchase that product, it got incredibly high scores. I think what was happening in a lot of the commentary that we see about that ad is that the people saying, Oh, I didn’t get it. It seemed like a false promise. You know, they’re, they’re criticizing the same thing that they’re part of and a lot of kind of negative feedback. But what you really care about as a marketer is. you know, how did it resonate with the target segment that you’re trying to reach? And so from that perspective, it scored really well and among the best performers. So I think it all depends on what your strategy is. It’s easy for us to be like armchair quarterback, you know, of the advertising Super Bowl and say, well, I like this. I like that. I don’t actually care what I like. It’s really all about what consumers like are likely to buy that product because that’s who these marketers care about.

Greg Kihlstrom: Yeah, yeah, that makes sense. And to your to your point, like the report itself, I mean, there are many categories, and you know, it’s everything from demographic to other things. So, you know, it’s, it would be tough to say that there is a specific winner, you know, per se, you know, because we’re talking about, you know, different different target audiences and everything like that. But were there any that Seemed like, you know, they were performing really well before the game, but then, you know, didn’t quite deliver, you know, after the game.

Nataly Kelly: Yeah, I mean, there were some that we were really hopeful for and there were some surprises. So the one that I was most hopeful for was actually the Nike ad. I was really looking forward to that because I knew they were featuring some really strong female athletes and I was excited for that. And I think a lot of us have seen over the years in the Super Bowl, in the Super Bowl, Delve does a really good job with this, like, you know, girls in sports and female empowerment. And so I think we all kind of had a feeling like maybe this Nike ad will do the same thing. Well, to our surprise, The NFL ad that was the flag 50 that was encouraging women, girls in flag football, that they’re trying to get more states to enable it, that performed so well with women. It didn’t perform very well with men, but it did perform very well with women. And the Nike ad, interestingly, did not perform as well on the scores. I think one of the reasons is Their goal might’ve been slightly different. You know, a lot of the best advertisers will try to get you to feel, you know, if they do give you a little bit of outrage, then they might want to resolve it at the end. And I think the Dove ad did that really well, where they showed the little girl running and how she loves her legs. And then they said they’ll be unbearable by the time she’s 14. And it makes you feel like, oh, I’m so happy seeing this little girl, but oh, I’m so sad that’s her future. And then it gives you hope at the end and makes you feel like you can be part of the change. Well, the Nike ad kind of did something a little different there. They, it was more like rebellion and like complain, like almost a little negative about complaining about the status quo, but then not giving the viewer as much hope of a positive future. It did a little bit of that toward the end. But it didn’t resolve that emotion completely to where the viewer kind of left with a positive feeling. Now, that may have been their goal. Maybe they wanted people to feel uneasy, you know, about the state of things. But that’s a more difficult thing because it doesn’t perform as well with consumers. So like in our scores, it might not have shown to be as positive, but you just never know what the marketing team behind it and the agency really had in mind with the sentiment that they wanted to kind of evoke from the audience.

Greg Kihlstrom: Yeah, yeah, makes sense. And, you know, maybe along those lines as well, are there any, you know, certainly, you know, you can look at the report and see there’s several that have, you know, very high scores, some high scores across the board and, you know, and variants, you know, across different areas as well. There are those that just kind of missed the mark. Are there, you know, Could you talk a little bit about, you know, are there some, I know you mentioned a couple that were surprises, but are there some that just clearly missed the mark, I guess?

Nataly Kelly: Yeah. I mean, there were a lot that didn’t make it into our report at all because they didn’t, because they did kind of miss the mark and they didn’t score above 75, which was our threshold for sales impact. In other words, if they’re not going to move the needle for sales impact, then we didn’t include them. And, you know, frankly, they might move it a little, but we were looking for the ones that were really outstanding in terms of moving the needle. You know, I would say some of them performed really well with men, but not with women. So, like, the Ray-Ban meta, you know, the Who Eats Art, you know, that had the three Chrises, Chris Pratt, Chris Hunsworth, and Chris Jenner, that really did very well with men, but not so great with women. You know, and again, maybe that was part of their intentional strategy. I also think one that was really surprising, you know, is the Hellman’s one. I think everybody was excited about this ad with Meg Ryan and Billy Crystal reenacting their scene from When Harry Met Sally, and they had Sidney Sweeney making guest appearance. That one, interestingly, you know, it performed above on all the metrics that we measure, that we included in the report. It scored high on sales impact, distinctiveness, unaided brand recall, emotion, and laughter. It was actually very high on laughter. But, you know, some of the ones that are more outstanding would have kind of higher, higher brand recall, higher, you know, levels of distinctiveness and most importantly, the sales impact. So I think that one, you know, I expected it to perform better than it did, but when you consider that we’re trying to kind of appeal to a lot of different demographics and different generations. Right. That’s a very hard thing to do, you know, because Hellman’s obviously, you know, anybody can buy mayo.

Greg Kihlstrom: Right.

Nataly Kelly: So who do you target? You know, who’s your, who you really want to target? Sometimes I think as an advertiser, you’re making strategic choices, like, I’m going to not include this demographic because the people you know, I’m not going to include a celebrity like Sydney because maybe nobody will know who she is in this demographic that’s older, you know, and maybe the younger ones will never have seen that movie. So it is tricky because you reach everyone, you know, all demographics in America that watch the game.

Greg Kihlstrom: Yeah, it’s it’s tough. Yeah, because usually it’s you you want that target. You know, you you would pick a show to advertise on to find that that target. But yeah, to your point, everybody’s watching and yet you can’t be everything to everyone. usually you can’t be everyone to everyone successfully, right? So it’s, you know, it makes sense that there would be some successes with others, you know, with specific demographics. So, you know, just to kind of ask a direct question here, you know, are these ads worth it? You know, they cost a lot of money, they take a lot of planning, a lot of time, you know, does the ROI make them a good investment? And, you know, if the answer is like, it depends, you know, what are those dependent factors?

Nataly Kelly: Yeah, I think the answer is yes. You know, Gary Vaynerchuk was in New Orleans on some different panels where Zappi was present and talked with VaynerMedia about how this is still the best media buy-in town. You know, you really do get the bang for your buck. It’s $8 million this year for each ad, but when you consider how many people you’re reaching, it’s still very worth it, and you do see the ROI. That said, it depends on the execution of the ad itself. You know, I didn’t mean to dodge your question about which ads I didn’t like. Oh, sure, sure. I will say the Squarespace one didn’t really perform so well. And I personally, you know, watching that was kind of confused and wondering like, why are they throwing laptops at people? I’m like, it was just kind of a weird, you know, no, maybe that got people talking and maybe it did appeal to some segment that they’re targeting, but as far as low performers, that one didn’t perform so well. You know, so I do think it’s worth it if you plan correctly and you know which segment you’re targeting, and you really nail it in terms of pre-testing, you know, making sure that you’re involving consumer feedback all along the way to make sure you’re nailing it, because that actually is the secret sauce to making sure that an ad performs well, is getting that consumer in the loop, because A simple thing like a celebrity who’s out of favor with the American public, if you’ve already invested in including them, do you decrease the time on screen? Do you do other things? There’s lots of strategies as a marketer you can use to use that feedback and iterate and make your advertising development process much more agile.

Greg Kihlstrom: Yeah, yeah, love it. So, last question before we wrap up here, you know, as, you know, Zappy is measuring this year over year, what are maybe some of the meta trends that you’re seeing? You know, are these Super Bowl ads changing as, you know, social media provides other opportunities? You know, some, I know we touched on a little bit, but you know, what are these ads gonna look like in five years at the rate they’re changing now?

Nataly Kelly: Yeah, I mean, we are starting to see, and I think this is in part due to better testing and better software that’s helping marketers and advertisers develop better ads. We’re starting to see better tie-ins to the brand and the product. So I think in the past, we saw a lot more ads that were maybe like, oh, just a celebrity, and you wouldn’t remember which company was advertising the product. So people would talk about, oh, that Ben Affleck ad, or oh, that David Beckham ad. But this year, A lot of these advertisers actually nailed it because there was high brand recall on a lot of the ads. I will say, and tying it back to the product and kind of the value prop of the entire, you know, what you’re selling and what you’re marketing, you know, WeatherTech did a great job at this, the ad that had the elderly women, kind of the golden girls going on a road trip and spilling things all over the car.

Greg Kihlstrom: Right.

Nataly Kelly: They made it very clear what the product was and how it helped, and they did it in a fun and entertaining way. I think another company that did a great job at this was the brand Michelob Ultra with the pickleball ad. They did such a great job of like, making that product desirable and making it like something that you’re going to want to do a Smackdown challenge for. But that ad was great. And I think also, you know, certain brands now have become perennial favorites like Dove, you know, they deliver something that we can kind of expect, but then also surprise us a little every year. And that tension between doing something that your audience really expects of you and in terms of alignment to your brand values. But still having an element of surprise to keep people interested is really difficult to pull off. And I think that’s important because All the research shows that younger generations, Gen Z and millennials really care that brands live by their values and that they don’t shy away from them. And so showing up repeatedly, you know, every year as a perennial Superbowl advertiser and like sticking, staying the course, you know, sticking with your commitment, but also having something novel, something new, something distinctive. I think that’s really where the tension is. And I think brands are getting better and better at that. And I think with the power of data, they’ll only continue to get better at that.

Greg Kihlstrom: Yeah. Yeah. Love it. Well, Nataly, thanks so much for joining today. One last question before we wrap up. I like to ask everybody here, what do you do to stay agile in your role and how do you find a way to do it consistently?

Nataly Kelly: Well, I luckily am constantly getting feedback from consumers that are looking, you know, at these ads and these stimuli every day and reacting to them. And so my number one thing to stay agile is to talk to my customers. Same thing, getting feedback and iterating and incorporating their feedback because I believe that’s how the best brands are built and that’s how the best products are built.

Image