In this episode
Nir Weingarten, co-founder and CEO of Eikona, argues that A/B testing has hit a structural ceiling in lifecycle and CRM marketing — not because marketers run it badly, but because the method itself fails on three counts: it doesn’t scale, it can only test one variable at a time, and it is biased toward whatever the team already believed would work. Weingarten, a published AI researcher with a master’s in machine learning from Reichman University, makes the case that reinforcement learning is the natural successor: a system that generates and tests dozens of creative variations continuously, learns from production feedback, and optimizes per customer rather than declaring one global winner. He and Greg Kihlström work through what that changes operationally — how the technology plugs into an existing stack as an “adaptation layer,” how uplift gets measured against a held-out control group, what leaders can and cannot know about why a variant won, and what the shift does to marketing roles.
Key takeaways
- A/B testing fails on three axes: scale, depth, and bias. Weingarten names scale as the largest — every test requires creative production, a CRM manager to run it, and analytics to interpret it, which caps how many tests an organization can actually execute.
- Marketers universally report under-testing, and the reason is capacity, not conviction. Across large and small brands, the answer to “how much are you A/B testing?” is consistently “not enough” — because it’s too hard and too time-consuming, not because teams doubt its value.
- Single-variable discipline is what makes A/B testing shallow. You change only the button color, or only the model, or only the intent — otherwise you can’t attribute the lift — so tests stay narrow by construction.
- Testing bias compounds the problem. Limited resources mean you only test what you already expect to work, and you read the results looking for what you expected to see.
- Reinforcement learning works the way trial-and-error learning works. Weingarten’s framing: a child eats candy and is rewarded, bites a lemon and isn’t, and learns to seek one and avoid the other. A neural net trained on production feedback does the same thing with creative — the same mechanism behind the major language models.
- Retention and lifecycle marketing is the highest-value place to apply it. These channels drive 50–60% of revenue at some companies, sit on rich first-party data, and by Weingarten’s account haven’t been fundamentally disrupted since Eloqua arrived in 1999.
- The product deliberately does not ask brands to change their stack or their creative process. It inserts an “adaptation layer” at the send step: instead of one piece of creative, the marketer sees roughly 20 variations, approves or disqualifies them, optionally comments on why, and sends.
- The 10/80/10 rule splits AI work into brief, execution, and judgment. The marketer supplies the first 10% (brief and initial creative), AI does the 80% of heavy lifting generating variations, and the marketer supplies the final 10% of brand judgment and curation. AI has the muscle and knows how; it doesn’t know what.
- Uplift is measured against a permanent randomized control group of 10–20%. Eikona reports incremental revenue with a confidence interval — a dollar range compared against business as usual — rather than engagement metrics alone.
- Weingarten refuses to explain why a winning variation won. No one knows what happens inside these models, so the system reports the inferred buy intents and dominant messaging type of the winner and how it differed from control, and leaves interpretation to the marketer.
Chapters
- 0:00 — The question: is your test winner wrong for most of your audience?
- 1:31 — Nir Weingarten’s background: data science, RL, and information theory
- 2:12 — The problem Eikona solves, and why lifecycle/CRM teams
- 5:03 — The three limitations of A/B testing: scale, depth, bias
- 7:11 — Reinforcement learning explained: candy, lemons, and neural nets
- 9:28 — Adopting it without replacing the martech stack
- 11:10 — The 10/80/10 rule for AI in the workplace
- 12:58 — How KPIs change: control groups and dollar uplift
- 15:18 — Visibility, approval, and the black-box problem
- 18:25 — What AI does to marketing roles
- 20:22 — Staying agile in a surplus of information
Why A/B testing breaks down at enterprise scale
Weingarten identifies three failure modes, and ranks scale first. Every test consumes a production or creative resource to build the variants, a CRM manager or marketer to run it, and a data science or statistics resource to infer anything from the result. That chain is why the answer he hears from marketers — at the biggest brands and the smallest — is that they aren’t testing nearly as much as they’d like to. The constraint is manpower and time, not belief in the method.
Why single-variable testing keeps tests shallow
The methodological rule that makes A/B testing valid also makes it narrow. You change one element per test — the button, the model, the intent — because changing two leaves you unable to attribute the uplift. Weingarten frames this as the depth problem, distinct from but related to scale: it’s hard to run many tests, and hard to make any individual test meaningful.
Reinforcement learning, explained without the math
Weingarten’s explanation is deliberately non-technical. A baby explores the world, finds candy, is rewarded by the taste, and learns to seek more; finds a lemon, dislikes it, and learns to avoid it. Reinforcement learning formalizes that: a neural net tries different things, observes what works in production, and uses that feedback to generate more of what worked. He points out this is the same approach underlying the major language models — systems trained on what kinds of answers people actually engage with.
The adaptation layer: change the output, not the stack
Changing a marketing stack is a heavy lift, and most brands aren’t R&D-heavy organizations. Eikona’s design response is to plug into the existing workflow rather than replace it, and to adapt to each brand’s cadence, ownership structure, and brand-guideline strictness. Weingarten calls the new component the adaptation layer. Operationally, nothing changes until the moment the marketer hits send — at which point they see roughly twenty variations instead of one, keep the ones that fit, kill the ones that don’t, optionally leave a comment explaining why something is off-brand, and send.
What the 10/80/10 rule means for marketing teams
Weingarten builds on the Pareto framing: 80% of the work gets done in 20% of the time. With AI, he says, that 20% splits into two tens on either side of the machine’s contribution. The first 10% is human — the brief, the first email or SMS, the direction. The 80% is the machine generating variations. The final 10% is human again — judgment and curation, the marketer saying “that’s off-brand” or “I didn’t like that.” His summary of the constraint: AI has a lot of muscle and knows how to do things, but it doesn’t know what to do, and it can’t supply the curation.
How to measure uplift when the system never stops learning
Weingarten’s answer is that performance marketing is the easy case — measurement is relatively straightforward. Whether the method is A/B testing or reinforcement learning, you hold back a randomized control group; in Eikona’s case, 10–20% depending on engagement metrics. Performance is measured against that control and expressed as a confidence interval on incremental revenue — his illustration is a monthly figure between $70,000 and $130,000 of incremental uplift versus business as usual. That dollar number is the first thing on the dashboard. His broader point: paid media has always been judged on CPM, CPA, and ROAS, while owned media defaulted to blasting the same email to a million people because no technology existed to do better.
What leaders can and can’t know about why a variant won
Asked how leaders keep visibility and control over an always-learning system, Weingarten calls it a fundamental question in AI and answers it honestly: from his experience in academia and industry, nobody really knows what happens inside these models. So the product doesn’t pretend to. Control comes from two places instead. First, the process is semi-automatic — a person on the client team approves content before anything sends. Second, when a variant beats control by 20% or more, the marketer gets a push notification, and the system reports what it can defend: one or two inferred buy intents, the dominant messaging register (urgency, novelty, a scientific explanation), and how the winner differed from control. It does not assert causation. A winning creative can also be pushed to Meta or other paid and social channels, where it has a reasonable — not guaranteed — chance of performing.
What happens to marketing roles
Weingarten doesn’t expect mass layoffs, and his reasoning is technical rather than reassuring: he doesn’t believe the models are intelligent enough at their core to make the decisions, and marketers know their brand, their audience, and the strategy in ways the systems don’t. The shift he expects is that AI takes the grunt work and marketers move toward orchestration and strategic decisions — which, in his read, increases their leverage inside the organization rather than reducing it. The obligation is to adapt and learn the tools. His historical analogy: marketing automation created a new job category of people operating CRM systems rather than eliminating one.
FAQ
What are the main limitations of A/B testing? Nir Weingarten identifies three: scale, because each test needs creative, CRM, and analytics resources and therefore doesn’t scale; depth, because valid tests change only one variable at a time; and bias, because limited resources mean teams test only what they already expect to work and read results accordingly.
How is reinforcement learning different from A/B testing in marketing? A/B test looks for one winning variation and stops. Reinforcement learning runs continuously — a model generates many variations, learns from live production feedback which ones perform, and keeps producing more of what works, optimizing per interaction rather than declaring a single global winner.
What is the 10/80/10 rule for AI? It’s Weingarten’s split of AI-assisted work: the marketer supplies the first 10% (brief and initial creative), AI does 80% of the heavy lifting generating variations, and the marketer supplies the last 10% of brand judgment and curation. AI knows how to execute but not what to make or which output to keep.
How do you measure the uplift from an AI optimization system? Hold out a randomized control group — Eikona keeps 10–20% depending on engagement metrics — and measure performance against it, expressed as incremental revenue with a confidence interval rather than engagement metrics alone.
Can you tell why an AI-generated variation outperformed the control? Not causally. Weingarten’s position is that nobody knows what happens inside these models, so the system reports the inferred buy intents, the dominant messaging type, and how the winner differed from control — and leaves the interpretation to the marketer.
Will AI replace lifecycle and CRM marketing roles? Weingarten doesn’t think so. His view is that the models aren’t intelligent enough to make brand, audience, and strategy decisions, so marketers shift toward orchestration and strategic judgment while AI absorbs the heavy lifting — the pattern he saw when marketing automation created CRM operations roles rather than removing them.
About Nir Weingarten
Nir Weingarten is the Co-Founder and CEO of Eikona, a start-up which is using generative AI and reinforcement learning to transform lifecycle marketing. A published AI researcher with a master’s degree in machine learning from Reichman University, Nir spent over a decade leading multi-disciplinary technical and product teams in the fields of AI, data and performance.
Nir Weingarten on LinkedIn: https://www.linkedin.com/in/nir-zvi-weingarten/
———- Resources ———-
Eikona: https://www.eikona.io
We’re proud to be a media partner for #MAICON26 – Oct. 13-15! Learn how AI can power your marketing and business and help you grow smarter. Use code AGILE150 to save! https://aglbrnd.co/r/7fe458ced0f04658
Reach your customers with Reddit. Spend $500 in ad spend, get $500 back in ad credit! Learn more: https://advertalize.com/r/491818c79fb1873f
Chaser is the only Slack-native project management platform that helps teams turn messages into tracked tasks, automate follow-ups, and maintain team-wide visibility, without adopting another tool. Now integrated with Claude and other GenAI tools. Learn more at trychaser.com and use code AGILEBRAND for a 3-month free trial (normal trial is 14 days).
The most influential minds in software, AI, and engineering leadership will be at WeAreDevelopers World Congress North America, September 23-25 in San Jose. Learn more: https://aglbrnd.co/r/60a7299222a7bcf1
Start building your own apps with Replit and get $20 off. Learn more: https://aglbrnd.co/r/93531742a7625a20
Enjoyed the show? Tell us more at and give us a rating so others can find the show at: https://aglbrnd.co/r/faaed112fc9887f3
Connect with Greg on LinkedIn: https://www.linkedin.com/in/gregkihlstrom
Don’t miss a thing: get the latest episodes, sign up for our newsletter and more: https://aglbrnd.co/r/35ded3ccfb6716ba
Check out The Agile Brand Guide website with articles, insights, and Martechipedia, the wiki for marketing technology: https://www.agilebrandguide.com
The Agile Brand is produced by Missing Link—a Latina-owned strategy-driven, creatively fueled production co-op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. https://www.missinglink.company
Transcript
[00:00:00] Greg Kihlström: Hi, I’m Greg Kihlström, host of The Agile Brand, and here’s a question for you. What if the winning variation from your A/B test is actually the wrong choice for the majority of your audience? Agility requires shifting from a mindset of finding a single winner to one of continuous learning and adaptation for every customer interaction. Today we’re going to talk about moving beyond the limitations of traditional A/B testing with reinforcement learning, how AI can be used to dynamically optimize and personalize customer experiences in real time, not just find a single best fit, and the practical impact of AI on marketing roles, and how to structure teams for a future human machine collaboration. To help me discuss this topic, I’d like to welcome Nir Weingarten, CEO and founder of Eikona. Nir, welcome to the show.
[00:01:18] Nir Weingarten: Hi, Greg. It’s a pleasure being here.
[00:01:20] Greg Kihlström: Yeah, looking forward to talking about this. Definitely, definitely top of mind for, for many, including myself. Before we dive in, though, why don’t you give a little background on yourself and your role at Eikona?
[00:01:31] Nir Weingarten: Sure. So I’m Nir. I’m, 36 years old. I live in Tel Aviv. I’m the co-founder and CEO of, Eikona, and before that I was a data scientist and a algorithms developer, and before that I did my, master’s, in AI and machine learning, and with a thesis and some, research in the fields of, reinforcement learning and, information theory.
[00:01:58] Greg Kihlström: Nice. Nice. Well, yeah. And, and why don’t we g- start a little bit with Eikona as well. Could you maybe tell our audience a bit about the company, what’s the core problem you’re solving, and, and who you’re solving it for?
[00:02:12] Nir Weingarten: Sure. So one of the biggest things we’ve, heard from marketers is that it’s really hard to know what content works. And, especially in performance marketing, that’s very, very, very important, and we result to guesswork, and it’s really hard to know in advance. You can be the best creative director, the best copywriter in the world, you’re not really gonna know what performance marketing content’s gonna resonate with your audience. And the best in class to date to solve this problem is A/B testing. Now, when you ask marketers, it could be the best marketers, the biggest brands, could be also smaller brands, you ask them, “How much are you A/B testing?” And we’ve heard across the board
[00:02:57] Nir Weingarten: that the answer’s almost always not enough-
[00:03:01] Nir Weingarten: … and, not enough as we would want to. And, and then you ask people why, and, and you constantly hear that, “It’s just too hard, and we don’t get the time to do it. We don’t have the manpower. We know it’s important. We just don’t do it.” So coming from a machine learning background, both Omer, my co-founder, and myself, and we think there’s a big why now moment to change A/B testing, to evolve it into a method that’s called reinforcement learning, and, and we’ll dive deeper into that in a second. And, but, that’s… We really see it as the natural evolution of A/B testing, solving all of these pains that marketers experience today. And our target
[00:03:46] Nir Weingarten: audience is specifically retention and life cycle teams, CRM teams, and, the reason is, is that we think the technology is the best fit there, and because you have… it’s, it’s a s- it’s such a big channel, such a big market. You know, we have companies having, 50, 60% of their revenue coming in from these channels-
[00:04:08] Nir Weingarten: … and it’s such an underserved channel, right? And basically hasn’t been disrupted since 1999 when Eloqua came in, right? [chuckles]
[00:04:19] Greg Kihlström: Right. Right.
[00:04:20] Nir Weingarten: S- so there’s a lot to be done there, and th- there’s also a lot of first party data, very, very rich in data, so for me as a data scientist, you know, that’s exactly where I wanna be.
[00:04:29] Greg Kihlström: Yeah. Yeah. So yeah, let’s, let’s dive in a little more here. And, you know, I, I know you touched on some of the limitations, just then, but let’s talk a little bit more about, you know, as, as you mentioned, you know, most marketing leaders, they really look at A/B testing as the, you know, a cornerstone of, of optimizing, you know, campaigns, you know, other things that they do. But, you know, large enterprises with diverse audiences, what are, what are some, some of those limitations that are really holding them back with, with this approach?
[00:05:03] Nir Weingarten: Well, I think there’s three major problems with A/B testing, and first and foremost is scale. It’s a manual process that doesn’t scale. And for each test that you run, you need people in, production or creative to produce the content, then you need the CRM manager or the marketer to test it, and then you need some data science or statistics to infer stuff from the results. And that’s something that just doesn’t scale. And the second part is multi-variant testing. So, you know, you, you’d wanna ask yourself, “What am I going to change? Maybe I’m gonna change the color of the button. Maybe, maybe I’m gonna change the model.”
[00:05:49] Nir Weingarten: “Maybe I’m gonna change the intent.” You always change one thing, because otherwise you won’t know what, caused the uplift. And so that’s, that sort of relates to the scale part, but it’s more on the depth side of it. So it’s really hard to get it wide in terms of the amount of, tests you make, and it’s hard to make them deep and meaningful, ’cause you only test one thing at a time.
[00:06:17] Nir Weingarten: And I would say that the third thing is bias. So, you know, you- you’re limited in your, resources, so you have to test certain things. You can’t test everything. You’re gonna test stuff that you think that are going to work, and when you’re, when you’re gonna look at the results, you’re gonna see the stuff that you would wanna see. So you have a certain bias on the process, and it’s really hard to take, tangible insights for that and actually do something on the next, campaign. So I would say these are the three biggest, and, and scale basically being the biggest of them.
[00:06:52] Greg Kihlström: Yeah. So then the, you know, this brings us to reinforcement learning, which you, I know you briefly mentioned in, and when you were talking about the company. Can you explain it in, in maybe practical terms how this approach differs from traditional testing and, and what does it unlock for a marketing org?
[00:07:11] Nir Weingarten: Sure. So I think reinforcement learning is very intuitive, ’cause it’s sort of similar to how we as people learn how stuff works in the world. So imagine a baby, and the baby is born, and they walk around in the world exploring their world, and they see a piece of candy.
[00:07:31] Greg Kihlström: [laughs]
[00:07:31] Nir Weingarten: And they reach out, and they eat that piece of candy, and it’s sweet, and it’s tasty, and now they’ve learned, ’cause they were rewar- rewarded, right? They’ve learned that candy is tasty, and they’re gonna seek out more candy-
[00:07:45] Nir Weingarten: … [laughs] for the good and bad. [laughs]
[00:07:46] Greg Kihlström: Right. Right. [laughs]
[00:07:49] Nir Weingarten: but, but on, on the other day, the baby walked around, and then, they saw a, a lemon. I don’t know, someone left a lemon, on the floor, and they, they took a bite, and it was sour. It was too much. So they really didn’t like that. So the baby learned by trial and error that candy is nice and lemons aren’t, and they’re gonna, seek more candy and avoid the lemons. Basically, a reinforcement learning is a type of machine learning that is based around these concepts. So y- today, you, you train what’s called a neural net, which is this, just this type of software that tries different things out, and it can learn by stuff that works in production and can use that feedback to actually create more stuff like that.
[00:08:34] Nir Weingarten: And when you look at all the big, models today, the big language models, ChatGPT, Anthropic’s models, you know, they’re all based on this approach of having the models learn what people, what type of, of, of answers people like and, and creating answers that people, engage with.
[00:08:55] Greg Kihlström: Yeah. Yeah. Got it. So then, you know, for all those orgs out there that are very, you know, A/B testing is just the way that things are done, you know, adopting new approaches means changing how teams work in, in many cases. You know, how have you seen successful organizations adapt their processes, you know, anywhere, anything from the initial briefs to the analysis of campaigns? You know, how have you seen them a- adapt their process to, to take advantage of this c- more continuous optimization model?
[00:09:28] Nir Weingarten: Sure. So I think for, for, you know, for brands in, in general, changing marketing stacks is, is a big pain. a lot of times brands are not, organizations that are heavy on R&D, right? So the first thing that we’ve built the product around is, plug it into that and not trying to change, not, not the tech stack and not necessarily the way the brand works and their creative cycle because each brand has a different way that they work, you know. Different people in the organization are responsible to different parts.
[00:10:03] Nir Weingarten: And there’s a different cadence, and sometimes, brand guidelines are very strict. Sometimes they’re looser. So what we try to do is adapt the solution, to each brand and to the way that they work, but at the end, what we’re trying to build is, like, this new stack in the, this new layer in the stack, which we call the adaptation layer.
[00:10:25] Nir Weingarten: So we’d want the marketer to go about their day as usual, and once they get to the part when they hit send or schedule for the campaign, they’re gonna see, instead of the one piece of creative that they’ve created on the usual track that they do however they do it, right?
[00:10:43] Nir Weingarten: Now suddenly they’re gonna see, like, 20 different ones all exploring different candies and lemons, and then they have the, the, the moment to choose the ones that they like, disqualifies the one, the ones they don’t like. If, if they have the patience, they can leave a comment, say, “Hey, I didn’t like that because that’s off brand, because we don’t say that,” et cetera. And then click send again. That’s it.
[00:11:10] Greg Kihlström: You’ve talked about a 10/80/10 rule for AI’s role in the, in the workplace. Can you un- unpack that a little bit and, you know, how does… How, how to apply something like that?
[00:11:23] Nir Weingarten: Sure. So, we’re all familiar with the 80/20 rule, right?
[00:11:26] Nir Weingarten: The Pareto rule.
[00:11:28] Nir Weingarten: right, so it says, like, that 80% of work is done by 20% of the time. You have this very big assignment coming up. You delay it to the last moment. Then you see you can, you can, you can squeeze it ’cause most of the work is done, like, at the… You know, can squeeze it to one hour. so there’s this saying online today that AI is sort of, like, using AI is sort of like this, only you take the 20% and you split it into two tens.
[00:11:55] Nir Weingarten: And now it, like, becomes a sandwich. so what does that mean? It means that AI is very capable. It has a lot of muscle, but it doesn’t know what to do. It knows the, the how to do it. But it won’t know the what, and it won’t know the, feedback like so- like the, the curation. So let’s give an example, right? Say you wanna do a marketing campaign. AI won’t know your brand, it won’t know, you know, what you wanna promote, it won’t know a lot of stuff. So you’re gonna g- give it the brief or do the initial piece of creative like we work, right? So you created the first email or the first SMS or whatnot. Now there’s the, the 80%. So the first ei- eight, 10% is creating that, then the 80% is all the heavy lifting
[00:12:40] Nir Weingarten: of creating the variations, for example, in this case.
[00:12:44] Nir Weingarten: And then you need the last 8% of the marketer to come along and say, “Hey, you know, that’s off brand” or, “I didn’t really like that,” applying their judgment, on the system. So yeah, that’s, that’s how the, uh-
[00:12:56] Nir Weingarten: … 10/80/10 works.
[00:12:58] Greg Kihlström: [chuckles] No, love it. Love it. So let’s talk a little bit about measurement of this, and I know we, I know we’ve been touching on that, all, all the while, but, you know, w- when you move from a kinda test and learn A/B testing model to a more continuous explore and exploit model, you know, how do the KPIs change, you know? Or do, do they change? You know, what, what should leaders be looking to measure, the true uplift and business impact of this, this approach?
[00:13:28] Nir Weingarten: Right. So one of the, best things about performance marketing is that, it’s, it’s pretty easy or straightforward to measure performance.
[00:13:38] Nir Weingarten: So in the world of testing, doesn’t matter if it’s A/B testing or reinforcement learning, you constantly keep a small part of, randomized control group.
[00:13:47] Nir Weingarten: Right? so in our, in our case, it’s between 10% and 20% depending on, on engagement metrics, but we keep this control group and we can measure, performance on top of that, and we can compute that using a confidence interval. So we’re gonna say, “Hey, this month, this is how much money we’ve made for your brand, and this confidence interval, and it’s gonna between, $70,000 and, and $130,000, of incremental uplift,” because we can compare that to business as usual, and that’s actually the first thing you see, when you log on to our app is this, the dashboard saying, “Hey, this is how much money we made,” and also, you know, uplifted engagement. But really the dollar is, is what counts, and we actually, we, we, we
[00:14:32] Nir Weingarten: know that. We know that from paid media, right?
[00:14:37] Nir Weingarten: So, so you’ll ask any marketer, you know, “What, what’s the most important numbers or performance numbers?” They, CPM, CPA, or, you know, ROS, because we really know that, like, paid media is all about that. But when we’re talking about, owned media, it’s like, I don’t know, I just blast the entire list and, you know, I send the same email to-
[00:14:57] Nir Weingarten: … a million people. I don’t know any better now because, you know, there’s no, there’s no technology mitigating that, so we’re, we’re trying to take that approach into, into owned media. And I think there’s a lot of very, very good AI tools today out there that created tons of value, but specifically in our domain, it’s pretty easy or straightforward to, to measure uplift, and that’s one of the best thing in it.
[00:15:18] Greg Kihlström: Yeah, yeah. And so also how do leaders get visibility on, you know, with, with a system that’s always learning, it, it, how, how do leaders maintain visibility on and, and control on, on what’s working and, and so they can kinda see what’s going on versus kind of the black box scenario?
[00:15:41] Nir Weingarten: I think that’s an amazing question. It’s, it’s, it’s, it’s a fundamental question in AI. To be completely honest, you know, at least from my experience, both in academia and, and otherwise, no one really knows what happens inside of these models.
[00:15:55] Nir Weingarten: It’s really, really hard to say, “That’s why the AI did that.” And so we’re not trying to do that in- inside our product. The approach is different. First, it’s a, we, it’s a semi-automatic process. The marketer has to approve the content. We don’t send anything, you know, without a, a person in your, in, in, in our partner’s team approving it. Second of all, when something does succeed, like if something like scored 20% ab- and above over the control, you’re gonna get a push notification for us saying, “Hey, we found a winner.”
[00:16:31] Greg Kihlström: Mm-hmm. Yeah.
[00:16:32] Nir Weingarten: And you’re gonna click, you can click it and then you see the winner, but we’re not gonna say why, ’cause we don’t know. You know, I, I, I can’t be presumptuous and say, “Hey, this won because of that.” No, what, what we’re gonna do is we’re gonna say, “Hey, these are the buy intents that we’ve, inferred from this, creative. One, two intents, two bullets max. This is the type of messaging that was most dominant in this creative. It’s urgency, it’s, novelty, it’s, I don’t know, maybe a scientific type of explanation, you know, different types of messages, one or two bullets, and this is how it’s dif- differs from the control.” And now, you know, you’re the marketer, you take the judgment, you decide what you wanna do with it. And there’s a, there’s a, there’s a small, a perk there
[00:17:17] Nir Weingarten: that you have a button that says, “Send to Meta.” So if you find a winner there, good chance, not sure, you know, ’cause it’s, it’s a different channel and there’s a lot of differences, but it has a, a good potential to score well also on paid media or on social or on-
[00:17:33] Greg Kihlström: Right
[00:17:33] Nir Weingarten: … other cha- channels. So-
[00:17:34] Greg Kihlström: Yeah
[00:17:35] Nir Weingarten: … constantly, constantly testing all these creatives gives you some power that you didn’t have before, that you can just ship them to other channels and, and, they can work well there as well.
[00:17:44] Greg Kihlström: Yeah, yeah. I mean, and it sounds like the, with the, with the 10/80/10 approach, you know, there’s, there’s plenty of room for humans to be involved in, in the process and guide and, and approve and everything like that. But, you know, still there’s, there’s a lot of talk about how AI is changing marketers’ roles. I, you know, I s- I think sometimes, you know, it’s for the worse, sometimes it’s for the better, the, the conversations around this. What, what’s your take on this? You know, how are roles like conversion rate specialists, brand managers, performance marketers, you know, how do you see that evolving over the next few years as tools like this become more, more prevalent?
[00:18:25] Nir Weingarten: Right. So first and foremost, I, I don’t think there’s gonna be massive layoffs or stuff like that. I think, you know, I, I can look at it from a, a product and, and, and technical perspective.
[00:18:40] Greg Kihlström: Yeah.
[00:18:40] Nir Weingarten: I don’t think that in their core the models are really intelligent enough to make decisions. We need the marketers to do that. And no one knows their brand better than the marketers. No one knows their audiences. No one understands the strategy. So we want AI to take the grunt work and the heavy lifting so that the marketers, you know, can, can become more of, orchestrators and, take more of the strategic decisions, focus more on, on, on managing this operation than the heavy lifting. And it only gives them more power inside of the organization, right? Not the other way around. You do, however, need to adapt and, and, and, and, you know, and, and, and, and learn the tools and,
[00:19:25] Nir Weingarten: and, and, and see what, what works for you and what you like working with. But, you know, it’s, it’s, it’s throughout human history there were big changes in technology-
[00:19:35] Nir Weingarten: … and, you know, people still have jobs and, and, you know, so you, you, like you had the Industrial Revolution, and before that we were all peasants and now we have tractors.
[00:19:47] Nir Weingarten: So, y- you’re, you’re, you’re, you’re farming the land with a tractor, so you, you, you, you, you have it easier. And so really I don’t think there’s, there’s, I’m not in the camp that thinks there’s a reason to, to worry. There’s another recent e- example, right? When marketing automation came out and, and s- people started building the automations, it just created more jobs. Now you had, you needed to have people that, operate the CRM systems-
[00:20:16] Nir Weingarten: … that weren’t there. yeah, so, so definitely that’s my take on it.
[00:20:22] Greg Kihlström: Yeah. Yeah. Ag- agreed. I think there’s, the, I, I think there’s gonna be a shift, but I do see, you know, I’ve been through a few, [chuckles] a few of these, these waves in my, in my career, and it, there do end up being more opportunities than, than less at the end of the day, so yeah. well, Nir, thanks so much for, for joining and, and for sharing your insights. have, one last question before we wrap up here. what do you do to stay agile in your role, and how do you find a way to do it consistently?
[00:20:55] Nir Weingarten: Love this question. you know, this question made me think, when I saw it on your podcast ’cause, you know, hey, what do I do to stay agile? [laughs]
[00:21:05] Nir Weingarten: So, yeah, you know, I try to keep some time to, to see the bigger picture. the thing is the bigger picture can look scary ’cause it’s big and you don’t know exactly what’s important and what’s not. So there is a complete surplus of information. And the thing is that works for me is that I just say, “Okay, I’m not gonna be able to ingest all of this information. I’m gonna take whatever I can and try to enjoy that, and you know, I’m gonna miss some stuff for sure. I’m gonna pick it up later, and when I talk to someone and when I read it, about it like two weeks after everyone else.” But, you know, that state of mind helps me, like, actually, you know, open up my X feed and, and, and start reading and seeing what happened. And I also subscribe to this newsletter called TLDR. do you know it? You know, it’s easy. It gives you, like, the big chunks each, each morning and I try to stay opinionated, you know, ’cause if you’re opinionated on stuff, then, you know, it, it makes it easier to, like, navigate the, the, the, the surplus of information.
