If you aren’t focused on being visible in AI-based searches, you’re missing the fastest-growing audience and missing out on some big opportunities to win new customers. But how do you get a comprehensive view of what these AI-based searches are turning up and where you fit in them?
Watch the Episode
Key takeaways from this episode:
- Most website traffic today comes from bots and AI agents, not human browsers — but sites built for humans are hard for retrieval bots to read.
- There are three bot types marketers should know: indexer bots (Google), training bots (feeding LLMs), and retrieval bots (fetching answers in real time for tools like ChatGPT).
- Scrunch creates an AI-readable “scrunched” version of each page — a large page of ~230,000 tokens is condensed so a retrieval bot spends far fewer tokens and reliably pulls your value propositions.
- This raises brand mentions and citations in AI answers, which drives higher-intent “AI referral” traffic back to your site.
- You don’t have to rebuild your site; Scrunch adds an informational layer alongside your existing SEO.
FAQs
- What is AI visibility? — How often and how accurately AI tools like ChatGPT, Perplexity, and Claude mention and cite your brand when users ask related questions.
- What is generative engine optimization (GEO)? — Optimizing your content and site structure so AI answer engines can read, understand, and cite you, versus traditional SEO which optimizes for ranked link results.
- Why is bot traffic to my website increasing? — Retrieval bots visit pages in real time to answer user questions; as AI search grows, these agents make up a rising share of visits.
- How do I show up in ChatGPT and Perplexity answers? — Make your content easily readable to retrieval bots with clear titles, structured headings, and reduced page “noise” so the LLM can extract and cite your information.
- What does Scrunch do? — It creates an alternate, AI-optimized version of each page so bots get clean structured content, increasing mentions, citations, and AI referral traffic.
About Devin Stevens
Devin Stevens is Head of Solutions Engineering at Scrunch, an AI visibility platform helping enterprise brands understand and change the way AI tools like ChatGPT, Perplexity, and Claude talk about them. Working daily with Fortune 100 companies as well as small businesses, Devin sits at the intersection of enterprise strategy and emerging AI technology — helping some of the world’s most recognized brands navigate the shift from traditional search to AI-driven discovery. He brings a practitioner’s perspective to one of the most consequential shifts happening in marketing and brand management today.
Devin Stevens on LinkedIn: https://www.linkedin.com/in/devincstevens/
Resources
Scrunch: https://www.scrunch.com
Transcript
Greg Kihlstrom (00:00)
If you aren’t focused on being visible in AI-based searches, you’re missing the fastest growing audience and missing out on some big opportunities to win new customers. But how do you get a comprehensive view of what these AI-based searches are turning up and where you fit in in them?
Welcome to One Amazing Thing About Scrunch. Today we’re talking with Devin Stevens, head of solutions engineering at Scrunch, and he’s going to be sharing their amazing platform with us today. Devin, welcome to the show.
Devin Stevens (00:35)
Yeah, thanks for having me, Greg. It’s great to be here.
Greg Kihlstrom (00:37)
Yeah, looking forward to seeing this in in action in a second here. But before we do, why don’t you give a little background on yourself and your role at Scrunch?
Devin Stevens (00:45)
Yeah, obviously I’ve been in the the SaaS world for many years and helping brands with social media, websites. and now, you know, the buzz is AI. So my team from you know, first interest from anybody on what the tool can do, how we use it, you know, all the way through implementation and deployment. So my team is is kind of a guide through that journey on making sure things get set up right.
Greg Kihlstrom (01:07)
Love it. Love it. So let’s talk a little bit before you share. you know, what are we gonna look at today? who’s the audience and and why is it important to them?
Devin Stevens (01:16)
Yeah, well, what I want to talk about today is our what we call our agent experience platform. And you you’ve probably seen it about a week ago. There was a Forbes article, and you know, we’ve had this dead internet theory out here for a long time, but you know, now we’re seeing the data that the main visitors of websites are are now bots and agents. and so what unfortunately we’ve spent you know years and even decades building in this really rich, you know, high experience browser, you know, based internet, it doesn’t work for them anymore. And so
you know, we we are set out and and we’re helping brands make, you know, a different version of the website and and make it be more friendly for for those AI agents.
Greg Kihlstrom (01:54)
Love it. Well yeah, let’s let’s take a look. Why don’t you share your screen?
Devin Stevens (01:57)
So, Greg, kind of talking you through this a little bit, you know, one of the first things is really starting to understand, you know, who’s visiting the website. So, you know, again, a lot of that data and really seeing, you know, the different types of bots. So, you know, as we start, you what you’re looking at here is actually our website. And so, you know, really starting to understand when a bot comes, what’s it looking for? What can it do? And there’s a lot of nuance as we start thinking about these different types of bots in terms of, you know, those classic indexer bots that we’ve been seeing for years as we think about, you know, Google.
And being in brave building those indexes. Also a lot of training bots out there as we think about people who are really reading your content, starting to, you know, bring it into those models so that they can answer questions, you know, quickly. And then where we see the biggest impact and we focus a lot with the the big companies today is these retrieval bots. So the idea that retrieval bot is you know someone’s sitting at a keyboard, they’re on their phone, they’re asking a question, and that bot is coming to your website to to try to see if you’re a part of the answer.
So we really start to break that down and make sense of, you know, who is it? What are they coming for? and how do they like to read information? So again, really starting to understand how the internet has shifted, and and not everybody’s a, you know, human on a Chrome or a, you know, web browser anymore.
Greg Kihlstrom (03:12)
think that’s the classic misunderstanding is that, you know, again, we’ve been doing SEO for I’ve been doing SEO for two longer than I care to mention, but you know, the there’s just a like fundamental difference, right, in in how how bots and you know, there’s been search and index bots for for
you know, decades at this point to your to your point. But you know, being able to look at this and see it, you know, what how does this help people make you know, substantive changes to their sites and and their content to be able to be indexed better and and be retrieved better?
Devin Stevens (03:47)
Yeah, that’s a great question. And so, you know, there’s a lot of different ways to look at it. But, you know, even coming down to the basics here, let me show you a little bit. So, you know, the the first part is really understanding is my website, you know, being visited? so then as we think about those visits, really understanding, hey, when that bot comes, you know, it doesn’t doesn’t have that browser, it’s not getting that full experience, it’s not gonna see that video load you know, in a player. So, you know, putting these side by side and starting to say, is my website even available?
Greg Kihlstrom (04:10)
Right.
Devin Stevens (04:16)
You know, is the information there? Can it be bred? you know, and then again, really just starting to understand even the access, firewall controls, robots.txt files, and how they may be interfering or helping as we think about those. So, you know, bringing that back even one level and then saying, hey, after a bot comes and visits my website, you know, what is it doing with the information? Is that helping me? Did it find what it needs? And am I showing up in different citations and different answers? So really starting to understand, hey, that.
That ChatGPT retrieval bot came, visited my my page, has my content, and then starting to roll that into, hey, now it’s using me in an answer because it can understand that information. Not only is it using me in an answer, but hopefully it’s giving me a link and a citation so that when I have a you know prospect, a consumer, someone who’s interested in learning more, they’re they’re one click away. Then they’re coming onto my website and in what we call that AI referral. So again, the whole marketing funnel has changed.
It’s very, very different as we think about qualifying visitors. And there’s not this click, click, click, click across my website that we all loved and spent years you know understanding. Now that whole conversation is is out of my eyesight. You know, the LLM and the and the user are interacting. And by the time I get that user, the the good news is when they do come to my website, the intent is way higher. We know they’ve done a lot of that explore. We can move them far more quickly.
you know, into those calls to action or the next stage that we’re looking for. Now, the problem is as you start building these bot-friendly websites, what happens is the you have to start to cater to to one visitor over the other. You know, as you start to make a bot-friendly website and you start to strip down that information, it it’s not as fun for you and I to browse the internet. The the videos are gone, you know, it’s not as rich. We don’t see as much JavaScript.
So, really starting to get a detailed understanding of that. And then ultimately what we do at Scrunch is we take all that information and we actually scrunch the page. So we we actually create an alternate version of every page on the website. So what you’re seeing on the screen here on the left side is what we call the human view. This is this is my origin, this is my you know website that I’ve been building that’s SEO optimized, and I’ve been doing this for years. And and a lot of brands have much bigger pages than this.
But this one has 230,000 tokens. So when those retrieval bots come and visit that page, they’re often, you know, spending about 10,000 tokens to understand what’s on it. So as we see how they parse this page as they go through, the likelihood of them to pull the information, the story, the value statements that I want off of this page, it’s very small. It’s going to fight through a lot of this noise, navigation, headers, and things that it doesn’t need, and it won’t read anyway.
Now on the right side we have that scrunched version of the page, if you ever were wondering where the name came from.
Greg Kihlstrom (07:11)
Yeah,
I did I did. So yeah, that’s great.
Devin Stevens (07:14)
So yeah, we’ve scrunched that down. Now what’s really cool is on this left and this right side here, we actually have the same amount of usable content. So the idea being is instead of fighting through the noise, instead of trying to figure out things that are meant for a browser and a human, the bot comes to this page and they get very, very clear, very structured information without any of that extra effort. We’ve got clear title and descriptions here. We’ve got H1 headers, H2, and we’re writing at a reading level.
These bots prefer. We’re watching different sentence structures and word count. And so what’s really exciting is you don’t have to go and rip apart everything you’ve done for years as a marketer. You don’t have to go to your web development team and say, hey, we’ve got to start over. What we’re doing at Scrunch is we’re making this happen very, very quickly. We’re not interfering with anything that’s already working on those websites and in those online experiences, but we’re able to create that informational layer immediately.
and really start to go lift, you know, mentions how often that brand gets included because the LLMs know more about you. It also drives into citations because they see information, they trust this URL, and then the LLM is far more likely to bring that user back because that had a great shopping experience and it knew exactly what was on that page, or it explained your value prop and you know walked through all the details, or you know, highlighted why your customer service and you know, opening a checking account at your bank is the next best option.
So, you know, really excited to have done this with a number of huge brands. and I’ll tell you, we got a lot more work to do with with making the internet a little bit easier for AI and and scrunching the websites as we go.
Greg Kihlstrom (08:53)
That’s that’s great. Great, great to see this. And you know, c a couple couple questions as we as we wrap up here then. you know, how much I I saw some some of this in the interface itself, but you know, how much because this is a relatively new thing, you know, there’s lots of people that know a lot about SEO and and kind of the the old school way of doing things, how much of this is also educational, right? For those that are just kind of wrapping their heads around it.
Devin Stevens (09:21)
Yeah, that’s a great question. and I will say a lot of the discussions and you know, Greg, I’m fortunate enough I probably talk to anywhere from ten to fifteen different companies a week about this. So you know, a lot of it is is just helping people understand. And, you know, one of the things that I will tell you is, you know, we’re still early in in this trend and and these models are changing and you know, there’s you know, hundreds of engineers that are that are updating and making these tools work differently every single day. So
Greg Kihlstrom (09:30)
Yeah. Yeah.
Devin Stevens (09:48)
You know, one of the things that that we do and and kind of tied into making sure that, you know, these work and that, you know, brands don’t have to know this stuff is we work really hard to invest heavily in our research. so really kind of taking a data science lens to how these models are functioning, what they’re doing differently, you know, how they read or don’t read certain table structures. So you know, it is a lot of education and we’re we’re excited to to be helping and giving that information and collaborating with you know the brands and and really showing the performance improvements.
Greg Kihlstrom (10:17)
Yeah, love it. Love it. Well, Devin, thanks, thanks so much for joining and and showing this off. definitely I’m sure lots more to come. This, you know, we’re still early days in this whole space, right? So it’s there’s probably thankfully there’s there’s platforms like this that so people don’t have to know every everything that changes the next 24 hours or whatever. But yeah. if people want to learn more about scrunch, what’s the best way for them to do that?
Devin Stevens (10:45)
Yeah, definitely come to our website. a lot of information on there, definitely ways to get in contact with us. we’re we’ve had a little bit of press recently and some exciting news in terms of you know, now we’re a a site core company, which is extremely exciting. So in terms of you know, roadmap, thank you, and everything we’re doing, it’s you know, it’s how do we do more? How do we create more features? How do we do you know, even more to to help these companies? So we’ve got some huge releases coming out at the end of the month. we’ll be excited to to share with the world.
Greg Kihlstrom (10:55)
Yeah.
Yeah, congrats.
Love it, love it, look forward to seeing it. Well again, I’d like to thank Devin Stevens, head of solutions engineering at Scrunch for joining the show. You can learn more about Devin and Scrunch by following the links in the show notes.
