Give three different AI shopping agents the same product, the same query, and the same set of competitors, and they don’t return the same answer. In a controlled study of frontier agents, Claude Sonnet 4 picked the Fitbit Inspire from a fitness-watch assortment about 45% of the time, while GPT-4.1 and Gemini 2.5 Flash landed on it closer to 25% (Allouah et al., 2025). Same catalog, same shelf, materially different verdict, and the only thing that changed was which model was doing the choosing.
That’s a problem for how an agent visibility assessment is usually read. The Brand Visibility for Agentic Commerce (BVAC) framework scores a brand’s data surface — what an agent can resolve, compare, and trust — and the implicit assumption is that the agent is a fixed quantity the data is measured against. The evidence says the agent is a variable too. A score is a reading taken against a particular model at a particular moment, and both halves of that qualifier matter more than the framework’s calendar-based instinct tends to assume.
A note on the evidence before leaning on it. The study built a provider-agnostic sandbox called ACES, ran randomized one-shot selection tasks across eight product categories of US retail goods, and tested three model families at two snapshots — the August 2025 frontier and its December 2025 successors. It’s descriptive rather than predictive, and the specific percentages are pinned to the models tested on the dates tested. The structural patterns are what travel, and the patterns are the point.
Upgrades behave like demand shocks
The numbers don’t just differ across providers at one moment. They lurch when a model is upgraded. The Fitbit Inspire’s share climbed from roughly 45% under Claude Sonnet 4 to about 77% under Claude Opus 4.5, and in the same window it fell from 25% under GPT-4.1 to 6% under GPT-5.1 (Allouah et al., 2025). One provider’s upgrade lifted the product; the other’s nearly erased it. In the iPhone-case category the modal choice flipped outright — GPT-4.1 picked one brand 63% of the time, and its successor picked a different brand 95% of the time, dropping the former favorite to 5%. A swing like that would trigger a fire drill in any other channel. Here it arrives with no warning and no cause the brand can point to, because nothing in the brand’s own catalog changed.
It doesn’t even take a headline launch. The study documents a quieter trigger: the transition from a Gemini preview build to its final release reallocated shares across products and inverted the model’s position bias. A finalization most brands would never notice functioned as an exogenous shift in demand. For a framework that already treats model releases as recalibration triggers, this is the empirical case for why the trigger has to include the upgrades nobody announces.
Concentration, and the brands that never appear
Aggregate the choices and a second pattern shows up: agents collapse demand onto a few modal products and leave the rest at zero. In the stapler category, Amazon Basics dominated the selections while one competitor, Arrow, was never chosen at all (Allouah et al., 2025). Not rarely. Never.
That’s decision invisibility with a measured cause. The brand isn’t filtered out by a data gap it could fix — it’s filtered out by a model prior, and the outcome is the same silent absence the framework was built to detect, now traceable to the agent rather than the catalog. The concentration effect isn’t confined to one sandbox, either. Jellyfish, simulating 50 running-shoe shopping tasks across several models on its Agent Shopper platform, found a single brand surfacing in 70% of them (Jellyfish, 2026). When the winner-take-most dynamic sets in, the brands outside the modal set don’t lose by a little. They don’t appear.
The biases survive the interface
A reasonable hope is that all of this is an artifact of agents squinting at cluttered webpages, and that the cleaner, structured-feed future the commerce protocols are building will behave more predictably. The study tested that hope directly and it didn’t hold. Position on the page exerted a large effect on selection, the direction of that effect differed sharply between models — the slot GPT-4.1 favored most was the one GPT-5.1 favored least — and telling an agent to ignore position didn’t remove it (Allouah et al., 2025). Most relevant for where commerce is headed, the same biases persisted in a headless setting where the agent saw only a ranked JSON list with no images at all. These aren’t visual-parsing quirks. They’re properties of the models, and they’ll carry into the standardized API surfaces a brand reaches through its protocol coverage. The structured-feed future isn’t a more stable one; it’s the same instability with the pictures removed.
What this changes in the measurement loop
The framework’s measurement layer tracks four directional signals — AI share of voice, citation rate, substitution rate, and brand-agent transaction performance — under the attribution gap. The study’s implication is that each of those has to be read as a model-indexed series rather than a single trend line, with a model release treated as a first-class recalibration point, including the silent finalizations. A baseline taken against last quarter’s frontier may describe an agent that no longer exists.
One nuance keeps this from being a counsel of despair. The models are getting better at the parts that should be stable: by December 2025 the latest versions almost never failed a simple test where one product was unambiguously cheaper or higher-rated. Competence on single-attribute rationality improved even as multi-attribute selection stayed model-dependent. The instability isn’t immaturity that capability will iron out; it’s a feature of how these systems weigh trade-offs, and it persists across generations.
The constructive half is that the surface is volatile but responsive. The same study had a seller-side agent rewrite product descriptions to better match the query — front-loading “Office” for an office-lamp search, appending “Fitness” to a watch title — and a single edit produced a statistically significant share gain in about a third of the category-model pairs, with five of the six models rewarding it on average (Allouah et al., 2025). Some edits backfired on individual models, which is the warning inside the good news. This is Differentiation Encoding under a moving target: encoding the right attributes still moves selection, but “encode once and move on” isn’t a strategy when the reader keeps changing.
Where to start
Take a brand that ran the assessment, scored Differentiated on its strongest dimension, and filed the result. Three months later a major model ships. Re-run the same assessment against the new model and the effective standing may have moved — possibly up, possibly toward the floor — with not a single change to the brand’s own data to explain it. The roadmap item that follows isn’t a remediation task. It’s a cadence: stamp every assessment with the model version it was measured against, and re-run the measurement loop when the model changes rather than when the quarter does. A brand operating in a multi-protocol, multi-agent field is being scored by several models at once, and the composite it cares about is the one its actual buyers’ agents produce this month.
A visibility score with no model attached to it has a hidden expiry date. Treating an assessment as a durable property of the brand means, sooner than feels intuitive, measuring against an agent that has stopped behaving the way the score assumed — and finding out only when the orders that used to arrive don’t.
References
Allouah, A., Besbes, O., Figueroa, J. D., Kanoria, Y., & Kumar, A. (2025). What is your AI agent buying? Evaluation, biases, model dependence, and emerging implications for agentic e-commerce (arXiv:2508.02630). arXiv. https://arxiv.org/abs/2508.02630
Jellyfish. (2026, March 2). Brand discovery is being reshaped by AI: Generative engine marketing and the future of influence. Jellyfish. https://www.jellyfish.com/en-us/blog/brand-discovery-is-being-reshaped-by-ai/
