The inside experiment | Your impressions (not clicks) are going places...
Hey there,
Your conversion rates look solid. Your test results are clean. But something feels off, doesn't it?
Traffic's down year-over-year and month-on-month, yet your business metrics are holding steady, maybe even climbing. Your beautifully optimized landing pages are converting like champions, but fewer people are seeing them. And that nagging feeling in your gut? It's telling you the rules of the game just changed while you weren't looking.
So what's happening?
The audience you've been running experiments for is quietly being replaced by one you never knew existed.
AI agents have been crawling your site, extracting your carefully crafted copy, and serving it up in ChatGPT responses—no clicks, no conversions, no trace in your analytics.
Your perfectly optimized experience? It's completely invisible to the traffic source that’s about to matter most.
👉 Are your tests actually reaching your full audience? Nope.
👉 Will we keep optimizing for humans while ignoring the machines? Probably, until it's too late.
The results we celebrate (while our audience disappears)
The headline that lifted CTR by 34%. But doesn't quite sound right when an AI tries to summarize it.
The perfectly A/B tested product page. That gets completely ignored when ChatGPT answers "what's the best solution for X?"
The mobile-first responsive design boosted conversions by 23%. While being completely unreadable to the bots that drive discovery.
Your dashboards may look productive, but your audience is slipping through the cracks unnoticed.
Metrics that’ve started to miss the point
1. Click-through rate
Great for humans who click. Not so great for AI agents that extract your content without ever visiting your site. You're measuring a shrinking audience.
2. Conversion rate optimization
You've perfected converting visitors into customers. But what happens when discovery happens in a chat window, and your "optimized" page never loads?
3. Engagement metrics
Time on page, scroll depth, and interaction rates are all meaningless when AI scrapes everything in milliseconds and leaves.
4. Page load speed
You've optimized for Core Web Vitals and human patience. AI agents don't care about loading animations. They want raw, structured content they can parse instantly.
5. Visual appeal testing
That beautiful hero section that tested 40% better? AI can't see images, doesn't care about your color palette, and treats your layout as meaningless markup.
These metrics aren't wrong; they're just incomplete. You're optimizing one half of a two-audience problem. It is why even your winning tests might be AI-hostile.
Meanwhile, that "boring" variation you killed in testing? The one with simple headlines and straightforward copy? That's exactly what AI needs to cite your content.
Two audiences, but still one dashboard?
First are the humans who arrive with high intent. Second are the AI agents that discover and distribute.
Traditional A/B testing optimizes for audience 1 while ignoring audience 2. But audience 2 will slowly control whether audience 1 ever finds you.
The fix: Don't kill conversion. Just lead with clarity and focus on impressions.
For AI discovery, structure content like a dataset. Clear headers, schema markup, and summaries that bots can actually parse and cite.
For human conversion, strip the friction. When someone arrives (probably from an AI recommendation), make it dead simple to validate and convert.
Ask:
What's in it for the AI agent?
Why does this content matter right now?
Could a language model extract your key points and still make sense?
TL;DR
Track the conversion metrics. Celebrate the conversion wins. Just don't build your entire strategy around optimizing for an audience that can't find you.
Watch for the "crocodile effect" where clicks decline, but impressions increase. It's your strongest signal that AI is consuming your content without sending traffic.
In case you missed it
Be where your audience is, and get an invite to the GEO webinar
AI is changing how people find answers, and your content may be invisible. Go here to learn why clicks are vanishing, what Generative Engine Optimization (GEO) means, and how to make your CMS and content visible to AI.
Why you should optimize for impressions instead of clicks
Impressions are the new clicks. In the era of AI visibility, being seen is now the ultimate SEO currency. Read the full blog here to learn more.
10 A/B test examples that work
We analyzed over 127,000 experiments across retail, SaaS, and B2B. Here are 10 best A/B testing examples that drove results and how to make them work for you.
Introducing the Optimizely MCP Server for Experimentation
Optimizely Experimentation MCP Server helps you manage your feature flags and experiment lifecycle entirely from your Integrated Development Environment (IDE). Read the full blog here to learn more.
Want the best of experimentation and content marketing delivered to your inbox? Subscribe to our newsletter for exclusive insights, practical tips, and standout content, straight from our team to you: https://www.optimizely.com/newsletter/