Your Site Has Two Audiences Now: Humans And AI Agents
This week's news shares one root: AI stopped being a feature marketers test and became infrastructure they have to govern, verify and measure.

Three threads mattered in marketing this week, and they share one root: AI stopped being a feature you test and became infrastructure you have to govern. United Airlines began treating search crawlers and AI agents as two different categories with two different permissions; fresh industry data suggested AI-generated audience insights are quietly hurting campaign performance; and video ad inventory kept splintering across Twitch, YouTube Shorts and LinkedIn’s connected TV.
The practical through-line for a working marketer: every AI system that touches your marketing now needs three things it did not need a year ago — a stated policy, a human check, and a way to measure it.
Thread 1: Crawling and acting are now separate decisions
For twenty years, the website question was binary — let the crawler in, or don’t. A crawler is an automated program that reads pages so a search engine can index them. That question is now two questions, because a second kind of visitor has arrived.
Search Engine Journal reports that United Airlines has kept its site open to search engine indexing while requiring AI agents to ask permission first. An AI agent is software that doesn’t just read a page — it acts on it, filling forms, comparing fares, and in some cases completing a booking on a person’s behalf.
That distinction is the whole story. Indexing affects your visibility. Agentic access affects your pricing, your inventory, your support queue and your conversion data. United is saying yes to the first and “ask first” to the second, which is a defensible position for any brand whose site is transactional rather than purely informational — Indian travel, ticketing, insurance comparison and direct-to-consumer brands very much included.
The volume question is no longer theoretical either. Search Engine Journal also covered a security firm’s observation that Meta’s Muse agent now accounts for the largest share of AI agent requests it sees, with many of those requests made live on behalf of a human user. Agent traffic is concentrating around a handful of large platforms, fast.
Why block lists won’t hold
The instinct is to reach for robots.txt — the file that tells automated visitors what they may fetch — and start adding names. Chris Green argues in Search Engine Journal that this approach is already behind the problem. Block lists are reactive, they go stale the moment a new agent launches, and they treat every bot as one category when the useful questions are about purpose, rate and identity.
A more durable setup asks: who is this, what are they allowed to do, how fast may they do it, and is any of it logged? That is bot management, not blocking.
Which makes the week’s quietest story the most useful one. Google published guidance on isolating search traffic in analytics and server logs, and server logs are suddenly fashionable again for a simple reason: your analytics tool shows you humans, while your logs show you everything. If you cannot see agent traffic, you cannot write a policy about it, and you certainly cannot explain a traffic drop to your CMO.
Thread 2: AI output is failing quality control
Two stories this week landed on the same bruise. Marketing Dive reported that AI-derived audience insights are associated with worse campaign results — the plausible-sounding synthetic persona performs worse than audience understanding grounded in real research. Separately, Marketing Dive’s Sociable column flagged that Meta’s AI ad tools generated ads containing discounts advertisers had never approved.
Put those together and you get the shape of the 2026 problem. AI is not failing at fluency; it is failing at accountability. A generated persona sounds like research. A generated ad looks like your ad. Neither carries a record of who approved what, and the second example is not just a performance issue — an unapproved discount is a pricing and legal exposure.
The counterweight is unglamorous: go find out what humans actually said. Brent Csutoras makes the case in Search Engine Journal that Reddit review threads surface product objections and comparison language that survey panels smooth over. HubSpot’s look at consumer sentiment toward AI influencers points the same way — audiences are more sceptical of synthetic creators than the launch announcements suggest, and sentiment data is cheap insurance before you commission one.
Thread 3: Video money is being re-cut across more screens
Three announcements, one direction of travel.
| Platform | What changed | Who should care |
| YouTube | Target Frequency can now pair non-skippable ads with Shorts | Brand teams buying reach and frequency together |
| Twitch | Shoppable and skippable ad formats | Gaming, D2C and youth-skewed brands |
| Connected TV ads launched in the UK | B2B marketers with brand budgets |
Connected TV, or CTV, means ads served on internet-delivered television — the same screen as broadcast, bought like digital. LinkedIn selling it signals that business-to-business advertising is finally allowed to behave like brand advertising rather than lead-gen plumbing.
For Indian marketers the relevant shift is frequency, not format. Attention is spread across Shorts, CTV, live streaming and short-form apps, so a plan that buys reach channel-by-channel will overexpose the same viewer three times and miss the next one entirely. A ₹20 lakh video budget split across four surfaces without a shared frequency view is not a media plan; it is four small campaigns competing for the same eyeballs.
Thread 4: “I use AI” is no longer a skill claim
One number worth keeping. Greg Jarboe notes in Search Engine Journal that 72% of workers say their AI skills have never been tested. For pay-per-click (PPC) managers and SEO specialists, that gap is an opportunity: the differentiator is shifting from AI familiarity to demonstrable AI judgement — documented prompts, before-and-after results, and a clear account of what you overrode and why.
What this means for you
- Write an agent policy this month. One page: which AI crawlers may index, which agents may transact, what rate limits apply. Separate indexing from action, as United did.
- Get access to your server logs. Ask your developer or hosting provider today. Analytics alone will not show you agent traffic, and this is becoming a board-level traffic question.
- Never ship an AI-generated persona as research. Use it to form a hypothesis, then validate against customer calls, support tickets or Reddit threads before it reaches a media brief.
- Audit automated ad output weekly. Specifically check prices, discounts and claims. Assume generative ad tools will occasionally invent an offer, and catch it before a customer does.
- Plan video by frequency, not by platform. Set a total weekly exposure target per person first, then split across Shorts, CTV and streaming.
- Build an AI proof file. Three documented cases where your judgement changed an AI output’s outcome. That is now a hiring asset.
Frequently asked questions
Should I block AI crawlers from my website?
Not as a blanket rule. Blocking AI crawlers can remove your brand from AI-generated answers, which is now a visibility channel. The better approach is to decide separately what automated visitors may read and what they may do — allow indexing, and gate actions like booking, pricing lookups or form submissions.
What is the difference between an AI crawler and an AI agent?
A crawler reads your pages so a system can index or summarise them. An agent takes action on your site on behalf of a user — searching, comparing, sometimes transacting. Crawlers affect your visibility; agents affect your operations, your analytics and your revenue data.
Can I trust AI-generated audience insights?
Treat them as hypotheses, not findings. Reporting from Marketing Dive this week links AI-derived audience insights to weaker campaign results. Use AI to draft the questions, then answer them with real customer evidence before spending media budget.
Is connected TV worth it for Indian B2B marketers yet?
LinkedIn’s CTV launch is UK-first, so India will wait. In the meantime, the transferable lesson is to start measuring total video frequency across YouTube, streaming and short-form now, so you have a baseline when business-to-business CTV inventory does reach the Indian market.
