In its Answer Economy report, published on 15 April 2026 from a March 2026 survey of 1,076 B2B software buyers, G2 found that 51 percent now start software research with an AI chatbot more often than with Google, up from 29 percent a year earlier.
Sixty-nine percent of those buyers said a chatbot led them to a different vendor than the one they had in mind, and about a third bought from a vendor they had never heard of.
Many brands losing those answers are not losing on content quality. They lose because a machine could not fetch the page, could not read the text on it, or could not establish that the page was about them.
What is LLM SEO, in plain terms
LLM SEO is the work of making your site retrievable, readable and attributable to the systems that now answer buyer questions: ChatGPT, Claude, Gemini, Perplexity and Google’s AI Overviews and AI Mode.
The same work is called generative engine optimization (GEO), answer engine optimization (AEO) or AI visibility optimisation. Google’s documentation defines both acronyms and adds that, from Google Search’s perspective, optimising for generative AI search is still SEO.
Three differences from classic SEO are worth internalising before you spend anything.
- The unit is the answer, not the ranking. Nobody sees position four. The model names your brand, or it does not.
- Retrieval is not ranking. Answers are assembled from several retrievals at once, including sub-queries the user never typed, so a page that ranks modestly can be cited and a page that ranks first can be skipped.
- Most citations are not yours. The pages describing you are frequently third-party: forums, encyclopaedias, review sites, press. Your own site is one input.
What stays the same: crawlable pages, real text, internal links, page experience and structured data matching what a visitor sees. Google states there are no additional technical requirements for its AI features and no special optimisations.
The five building blocks
- Access. Can the AI crawlers reach you at all?
- Retrievability. Can they read the page once they arrive?
- Content. Is anything on it worth quoting?
- Off-site evidence. Do the sources models cite say the same thing?
- Measurement. Do you know where you stand, and against whom?
Fixing four while one is broken produces nothing, so work them in order.
Block 1: Access, and the default that changes on 15 September
On 1 July 2026 Cloudflare launched controls that sort AI bot traffic into three behaviours, Search, Agent and Training, available to every customer including the free tier. Many crawlers fall into more than one category.
Two things happen on 15 September 2026, and the second has been widely misreported.
The ad-page default. In Cloudflare’s own words, for all new domains onboarding to Cloudflare, Training and Agent are blocked by default on pages that display ads, while Search stays allowed. Several news reports described a broader scope, including existing free-tier accounts. Cloudflare’s post specifies new domains. Open your security settings and read what your own zone says.
The Googlebot side effect. From 15 September, multi-purpose crawlers that combine Search with Training are judged by all of their behaviours. Because the most restrictive applicable setting wins, Cloudflare states that Googlebot, Applebot and BingBot will be blocked for customers who selected to block Training. That includes anyone who used the legacy one-click “Block AI bots” service.
Read that again if you flipped that switch during the AI-blocking wave of 2024 or 2025. A setting chosen to keep model trainers out can now take Google Search crawling with it.
Owners can opt out of the new defaults in security settings before the date. Enforcement happens at Cloudflare’s edge, so nothing in your robots.txt will show it. Server logs are where you will see it, or not see it. The pages that cite you run under the same rules, so comparison articles and industry blogs may quietly become less retrievable too.
One company, several bots
Treating each AI company as a single crawler is the most common technical error in this field.
OpenAI documents four user agents.
- GPTBot collects content that may be used for training.
- OAI-SearchBot surfaces sites in ChatGPT’s search features. Sites opted out will not appear in ChatGPT search answers.
- ChatGPT-User handles fetches triggered by a user action. OpenAI indicates robots.txt rules may not apply to it.
- OAI-AdsBot, added in 2026, visits pages submitted as ChatGPT ad landing pages.
The settings are independent. Blocking GPTBot is a training decision with no effect on ChatGPT search visibility. Blocking OAI-SearchBot is a visibility decision. Changes take roughly 24 hours to reach search.
Anthropic documents three robots.
- ClaudeBot collects content that may contribute to training.
- Claude-User retrieves pages in response to a user’s question. Anthropic states disabling it may reduce visibility for user-directed web search.
- Claude-SearchBot indexes content for search results. Disabling it may reduce a site’s visibility and accuracy there.
If your robots.txt still names the older Claude-Web token, it covers none of these.
Perplexity documents two. PerplexityBot builds the search index, and Perplexity recommends allowing it plus its published IP ranges. Perplexity-User handles user-triggered visits. Both take up to 24 hours to apply.
Google’s split catches experienced SEOs. Google-Extended is not a crawler and has no user agent string. It is a robots.txt token governing whether content trains future Gemini models, plus grounding in Gemini Apps and Vertex AI. Google states it does not affect inclusion in Google Search and is not a ranking signal.
So disallowing it does nothing to AI Overviews or AI Mode, which run on Search. It can affect how the Gemini app describes your company.
Below robots.txt sits the failure mode nobody audits: bot protection and rate limits returning 403 or 429 to crawlers the file explicitly allows. Perplexity publishes its own WAF whitelisting guidance, which tells you how often the file says yes and the firewall says no. Check your logs for the agents you believe you are allowing.
Block 2: Retrievability, or why your page may be an empty shell
Most AI crawlers do not run JavaScript.
The reference dataset is the Vercel and MERJ server-log analysis from December 2024, covering hundreds of millions of real crawler fetches. It found no evidence of JavaScript execution by the major AI crawlers. GPTBot downloaded JavaScript files in roughly 11.5 percent of requests and ClaudeBot in roughly 23.8 percent, without running them.
Googlebot renders. So the same page can rank respectably in Google Search and be invisible to ChatGPT, Claude and Perplexity.
The test takes a minute: fetch one of your key URLs without a browser, then search the raw response for a sentence you can see on screen. If it is not there, the retrieval crawlers do not have it. Recurring culprits:
- Client-rendered pricing and product detail
- Content in tabs or accordions that loads on interaction
- Specifications held in a JavaScript widget
- Main content behind a consent banner, which leaves the crawler holding the banner
The fix is server-side rendering, static generation, or hydration that still ships real text in the first response. On most sites this is the highest-value technical work available.
Block 3: Content a model can quote
Four properties separate pages that get cited from pages that do not.
- Specific, extractable facts. Prices, limits, integrations, certifications, coverage, versions, dates. Marketing adjectives cannot be quoted as evidence. Numbers and constraints can, and facts locked in images, gated PDFs or scripts do not exist for retrieval.
- Answers to the sub-questions. Comparison, integration, use-case and pricing pages resolve the narrow questions models generate on their own. A homepage rarely does.
- Entity clarity. If a model cannot tell what category you are in, who you serve and where you operate, it describes you cautiously or not at all. One canonical self-description, consistent naming, Organization data matching the visible page.
- Freshness signals. Publication and update dates give a model a reason to prefer your current statement over a two-year-old third-party summary of it.
What Google says you can stop doing
Google’s guide to optimising for generative AI features, updated 10 July 2026, contradicts a lot of confident advice in writing. Its mythbusting section says you can ignore, for Google Search:
- llms.txt and other special AI files, markup or Markdown. Search does not use them.
- Chunking content into small pieces for a model’s benefit.
- Rewriting text specifically for AI systems.
- Chasing inauthentic mentions across the web.
Structured data stays worth maintaining for rich results, but Google states it is not required for generative AI features and there is no special schema to add. Spinning up separate pages for every query variation, fan-out variations included, falls under its scaled content abuse spam policy.
On llms.txt: no major provider has publicly documented using the file in its consumer answer surfaces. Ahrefs found the large majority of llms.txt files receive no crawler requests at all, and an SE Ranking study across roughly 300,000 domains found no relationship between having one and being cited. Ship it if you serve developers or agents. Keep it out of a citation business case.
Eligibility stays binding: a page must be indexed and eligible to appear in Search with a snippet, and the site must not be excluded from Search generative AI features in the Search Console setting that governs that.
Block 4: Off-site evidence, digital PR and Reddit
Independent studies keep finding that a large share of AI citations goes to third-party platforms rather than brand websites. Reddit, Wikipedia, YouTube and LinkedIn sit near the top of nearly every 2026 dataset, from Similarweb’s analysis of roughly 600,000 citation events to the consolidated indexes in the trade press.
Treat the headline percentages with care. One study puts Reddit around 12 to 13 percent of US ChatGPT citations, another at 40 percent across engines. Volatility is worse than the spread: Semrush tracked the share of ChatGPT responses citing Reddit falling from close to 60 percent in early August 2025 to around 10 percent by mid-September.
The reading is not “go win Reddit”. It is that your domain is one voice among several, and the others are easier to influence than most teams assume.
- Fix the encyclopaedic layer first. An outdated Wikipedia entry, Crunchbase profile or directory listing keeps resurfacing. Corrections there are cheap and durable.
- Do digital PR for citation, not for links. One placement in a publication models actually retrieve beats ten links on sites nobody cites. Pitch data, original research and named expert commentary.
- Use Reddit as a participant, not a campaign. Answer questions in the relevant subreddits from an account with history, disclosing who you work for. Undisclosed promotion breaches Reddit’s rules and is exactly what Google calls inauthentic mentions.
- Keep review profiles current. G2, Capterra, Trustpilot and sector equivalents are structured, dated and heavily retrieved. Old feature lists there age into AI answers.
- Use video and LinkedIn deliberately. Both rank high in citation studies, and a clear explainer or a named specialist’s profile is often the evidence a model uses to describe capability.
- Let your citation data set the target list. Whatever recurs as a source in your category is the PR brief. Everything else is guesswork.
Block 5: Measurement, or where you stand right now
On 3 June 2026 Google announced dedicated Search generative AI performance reports in Search Console. The Search view isolates impressions from AI Overviews and AI Mode by page, country, device and date, and the rollout has been staged, so an empty tab is not necessarily your fault.
It reports impressions, not clicks, click-through rate, average position or queries. It confirms a link to your page was displayed inside an AI feature. It does not say whether the answer named your brand, who was named alongside you, in what order, or in what framing. It covers Google’s surfaces only, so ChatGPT, Claude and Perplexity stay unmeasured.
For teams comparing options, the Best Tool for LLM SEO is the one that records the actual answer, with citations attached, on a schedule, across the engines your buyers use.
Truffle runs real buyer prompts daily across six surfaces, ChatGPT, Claude, Gemini, Perplexity, Google AI Mode and Google AI Overviews, alongside the classic SERP top ten, with Grok as a paid add-on. Each run stores the response, the mentions, the competitors named and the sources cited.
One caveat belongs in the open, and Google makes the same point: no third-party tool has access to Google’s internal ranking or AI systems. What a tool can do is run the prompt, record the output and turn observations into a trend. Most AI visibility tools are mirrors: they report how often your brand shows up in ChatGPT, Perplexity or AI Overviews and stop there. Truffle’s GEO Strategy feature goes one step further and turns that data into specific recommendations — what to fix on your own site, and which external sources to influence, since those are what the models actually cite. Because it covers both the onsite and the offsite side, you end up with a prioritised roadmap rather than another dashboard to interpret. That shift from monitoring to acting is Truffle’s core differentiator.
Monitoring competitors properly
Competitive monitoring only works at the level of the individual question. A company-wide share of voice averages the prompts you own with the ones you never appear in, which makes it unusable.
Track per prompt who else is named, in what order, with what framing, and which sources support them. A rival appearing with three citations from one trade publication is a PR brief. A rival appearing because your pricing page is client-rendered is an engineering ticket.
Organise prompts by persona, market, language and product line rather than as a flat keyword list, because the questions that decide a deal are full sentences with constraints inside them.
Truffle turns each gap into a prioritised plan across content, onsite technical work and offsite sources, with connectors for Search Console, Google Analytics 4 and Looker Studio, plus alerts on a mention drop or a new competitor. Recommendations are in every plan, including the seven-day trial, which covers one brand and needs no credit card. A free single audit runs without an account.
Whatever you use, record a baseline first: twenty real buyer prompts across the engines that matter, with citations and competitors, before you change anything. Without it you cannot tell a fix from noise.
Frequently asked
Can blocking AI training really block Googlebot? On Cloudflare, from 15 September 2026, yes. Multi-purpose crawlers are judged by all their behaviours under the most restrictive applicable rule, and Cloudflare names Googlebot, Applebot and BingBot.
Does disallowing Google-Extended remove me from AI Overviews? No. Google states it does not affect inclusion in Google Search. It governs Gemini Apps grounding and model training.
Do I need an llms.txt to be cited? There is no evidence for it. Google says Search ignores the file and third-party studies found no citation effect. It is useful for agents and developer tooling.
My page ranks first in Google. Is it readable by ChatGPT? Not necessarily. Googlebot renders JavaScript and the AI retrieval crawlers do not.
How long does LLM SEO take? Access and rendering fixes can change retrieval within days. Content and off-site work moves answers over weeks, and surfaces refresh at different speeds, which is why you re-run the same prompt set instead of checking once.
