AI Crawlers & Technical
JavaScript Rendering
JavaScript rendering refers to content being generated in the browser by JavaScript after the initial HTML loads, as in single-page applications. Most AI crawlers do not execute JavaScript, so client-side-rendered content is invisible to them. Server-side rendering or pre-rendering is essential for AI search visibility.
Why AI crawlers see a different page than users
In a client-side-rendered app, the initial HTML is a near-empty shell; the visible content materializes only after JavaScript runs in the browser. Googlebot invests in a rendering pipeline that executes JavaScript, eventually. Most AI crawlers do not: GPTBot, ClaudeBot, PerplexityBot and the retrieval fetchers generally read the raw HTML response and move on. If your product descriptions, comparisons and documentation exist only post-render, AI engines receive a blank page, and content that does not exist for the crawler can never be cited, regardless of its quality.
Diagnosing the problem on your site
The decisive test: fetch a page with JavaScript disabled, or curl it, and inspect the raw HTML. If headlines, body copy and key facts are present, AI crawlers can read you. If you see an empty root div and script tags, you have an AI visibility problem hiding behind a perfectly normal-looking site. Frameworks like Next.js and Nuxt make this recoverable through server-side rendering or static generation, and pre-rendering services can patch older stacks. Watch for partial failures too: pages where the shell renders but pricing tables, FAQs or structured data are injected client-side.
Rendering strategy for AI visibility
The rule is simple: anything you want cited must be in the initial HTML response. Server-side rendering, static site generation or pre-rendering all satisfy this; pure client-side rendering does not. Prioritize money pages, documentation and comparison content first. After shipping a fix, confirm AI crawlers return and fetch the now-readable pages, since crawl activity is the earliest evidence of recovery. Geonimo's page optimization audits include checking that key content is server-rendered and machine-readable, alongside citability factors.
Frequently asked questions
Do any AI crawlers execute JavaScript?
As a working assumption, no. Google's infrastructure renders JavaScript for search, which partly benefits Gemini and AI Overviews, but the dedicated AI crawlers from OpenAI, Anthropic and Perplexity read raw HTML. Building for the lowest common denominator, complete content in the initial response, is the only safe strategy.
How do I check if my content is visible to AI crawlers?
Request your page with JavaScript disabled or via curl and read the returned HTML. If your important text, headings and data appear, you are visible. If the body is an empty application shell, AI crawlers see nothing. Test your highest-value pages individually, since rendering setups often vary across templates.
Is server-side rendering worth it just for AI visibility?
If AI engines are a meaningful discovery channel for your market, yes, because client-side rendering makes citation impossible no matter how good the content is. SSR or pre-rendering also improves load performance and resilience. Migrating key marketing and documentation pages first captures most of the benefit without a full rebuild.
Related terms
AI Crawler
An AI crawler is an automated bot operated by an AI company that fetches web pages to collect training data for language models or to retrieve fresh content for AI search answers. Examples include GPTBot, ClaudeBot and PerplexityBot. Each identifies itself with a user agent string and can be allowed or blocked via robots.txt.
Crawl Budget
Crawl budget is the number of pages a crawler will fetch from a site within a given period, shaped by the site's server capacity and the crawler's assessment of its value. Originally a Google Search concept, it now extends to AI crawlers, which typically fetch fewer pages and prioritize fresh, authoritative content.
Structured Data
Structured data is machine-readable markup, usually Schema.org vocabulary embedded in web pages, that explicitly describes what content means: an organization, product, FAQ, article or review. It helps search engines and AI systems disambiguate entities and extract facts reliably, supporting rich results and cleaner interpretation by answer engines.
Citability
Citability is the degree to which a web page's content can be easily retrieved, extracted, and cited by AI engines. Highly citable pages contain self-contained answer passages, explicit facts and statistics, clear structure, and current information, making them preferred sources when engines ground their generated answers in web content.
Last updated: 2026-06-11
Track this for your brand
Geonimo monitors how ChatGPT, Perplexity, Claude, Gemini and Google AI talk about your brand — and generates the content that gets you cited.
Get your free audit