Google-Extended

Google-Extended never appears in your server logs as a visiting bot. It is a directive token that Google's existing crawlers check when deciding whether fetched content may be used for Gemini model training. Adding User-agent: Google-Extended with Disallow: / to robots.txt tells Google: keep crawling and indexing as usual, but do not feed this content into generative model training. This design separates the training decision from the search indexing decision entirely.

The most consequential misunderstanding about Google-Extended: blocking it does not remove you from Google AI Overviews or AI Mode. AI Overviews are built on Google's normal search index, populated by standard Googlebot crawling. The only way to stay out of AI Overviews is to restrict regular search indexing with noindex or nosnippet controls, which also damages your classic search presence. So publishers can block Google-Extended to withhold training data while keeping every bit of their Search and AI Overviews visibility intact. The two systems are governed independently.

Because blocking Google-Extended carries zero search or AI Overviews penalty, it is the lowest-risk training opt-out available, and many large publishers use it. The trade-off is subtler: content excluded from Gemini training may leave future Gemini models with weaker baseline knowledge of your brand, relevant as Gemini powers more Google surfaces. Brands prioritizing AI visibility typically leave it allowed; teams tracking how Gemini and AI Overviews represent their brand can monitor outcomes with Geonimo's multi-model monitoring before changing policy.