AI search
Allow Google-Agent traffic: Google’s new guide to letting its AI agents into your site, the six ways firewalls still block them, and why ChatGPT Wallet makes this the week to check
Google published “Allow Google-Agent traffic” on 17 November 2026. What the agent is, how to verify it, why robots.txt cannot stop it, and six blocks to fix.

Key takeaways
- On 17 November 2026 Google published a technical guide titled “Allow Google-Agent traffic”. Google-Agent is “the name used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request”, and the guide exists, in Google’s words, “to help publishers that want to enable Google-Agent for their website”. It is the first time Google has told site owners how to let its AI agents through a CDN, firewall or bot-detection service.
- Google-Agent is not new. Google added the user agent and its IP range file on 20 March 2026 and said it would be “used by Google agents hosted on Google infrastructure to navigate the web and perform actions upon user request”. What is new is the operational detail: three ways to verify a request, a four-step allow rule, and six reasons a legitimate request can still be refused after you think you have allowed it.
- robots.txt will not keep Google-Agent out and will not let it in. Google classes it as a user-triggered fetcher, and its documentation says those fetchers “generally ignore robots.txt rules” because a person asked for the fetch. The decisions that matter are made in your CDN and web application firewall, which is exactly where accidental blocks happen.
- The same week, OpenAI launched ChatGPT Wallet (13 November) and said plainly that “websites need to ensure agents are not inadvertently blocked” for its agents to pay. Cloudflare added per-agent policies on 4 November, and its defaults for new domains since 15 September block agents on pages with ads. Blend Commerce’s September audit of 200 retailers found 29 refused an AI shopping agent outright on two attempts. Blocking is the default state of much of the web, and it is now a visibility problem.
- What to do now: find Google-Agent and chatgpt.com signatures in your logs, write an allow rule that verifies user agent, IP range and signature rather than trusting the string alone, make sure it outranks your broad bot rules and covers search, filters, add-to-cart and API calls, and keep the IP list synced. The checklist below follows Google’s own troubleshooting list point by point.
What did Google publish on 17 November 2026, and why now?
Google’s crawling documentation changelog for 17 November 2026 has a single entry: “Added guidance on allowing Google-Agent”. The what: “Published a technical guide on how to ensure that Google-Agent has access to your website.” The why: “To help publishers that want to enable Google-Agent for their website.” The guide itself, “Allow Google-Agent traffic”, opens with a definition and a promise. Google-Agent “is the name used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request. Explicitly allowing Google-Agent traffic in your content delivery network (CDN), web application firewall (WAF), or bot-detection service helps prevent legitimate requests from being blocked while preserving your existing security controls.” It then covers three things: how Google-Agent identifies its requests, how to allow the traffic in a CDN, firewall or bot-detection service, and troubleshooting.
Barry Schwartz covered the document the same morning on Search Engine Roundtable and made the point that matters for anyone who thinks they have seen this before: “Google-Agent is not new, we covered it in March, but this is a new help document on allowing access for Google-Agent.” The March entry in the changelog, dated 20 March 2026, reads: “Added the Google-Agent user agent and added the IP ranges for user triggered agents to the documentation on verifying requests from Google crawlers and fetchers.” Its reason: “The Google-Agent user agent is rolling out over the next few weeks, and will be used by Google agents hosted on Google infrastructure to navigate the web and perform actions upon user request.” Eight months later, Google has decided that site owners need a how-to, not just a line in a reference list.
The timing is not accidental. Google’s own guide to optimising for generative AI features defines AI agents as “autonomous systems that can perform tasks on behalf of people, such as booking a reservation or comparing product specifications”, and says “browser agents may access your website to gather the data they need to complete these tasks”. On 4 November Google added booking and quote links for US home services inside AI Mode, through partners Robby Stein named as Angi, Thumbtack, Housecall Pro and Jobber, with ServiceTitan “coming soon”, as Search Engine Journal reported. On 11 November Google’s Search Central blog warned, in a post about moving to a CDN, that blanket bot protection without exemptions means “your pages, shopping listings, and your chicken pictures will begin to drop out of Search”. The agent guide is the third piece of the same message: Google wants its automated visitors, agents included, to get through your front door.
- 20 March 2026Google adds the Google-Agent user agentGoogle’s crawling changelog adds Google-Agent and the user-triggered agents IP range file, saying the agent “is rolling out over the next few weeks”. Semrush notes Project Mariner is named as an example.
- 4 May 2026Google documents Web Bot AuthGoogle adds a guide to verifying requests with Web Bot Auth “during the experimental phase, as some Google user agents are now starting to use Web Bot Auth”.
- 1 July 2026Cloudflare splits AI traffic into Search, Agent and TrainingAll Cloudflare customers, including the Free plan, can set separate policies for the three behaviours. Agent “covers automated activity acting in real time on a person’s behalf, such as chat fetch bots and browser-use agents”.
- 15 September 2026Cloudflare changes defaults for new domainsNew domains onboarding to Cloudflare get Training and Agent bots “blocked on pages that display ads, while Search remains allowed”.
- 4 November 2026Cloudflare adds per-agent policies; Google adds AI Mode booking linksCloudflare launches a Personal Assistants analytics category and lets site owners set Allow, Block on pages with ads, or Block for each supported personal agent. The same day Google adds US home-services booking and quote links in AI Mode.
- 11 November 2026Google: exempt verified crawlers from bot challengesA Search Central blog post on CDN migrations says blanket WAF or CAPTCHA rules leave Google’s crawlers “stuck on an error page” and that pages and shopping listings “will begin to drop out of Search”.
- 13 November 2026OpenAI launches ChatGPT Wallet and a Commerce Partner PlaybookOpenAI says “websites need to ensure agents are not inadvertently blocked” and asks merchants to configure CDNs and firewalls to recognise Web Bot Auth signatures.
- 17 November 2026Google publishes “Allow Google-Agent traffic”A technical guide with three verification signals, a four-step allow rule and six troubleshooting checks. The same day Brodie Clark reports Google forcing a sign-in after a first search, which Google’s Rajan Patel says he does not believe is intended.
What is Google-Agent, and how is it different from Googlebot?
Google sorts its automated clients into three groups. Common crawlers such as Googlebot “always respect robots.txt rules for automatic crawls”. Special-case crawlers such as AdsBot work under an agreement with the site. And user-triggered fetchers, per Google’s list of user-triggered fetchers, “are initiated by users to perform a fetching function within a Google product”, and “because the fetch was requested by a user, these fetchers generally ignore robots.txt rules”. Google-Agent sits in that third group. Its entry says it “is used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request” and “uses IP ranges from user-triggered-agents.json”. That is the practical difference. Googlebot reads your site to build an index. Google-Agent visits because a specific person asked Google to do something on it, and it may search, filter, click into a product and add to a basket on that person’s behalf.
Leigh McKenzie, writing for Semrush in March when the user agent first appeared, put the distinction simply: “Most crawlers you see in your server logs are performing background processes like Googlebot crawling your pages.” Google-Agent “reflects that a real person asked a Google AI agent to do something on their behalf — and the agent went to your site to do it.” He also warned against over-reading it, and that caution still applies in November. Google’s guide does not say which products send Google-Agent today, how much traffic it generates, or how a site earns a visit. It only says how to let the visits through.
“Google-Agent appearing in your logs doesn’t mean AI agents are out there completing purchases and filling out forms at scale today. The protocols, standards, and functionality that will make that seamless are still being built.”
How do I verify a request is really from Google-Agent?
Google’s guide gives three signals, and the order matters because each one is harder to fake than the last. First, “Each Google-Agent request includes a User-Agent HTTP request header containing “Google-Agent””. The fetcher list prints the two full strings, and both begin with “Mozilla/5.0” and look like Chrome on Android or Linux until the “compatible; Google-Agent” token near the end, which is why a rule that keys on “Mozilla” misfires. Second, the request “comes from an IP address from user-triggered-agents.json”. Google publishes that file at a fixed URL; the version we fetched this run carries a creation time of 3 March 2026 and lists three IPv4 prefixes and one IPv6 prefix. Third, and still labelled an experiment, Google “signs requests made by Google-Agent with the HTTP Message Signatures standard (RFC 9421)”. Participating requests carry a Signature-Agent header set to g="https://agent.bot.goog" plus Signature and Signature-Input headers labelled g, and the public keys live at https://agent.bot.goog/.well-known/http-message-signatures-directory.
| Signal | What to check | Caveat in Google’s own words |
|---|---|---|
| User-Agent header | Contains the token “Google-Agent”; full mobile and desktop strings are listed on the user-triggered fetchers page | “Caution: The user agent string can be spoofed.” |
| IP address | Falls inside a prefix in user-triggered-agents.json (3 IPv4 prefixes and 1 IPv6 prefix in the file dated 3 March 2026) | “Confirm that IP-based rules stay synced with the latest ranges.” |
| Web Bot Auth signature | Signature-Agent: g="https://agent.bot.goog", with Signature and Signature-Input headers labelled g, verified against the key directory at agent.bot.goog | “We don’t sign every request of a particular agent. Be sure that you fall back to the established methods of bot verification.” |
| robots.txt | Not an access control for this agent | User-triggered fetchers “generally ignore robots.txt rules”. |
Google’s separate Web Bot Auth page explains the half-finished state of the third signal. The protocol is “a draft specification developed by the IETF WBA Working Group, and it may change over time”, “Not all Google user agents are using Web Bot Auth”, and “Google is not yet signing every request of agents using the protocol”. Google therefore recommends that “in addition to Web Bot Auth you continue relying on IP addresses, reverse DNS, and user-agent strings as we gradually roll out signed traffic”. It also says that “Major bot-detection services, CDNs, and WAFs support Web Bot Auth” and that a subset of Google-Agent requests “are authenticated as https://agent.bot.goog. If your provider supports the protocol, they likely verify this automatically.” The allow rule Google proposes for everyone else has four steps: confirm the User-Agent contains Google-Agent, confirm the IP is in the JSON file, verify the signature if one is present, and “For verified requests, skip bot challenges.”
Why does Google-Agent still get blocked after I allowed it? Google’s six troubleshooting checks
The most useful part of the guide is the troubleshooting list, because it describes the real failure modes of modern edge security rather than the textbook ones. All six are quoted from the document, and each maps to a setting you can open today.
| Check | What Google tells you to confirm | Where to look on your side |
|---|---|---|
| Rule priority | Your allow rule “has higher priority than (and evaluates before) broad cloud or data center ASN blocks, managed bot rules, or category-level AI agent block policies that could override a verified bot setting” | Rule order in the WAF; any “AI agents” category toggle; ASN or data-centre block lists |
| Interactive challenges | The rule “skips interactive CAPTCHAs and JavaScript challenges in addition to HTTP blocks, including standalone in-page CAPTCHA widgets triggered on search or form submission” | Challenge actions, not just block actions; CAPTCHA widgets embedded in site search and forms |
| In-journey actions and APIs | Allowing covers “actions beyond initial homepage loads, including on-site search, filtering, clicking into detail pages, add-to-cart, booking, and underlying XHR, GraphQL, or api.* endpoints” | Rules scoped to the homepage or HTML only; separate protection on api.* hosts |
| Shared-IP rate limits | Per-IP rate limiting rules “account for or exempt verified Google-Agent IP ranges, as concurrent requests from multiple users share the same egress pool” | Rate-limit thresholds that assume one IP is one person |
| IP range updates | IP-based rules “stay synced with the latest ranges in user-triggered-agents.json” | Hard-coded prefixes copied once and never refreshed |
| Proxy headers | Load balancers and proxies “preserve the original client IP, the User-Agent header, and (for signed requests) the Signature, Signature-Input, and Signature-Agent headers” | Header stripping or rewriting between the edge and the origin |
Two of these deserve a closer look because they are where a site that “allows bots” still fails. The in-journey point is a direct consequence of what an agent does: a shopper’s agent that can load your homepage but is challenged on the search results page, or receives a 403 from the GraphQL endpoint behind add-to-cart, cannot finish the task it was sent to do. The shared-IP point explains a class of mysterious blocks: Google says many users’ agent sessions leave through the same egress pool, so a rate limit tuned for one human per IP will trip on legitimate traffic. If you run technical SEO for a transactional site, these two checks belong in the next crawl-access review alongside Googlebot and Storebot.
Is this the same problem OpenAI is describing with ChatGPT Wallet?
Yes, and OpenAI said it more bluntly. Introducing ChatGPT Wallet, published on 13 November 2026, describes a wallet that lets OpenAI’s agents, called dots, pay through a merchant’s existing checkout using a network payment token, so that “ChatGPT Wallet” never shares “their actual card details with the business or agent”. It is “initially available to ChatGPT Pro subscribers aged 18 and older in the US with eligible US-issued cards”, built “in partnership with American Express, Chase, Mastercard, and Visa”. OpenAI puts the weekly ChatGPT audience at 1.2 billion people. Then comes the sentence every ecommerce team should read twice: “To accept payment from ChatGPT Wallet, websites need to ensure agents are not inadvertently blocked, and payment and risk systems need to recognize legitimate purchases made by agents.” Further down: “Some sites may unintentionally block purchases through ChatGPT Wallet or other agentic payment products.”
OpenAI’s fix sits in its agent access guide for developers, and it mirrors Google’s structurally while differing in detail. “ChatGPT’s cloud browser can already use your site. You don’t need to add it to an allowlist. However, your security settings may unintentionally block these visits.” Signed requests carry a Signature-Agent header set to "https://chatgpt.com", with public keys at chatgpt.com/.well-known/http-message-signatures-directory, and OpenAI is explicit that “This guidance applies to signed cloud browser requests, not web crawlers.” Where Google tells you to “refer to your provider for the exact steps”, OpenAI names four providers that “verify ChatGPT request signatures automatically” and gives the label under which each lists the agent. The guide also answers the question security teams ask first: “across all our users, we require human review before any consequential action.” And it states the commercial stakes without hedging: “Sites that block access may lose users or customers who turn to other sources to make decisions.”
| Google-Agent | ChatGPT cloud browser (dots) | |
|---|---|---|
| User-Agent token | “Google-Agent” inside a Chrome-like string | Not the primary signal; OpenAI leans on the signature |
| Published IP ranges | Yes, user-triggered-agents.json | Not given in the agent access guide |
| Signature-Agent header | g="https://agent.bot.goog" | "https://chatgpt.com" |
| Key directory | agent.bot.goog/.well-known/http-message-signatures-directory | chatgpt.com/.well-known/http-message-signatures-directory |
| Signing status | Experimental; “We don’t sign every request of a particular agent” | Applies to “signed cloud browser requests, not web crawlers” |
| Vendor labels given | “Refer to your provider for the exact steps” | Akamai and HUMAN: “ChatGPT Agent”; Cloudflare: “dots, OpenAI” under Personal Agents, or detection ID 129220581; Vercel: chatgpt-operator |
| robots.txt | “Generally ignore robots.txt rules” (user-triggered fetcher) | Separate from OAI-SearchBot and OpenAI’s crawlers, which have their own controls |
| Human in the loop | Not addressed in the allow guide | “We require human review before any consequential action” |
The payment networks are converging on the same identity layer. Cloudflare, which maintains the Personal Agents directory both Google and OpenAI point to indirectly, has written that Visa’s Trusted Agent Protocol and Mastercard’s Agent Pay “leverage Web Bot Auth as the agent authentication layer”, and that both “introduce a new tag that agents must supply in the Signature-Input header, which indicates whether the agent is browsing or purchasing”. The practical upshot for a merchant is that one correctly built verification rule, signature first with IP and user agent as fallback, is what every serious agent operator now expects to meet at your edge. If you claimed your listing in the ChatGPT Business Hub last week, this is the next step on the same path.
How many sites are blocking agents? The numbers that exist
Nobody publishes a web-wide block rate for Google-Agent, and we are not going to invent one. Two sources give a real, if partial, picture. Cloudflare’s 1 July changelog introduced the Search, Agent and Training split and set the default that applies to every domain that has joined Cloudflare since 15 September 2026: “Bots classified as Training or as Agent are blocked on pages that display ads, while Search remains allowed.” Cloudflare’s 4 November post then gave site owners an Allow, Block on pages with ads, or Block choice per supported personal agent. In other words, a large share of newer sites block agents on monetised pages by default, and the override now exists but has to be chosen.
“Blocking agents altogether can feel like the safest choice, but it can mean losing customers. Better visibility into agent activity allows you to decide what access makes sense for your site.”
The second source is a field test. Blend Commerce, a Shopify conversion agency, sent an AI shopping agent to 200 retailers, 100 in the UK and 100 in the US across 20 sectors, over a week in September 2026 and scored each on whether the agent could access the site, find products, read them and get a basket to checkout. Its headline: “29 of the 200 wouldn’t let us in at all, even when we tried TWICE”, a list Blend says includes several household-name chains. Blend’s own conclusion is the one that matters for visibility: “if an agent can’t load your product page, it can’t recommend what’s on it. It DEFINITELY can’t buy it.” This is one vendor’s test with one agent, so treat the percentages as indicative, but the mechanism it exposes is exactly the one Google’s six checks address.
- 29 of 200Retailers that refused the AI shopping agent on both attemptsBlend Commerce
- 65 / 100Average agent-readiness score across all 200 retailersBlend Commerce
- 88 vs 49Highest-scoring sector (baby and nursery) against the lowest (fashion)Blend Commerce
- 1.2bnPeople OpenAI says use ChatGPT each week, per its 13 November announcementOpenAI
- Say their checkout cannot currently be completed by an AI shopping agent63%
- Have no 2026 budget for AI agent optimisation61%
- Have nobody who owns AI readiness internally57%
Share of 59 ecommerce leaders surveyed by Blend Commerce. A small, self-selected sample; shown for direction, not precision.
Source: Blend Commerce
The money side is early too, and the people closest to it say so. Mastercard’s chief executive told analysts on 29 October 2026 that agent-led transactions are growing fast from a small base, as reported by The Paypers. Digital Commerce 360 notes that not every business wants agent purchases, reporting that Amazon blocked Meta’s Muse agent in September, and that OpenAI “has yet to identify any specific retailers accepting ChatGPT Wallet transactions”. The decision to allow or block is yours. The point of this piece is that it should be a decision, not an accident of a default toggle.
“It is still early days, and the numbers are not yet significant in the grand scheme. But as the AI Agentic assistants have emerged, Agentic transactions are ramping up at impressive weekly growth rates.”
What should I do now? A checklist that follows Google’s guide
- Find the traffic you already have. Search 30 days of edge or server logs for the token “Google-Agent” and for a Signature-Agent header containing agent.bot.goog or chatgpt.com. Note the status codes those requests received. A run of 403s or challenge pages on product, search or API paths is the problem in plain sight. On Cloudflare, the new Personal Assistants widget in Security Analytics does this for supported agents.
- Decide, per agent, what you want. Google-Agent and ChatGPT’s cloud browser act for a named person who asked for something on your site. That is different from a training crawler, which is why Google’s own documentation treats AI training controls and agent access as separate questions. Most businesses that sell or take bookings online will want agents in; publishers who monetise with ads may choose Cloudflare’s “Block on pages with ads” and allow elsewhere.
- Write a verification rule, not a string match. Follow Google’s four steps: User-Agent contains Google-Agent, IP within user-triggered-agents.json, signature verified when Signature-Agent is g="https://agent.bot.goog", then skip bot challenges for verified requests. Mirror it for "https://chatgpt.com" using OpenAI’s key directory. If your CDN supports Web Bot Auth, use its built-in verified-agent entry instead of hand-rolling.
- Put the allow rule above everything that could override it. Google’s first troubleshooting check: ASN or data-centre blocks, managed bot rules and category-level “AI agent” policies must evaluate after your allow rule, not before.
- Skip challenges, not just blocks. Make the rule bypass interactive CAPTCHAs and JavaScript challenges, including CAPTCHA widgets embedded in site search and forms, which Google calls out specifically.
- Cover the whole journey. Scope the rule to on-site search, filters, detail pages, add-to-cart, booking flows and the XHR, GraphQL and api.* endpoints behind them. A homepage-only exemption is the most common half-fix.
- Fix rate limits for shared egress. Exempt the verified ranges from per-IP limits or raise thresholds for them, because many users’ sessions leave Google through the same pool.
- Automate the IP list. Pull user-triggered-agents.json on a schedule rather than copying prefixes once. The file carries a creation timestamp you can diff.
- Check your proxies preserve headers. Load balancers and origin shields must pass the original client IP, the User-Agent and the Signature, Signature-Input and Signature-Agent headers intact, or the origin can never verify what the edge saw.
- Do not touch Googlebot or Storebot access while you are in there. Google’s 11 November post is clear that blanket bot protection without verified-crawler exemptions drops pages and shopping listings out of Search. Review the exemptions for all three together.
- Re-test as an agent would. Load the homepage, run a search, filter, open a product, add to cart and hit checkout from a client that sends the agent headers, and read the status codes. Then put the test on a monthly schedule.
How do I measure whether agents can reach me, and whether it matters?
There is no Search Console report for Google-Agent, and Google’s guide does not promise one. Measurement therefore lives in three places you already own. Edge analytics: Cloudflare now classifies Personal Assistants separately and shows “the paths it requests, the status codes it receives, and the networks it comes from”; other CDNs expose verified-bot logs with the same fields. Server logs: filter on the Google-Agent token and on the Signature-Agent header, then chart requests and error rates by path over time. Commerce data: orders and bookings whose session carried agent headers, and declines your payment provider attributes to agent transactions, which OpenAI says it is working on with Forter, Riskified, Sardine, Signifyd, Adyen, Checkout.com and Stripe. Put the three next to your AI Mode and AI Overviews visibility, which the Generative AI performance report in Search Console covers, and you have a complete picture of what Google’s and OpenAI’s systems can see and do on your site. That is the view our reporting work builds for clients, and the one an answer engine optimisation programme should be judged against from here on.
One last caution, because the hype around agents is loud. Google’s guide is for “publishers that want to enable Google-Agent”. It is an option, not an obligation, and Google says nothing about rankings. The reason to act this week is narrower and more concrete: the defaults on much of the web now block agents on at least some pages, two of the largest agent operators have published the exact headers and keys they send, and the fix is a firewall rule you can write in an afternoon. Sites that get it right will be the ones an agent can actually buy from, book with, or quote. Sites that leave it to the defaults will find out in a year why their local and ecommerce enquiries went somewhere else.
Frequently asked questions
What is Google-Agent?
Google-Agent is the user agent Google uses for AI agents hosted on its infrastructure that “navigate the web and perform actions upon user request”. Google added it to its crawler documentation on 20 March 2026 and classes it as a user-triggered fetcher, meaning a person asked for the visit. On 17 November 2026 Google published a guide, “Allow Google-Agent traffic”, on letting it through CDNs, firewalls and bot-detection services.
Does robots.txt block Google-Agent?
No. Google’s documentation says user-triggered fetchers “generally ignore robots.txt rules” because the fetch was requested by a user. Access to Google-Agent is decided by your CDN, web application firewall or bot-detection service, not by robots.txt. Blocking it there is possible; so is accidentally blocking it with a broad bot rule, which is what Google’s new guide is written to prevent.
How do I verify a request is really from Google-Agent?
Google gives three signals: the User-Agent header contains “Google-Agent”, the IP address falls inside the ranges published in user-triggered-agents.json, and, for a subset of requests, a Web Bot Auth signature with Signature-Agent set to g="https://agent.bot.goog" that you verify against the keys at agent.bot.goog. Google says the user agent string can be spoofed and that it does not yet sign every request, so use all three.
Should I allow Google-Agent on my website?
Google frames it as a choice for “publishers that want to enable Google-Agent”. If people use Google to research, compare, book or buy what you sell, an agent that cannot load your pages cannot act on them. Publishers who rely on ad revenue may prefer Cloudflare-style “block on pages with ads” for agents and allow elsewhere. The point is to decide deliberately rather than inherit a default.
Why does Google-Agent still get blocked after I allowed it?
Google lists six reasons: your allow rule sits below broad ASN, managed-bot or AI-agent category rules; it skips HTTP blocks but not CAPTCHAs or JavaScript challenges; it covers the homepage but not search, filters, add-to-cart or API endpoints; per-IP rate limits trip on shared egress pools; your IP list is out of date; or a proxy strips the client IP, User-Agent or signature headers.
Is allowing Google-Agent the same as allowing ChatGPT’s agent?
Same mechanism, different identifiers. Both sign requests with Web Bot Auth (RFC 9421). Google-Agent uses Signature-Agent g="https://agent.bot.goog" and publishes IP ranges; ChatGPT’s cloud browser uses "https://chatgpt.com" and keys at chatgpt.com. OpenAI says Akamai, Cloudflare, HUMAN and Vercel verify its signatures automatically; Google says to ask your provider. Allow each explicitly.
Sources
- Allow Google-Agent traffic — Google Crawling Infrastructure documentation, 17 November 2026
- Google’s Crawling Documentation Changelog (entries for 17 November, 4 May and 20 March 2026) — Google Crawling Infrastructure documentation, 17 November 2026
- List of Google user-triggered fetchers — Google Crawling Infrastructure documentation
- Authenticate requests with Web Bot Auth (experimental) — Google Crawling Infrastructure documentation
- user-triggered-agents.json (IP ranges for Google-Agent) — Google, 3 March 2026
- Moving to a CDN without overwhelming your origin (or blocking Googlebot) — Google Search Central Blog, 11 November 2026
- Optimizing your website for generative AI features on Google Search — Google Search Central documentation, 10 July 2026
- Google Search Help Document On How To Allow Google-Agent Traffic — Search Engine Roundtable, 17 November 2026
- Google Tests Sign In Protections After First Search To Block Scrapers — Search Engine Roundtable, 17 November 2026
- Google’s releasing Google-Agent: Here’s what to know — Semrush, 26 March 2026
- Introducing ChatGPT Wallet — OpenAI, 13 November 2026
- Help ensure cloud browser access to your site — OpenAI Developers (Agentic Commerce)
- See the agents, set the terms: new visibility and control for personal agents — Cloudflare Blog, 4 November 2026
- New options to manage AI traffic — Cloudflare Changelog, 1 July 2026
- Securing agentic commerce: helping AI Agents transact with Visa and Mastercard — Cloudflare Blog
- AI Shopping Audit 2026: How 200 UK and US Retailers Score — Blend Commerce
- Google Adds Home Service Booking & Quote Links To AI Mode — Search Engine Journal, 4 November 2026
- OpenAI launches ChatGPT Wallet for AI purchases through retailer checkouts — Digital Commerce 360, 16 November 2026
- OpenAI launches ChatGPT Wallet for agentic payments in the US — The Paypers, 16 November 2026
- Google-Agent identity page (Web Bot Auth) — Google (agent.bot.goog)


