How we measure stores

Which stores

Each study scans a list of publicly reachable online stores, chosen to cover the major commerce platforms and a spread of retail categories. The list for each study is fixed before the scan starts, and the study states how many stores it includes. We identify a store’s platform from what its pages and response headers publicly show.

The lists are not random samples of all online stores. They lean towards established stores with a real catalogue and checkout, because those are the stores agents are sent to buy from. Read the results as a picture of that population, not of every website that sells something.

What we check

• Deterministic checks only. We read public pages and files the way an agent would: robots.txt, sitemaps, product pages and their structured data, published agent profiles (such as UCP), and the purchase path up to the basket and checkout. No purchase is attempted and no account is created.
• Two identities. Each stage of the purchase path is requested twice: once by tragent’s agent signing with Web Bot Auth (RFC 9421 HTTP Message Signatures), and once unsigned. That is how we can say whether a store refuses an agent that proves who it is, and not only one that does not.
• The same scoring as the product. Readiness scores use tragent’s published scoring (readiness-v2), so a number in a study means what it means in a report.

How we treat the stores we scan

• robots.txt is honoured. A store that asks automated clients not to fetch is skipped and counted as declined, not scanned.
• Requests are spaced. Stores are scanned one after another with a delay between them, so no store sees a burst. Each check fetches a page once per identity.
• Requests identify themselves. They come from TragentBot, whose page explains what it does and how to block it.

What we publish

Aggregates only. We never name a store, publish a store’s individual result, or show a breakdown small enough to identify one: any platform or category with fewer than 12 stores is suppressed rather than shown. Stores that could not be reached are counted separately and excluded from percentages.

Limits of the data

• A scan is a snapshot. Bot rules and storefronts change weekly; a store blocked on the day of the scan may not be blocked now, and the reverse.
• We see what a request from our infrastructure sees. Some bot managers treat traffic differently by country, network or time of day.
• Deterministic checks show whether an agent is admitted and can read what it needs. They do not complete a purchase, so they cannot prove that one would succeed end to end.
• Platform labels come from public signals and can be wrong for heavily customised stores.

Questions and corrections

If you run a store and want to know whether it was included, or think a figure is wrong, write to [email protected].