Issue reference
robots.txt blocks every agent
- ROBOTS_BLOCKS_EVERYONE
- Find products
- —
- High
What it means
tragent reports this at the “Find products” stage. The question there is “Can an agent find your products at all?” An agent has to reach your site and locate products by searching or browsing before anything else can happen.
What it does to an agent
Most agents fail at this point. The few that find a way round it spend more time and model calls doing so, and may give up before checkout.
How to fix it
A blanket Disallow hides your catalogue from AI shopping surfaces. Allow product and search paths for the agents you want to sell through.
Check the fix
Run the check that reported ROBOTS_BLOCKS_EVERYONE again after the change. The scan needs no account; an issue found during an AI shopping test needs that test run again.
npx @tragent/cli scan https://your-store.com