Web Bot Auth explained: how signed AI agents prove identity

For thirty years, a bot announced itself with a line of text: its user agent. Anyone could type that line, so sites learned to distrust it and fell back on IP address lists, which break whenever an operator adds a data centre. Neither works for AI agents, which run in shared cloud infrastructure and act for millions of different people.

Web Bot Auth replaces both with cryptography. This post explains what a signed request contains, how a site verifies it step by step, what the result does and does not prove, and what a store needs to do about it.

What is Web Bot Auth?#

Web Bot Auth is a way for automated clients, including AI agents, to sign their HTTP requests so a website can verify who operates them. It is a profile of HTTP Message Signatures, the IETF standard published as RFC 9421 in 2024, plus a convention for publishing the keys.

The protocol document, HTTP Message Signatures for Bots, was adopted by the IETF's Web Bot Auth working group and published as a Standards Track Internet-Draft in September 2026, authored by engineers from Cloudflare and Google. It is not yet an RFC, so details can still change, but the core has been stable through several drafts and is already deployed: OpenAI signs its agent's traffic, and Cloudflare, Akamai and DataDome verify signatures.

Why isn't a user agent or an IP allowlist enough?#

A user agent can be copied by anyone, and IP lists cannot describe agents that run on shared clouds for many different users. Both were built for a world of a few large crawlers.

• User agents are free text. A scraper that wants to be treated like a well-known agent sends that agent's user agent. Rules that trust the string reward the forgery.
• IP lists go stale. Operators add regions and providers. Every published range is a list someone must keep updated on both sides, and a cloud IP is shared with thousands of other customers.
• Agents act for many people. Cloudflare's July 2026 classification change put this in words: a verified bot can be operated directly by one company, or by an intermediary, an agentic service that a wide range of end users can operate. You may trust the operator without trusting every person driving it.

A signature answers the first two problems outright. It cannot be produced without the operator's private key, and it does not depend on where the request came from. The third problem it makes manageable: you know which operator to hold accountable.

What does a signed request look like?#

An ordinary HTTP request with three extra headers: who signed it, what was signed, and the signature. Here is a signed request for a product page:

GET /products/trail-shoe HTTP/1.1
Host: shop.example
User-Agent: Mozilla/5.0 (compatible; ExampleAgent/1.0)
Signature-Agent: sig1="https://agent.example"
Signature-Input: sig1=("@authority" "signature-agent";key="sig1");created=1760000000;expires=1760000300;keyid="poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U";tag="web-bot-auth"
Signature: sig1=:uz2SAv+VIemw+Oo890bhYh6Xf5qZdLUgv6/PbiQfCFXcX/vt1A8Pf7OcgL2yUDgYLvZlJ3vAgFsCSY/ztP4pDg==:

The key id and signature here are illustrative values, not a real operator's.

• Signature-Agent names the operator, as a URL. The label sig1 ties it to the signature below. A site will fetch the operator's keys from this origin.
• Signature-Input lists the covered components, the parts of the request the signature protects, and its parameters: when it was created, when it expires, which key signed it (keyid), and a tag of web-bot-auth so the signature cannot be mistaken for another protocol's.
• Signature is the signature itself, base64-encoded between colons.

The draft requires a signature to cover at least the target (@authority or @target-uri) and the Signature-Agent value, and to carry created, expires, keyid and the tag. It recommends expiry within 24 hours; in practice operators use minutes.

How does a website verify the signature?#

It fetches the operator's public key, rebuilds exactly what was signed, and checks the signature against it. Step through it below. The values are illustrative.

1GET /products/trail-shoe HTTP/1.12Host: shop.example3Signature-Agent: sig1="https://agent.example"4Signature-Input: sig1=("@authority" "signature-agent";key="sig1")5;created=1760000000;expires=1760000300;keyid="poqkLGiy…";tag="web-bot-auth"6Signature: sig1=:uz2SAv+VIemw…:7 8"@authority": shop.example9"signature-agent";key="sig1": "https://agent.example"10"@signature-params": ("@authority" "signature-agent";key="sig1");created=…11 12GET https://agent.example/.well-known/http-message-signatures-directory13{14"keys": [{15  "kty": "OKP", "crv": "Ed25519",16  "kid": "poqkLGiy…",17  "x": "JrQLj5P_89iXES9…",18  "use": "sig"19}]20}

The verifier's job is mechanical, which is why bot managers can do it for every request. The draft also asks verifiers to bound the directory fetch: limit its size and the number of keys, time out, refuse private and loopback addresses, and cache failures for no more than five minutes so a directory outage does not lock an operator out for long.

What does the key directory look like?#

A JSON Web Key Set served at a fixed path on the operator's domain. The path is /.well-known/http-message-signatures-directory, the media type is application/http-message-signatures-directory+json, and it is cacheable like any HTTP response.

{
  "keys": [
    {
      "kty": "OKP",
      "crv": "Ed25519",
      "kid": "poqkLGiymh_W0uP6PZFw-dvez3QJT5SolqXBCW38r0U",
      "x": "JrQLj5P_89iXES9-vFgrIy29clF9CC_oPPsw3c5D0bs",
      "use": "sig"
    }
  ]
}

Each key's id is its JWK thumbprint: a SHA-256 hash of the key's own fields. That is what lets a signature name a key unambiguously, and lets an operator rotate keys by publishing the new one alongside the old before switching. The current draft also allows two other discovery types, a direct jwks_uri and a client ID metadata document, chosen with a type parameter on Signature-Agent; the directory is the default.

What does a valid signature prove, and what doesn't it?#

It proves which operator sent the request, that the covered parts were not altered, and that it is recent. It does not prove the shopper meant it.

A valid signature tells you:

• The request was signed by whoever holds the private key for that operator's published key.
• The covered components, at minimum the host and the Signature-Agent value, are what the operator signed.
• The request falls inside the signature's validity window.

It does not tell you:

• Who the end user is. An intermediary's agent acts for many people. Their identity, if you need it, comes from sign-in or a payment credential.
• That the purchase is authorised. That is what payment mandates such as AP2 and delegated payment tokens are for.
• That the request cannot be replayed. A signature is valid until it expires. Short expiry, plus covering the method and target, narrows what a replay could do.

Two outcomes deserve different treatment. A signature that fails to verify is worse evidence than no signature at all: someone tried to claim an identity they cannot prove. A directory that cannot be reached is not a forgery; it is an outage, and treating it as an attack would punish an operator for its hosting provider's bad afternoon.

Does a store need to verify signatures itself?#

Usually not. Your CDN or bot manager verifies them; your job is to allow the verified agents you want. Verification is built into Cloudflare's verified bots and signed agents, DataDome's bot authentication, and Akamai's agent handling, according to OpenAI's allowlisting guidance and the vendors' own documentation.

What a store does need to decide:

1. Which operators to admit, and on which paths. Shopping agents need product pages, search, the basket and checkout.
2. Where the allow rule sits. It must run before the rules that challenge automation, or the challenge wins. Our post on why AI shopping agents get a 403 at your checkout has the steps for each bot manager.
3. That nothing else trusts the user agent. If an unsigned request claiming to be a known agent gets further than a signed one, a rule somewhere is matching strings (an unproven copycat is let in).

If you run your own edge and want to verify signatures yourself, use a maintained library rather than writing the canonicalisation by hand: Cloudflare publishes a reference implementation as the web-bot-auth npm package, and Go has github.com/yaronf/httpsign, which tragent uses. Cache the directory, bound the fetch, and do not follow redirects.

How does tragent use Web Bot Auth?#

tragent's agents sign their requests, and its checks compare how a store treats a signed agent with how it treats an unsigned one. That comparison is the point: it separates "your bot manager refuses agents that prove who they are" from "your bot manager refuses anonymous automation", which are very different findings.

tragent signs with Ed25519 keys through yaronf/httpsign, checks its output against Cloudflare's reference verifier in CI, and publishes its public keys at tragent.io/.well-known/http-message-signatures-directory, where anyone can fetch them. Requests with a body cover the method, target and body digest. When a store refuses a signed request, the report includes the request exactly as sent, signature headers intact and credentials redacted, so the store's security team can verify it against the directory themselves.

Written by the tragent team and checked against the sources listed above on October 19, 2026. See how we write and correct posts in our editorial policy.