Monitoring AI shopping agents in production with tragent

A green CI build says the code you shipped this morning works for AI agents. It says nothing about the bot-management rule a security engineer tightened at lunch, the CDN setting that changed overnight, or the payment provider that started challenging unfamiliar traffic last week. None of those go through your pipeline, and any of them can stop agents buying.

That is the gap tragent's monitoring closes. This post explains how it works: where the data comes from, how traffic is classified, what the funnels show, and when an alert fires.

Why do AI shopping agents need monitoring, not just testing?#

Because most of what breaks agent journeys in production changes outside your code. A release gate checks a deploy before it ships. It cannot see changes that never go through a deploy.

The usual culprits are configuration:

• Bot management. A new rule after a card-testing attack starts challenging automated requests at checkout.
• The CDN and edge. A caching or redirect change alters what agents receive.
• Third-party scripts. A consent banner, chat widget or fraud script adds a step an agent cannot get past.
• Payment providers. A tightened risk check declines sessions without a browser history.

Uptime monitoring will not catch any of these. It proves your site answers; it cannot tell you that an assistant reached checkout and was turned away. Our guide to testing your store with AI shopping agents in CI covers the pipeline half. This post is about the other half.

What does tragent monitor?#

Two sources feed one picture: scheduled checks you control, and the real agent traffic already arriving at your store.

Scheduled checks are synthetic: you know exactly what was attempted, so a failure is unambiguous. Traffic is real: it shows what agents actually do, including journeys you never thought to test. Each covers the other's blind spot.

How do scheduled checks work?#

The same test plans your release check uses, re-run against production on an interval of your choice, from 60 seconds upwards. A failing run is reported like any other run, with the stage that failed and the evidence.

Two design choices keep scheduled checks trustworthy:

• Missed slots are skipped, not backfilled. If the scheduler is down for six hours, it does not come back and run six hours of checks at once. At least one full interval always separates two runs.
• Several workers never double-fire. Claiming a due check and moving its next run time happen in one database statement, so running more workers adds capacity, not duplicate runs.

Scheduled-check history has its own retention, kept separate from the runs a release gate's baseline depends on. Pruning months-old monitoring runs can never delete the reference a deploy is compared against.

How does tragent see real agent traffic?#

From your edge or CDN logs, through a ready-made connector, or by posting events to the ingest API. Most agents never run your analytics script, so the edge is the only place they all appear.

There are four connectors, each configured with a per-project token that is shown once and stored only as a hash:

• Cloudflare Workers. A small Worker on your store's route passes every request through unchanged and ships a record of it after the response is sent. It cannot change or delay a response: shipping happens after the fact, and a failure is swallowed. Static assets are skipped, and cookies and authorisation headers are never sent.
• Cloudflare Logpush. Point a Logpush job for HTTP requests at your token's URL. Add the Web Bot Auth signature headers as custom fields and tragent can verify signed agents from the logs.
• Vercel log drains. Request entries are kept and function output is ignored. Drain signatures are verified.
• Fastly. Real-time log streaming to your token's URL.

If your stack is something else, any log shipper, SDK or proxy can post batches of up to 1,000 events to /v1/ingest/events. A malformed row is rejected on its own with its reason, rather than failing the batch, so one bad record never costs a shipper its buffer.

How is agent traffic classified?#

Every request is assigned to one of five families, using the strongest evidence available. The classifier is versioned, currently classifier-v2, and each event records the version that judged it, so history stays comparable when the rules change.

Evidence is read in descending order of trust: a verified signature first (Web Bot Auth), then a declared user agent, then behaviour. Confidence reflects which tier answered. Two rules matter in practice. A signature that fails to verify scores lower than a request that claimed nothing, because someone tried to borrow an identity they cannot prove. And an agent operator's key directory being unreachable is treated as an outage, not a forgery. Our explainer on Web Bot Auth covers why.

Assistants are deliberately never merged with crawlers. An assistant may complete a purchase and a crawler never will; counting them together averages away exactly the conversion problem you need to see.

What do the funnels show?#

Daily, stage-by-stage conversion for each agent family, in the order of the nine stages, so a drop that only affects assistant agents is not drowned in human traffic.

Funnels are rebuilt every 15 minutes, once when a worker starts, and on demand. Late-arriving events are handled by recomputing the last two days, so a slow log shipper does not leave a hole in yesterday. And days with no recorded outcomes are left out of baselines entirely: a shipper that silently stops sending outcomes would otherwise look exactly like conversion collapsing to zero.

conversion_drop · alert

Reach checkout down 37% against its 7-day mean

Worst family: assistant agents. Sent to Slack and email; repeats held for an hour.

Which alerts fire, and when?#

Four kinds of rule, each comparing against a baseline that does not flinch at normal variation.

The seven-day mean is the detail that makes these livable. Comparing with yesterday would page someone every Monday morning, when weekend traffic gives way to weekday traffic. Each alert also has a deduplication key and a one-hour cooldown by default, so one incident is one alert, not one per pass.

Alerts go to the Slack channel and email addresses you set for each project. Only Slack incoming-webhook URLs are accepted, and a stored webhook is never shown again in the API. Delivery is optional: every alert is recorded and visible in the dashboard either way, and a failed delivery never interrupts the monitoring pass that raised it.

What does a drop actually cost?#

Once enough agent traffic is flowing, tragent prices failures from your own funnel instead of from assumptions. Measured revenue at risk uses your agent sessions, the stage each one reached, and the orders that resulted.

It refuses to produce a figure from too little data: below 200 agent sessions, with no recorded outcomes, or with no order value, the measured figure is withheld rather than shown as zero, because a partial feed looks exactly like a broken store. Until then, reports use an estimate and label it as one. Our post on how much revenue you lose when AI agents can't buy explains the model and why each lost order is counted only once.

What happens to the data?#

Only what monitoring needs is collected, and raw events are short-lived.

• No credentials. The Cloudflare Worker drops cookies and authorisation headers before anything leaves your edge.
• No addresses in sessions. When a log has no session id, requests are grouped into a visit by client, agent and day, and the resulting id is hashed, so it carries no IP address.
• Short retention. Raw events are kept for 30 days by default (seven at minimum) and then deleted. The daily funnel aggregates built from them are kept.

How does monitoring fit with testing?#

Testing guards each release; monitoring guards everything in between.

1. Before a deploy, the CI gate runs your test plans against staging and fails the build on a regression.
2. After a deploy, scheduled checks run the same plans against production on an interval.
3. All the time, real agent traffic shows what agents are doing and alerts you when a family's conversion falls away from its baseline.

The first person to notice an agent being turned away at checkout should be you, not a customer's assistant. See the monitoring product page for what is included, or our guide to measuring AI agent traffic for doing it with your own tools.

Frequently asked questions

What is AI agent monitoring?

Watching whether AI shopping agents can still complete purchases on a live store, using scheduled checks against production and classified real agent traffic, with alerts when a stage's conversion falls.

How does tragent get my agent traffic?

Through a Cloudflare Worker, Cloudflare Logpush, a Vercel log drain or Fastly real-time logging, each with a per-project token, or by posting events to the ingest API from any log shipper.

Will the Cloudflare Worker slow my store down?

No. It passes each request through unchanged and ships the record after the response is sent. A shipping failure is ignored, and static assets are skipped.

How does tragent tell an AI agent from a person or a scraper?

It reads a verified Web Bot Auth signature first, then a declared user agent, then behaviour. A signature that fails to verify counts against the request, not for it.

Why compare conversion with a seven-day mean?

Because comparing with yesterday flags every normal weekly swing, like weekend to Monday. A seven-day mean keeps alerts for real drops.

Sources

1. Logpush — Cloudflare docs
2. Cloudflare Workers — Cloudflare docs
3. Log drains — Vercel docs
4. Real-time log streaming — Fastly docs
5. HTTP Message Signatures for Bots — IETF Web Bot Auth working group

Written by the tragent team and checked against the sources listed above on October 19, 2026. See how we write and correct posts in our editorial policy.