Monitoring

Testing asks whether it works. Monitoring asks whether it still does.

Scheduled checks and classified real agent traffic, per agent family, with alerts that fire on a trend rather than on a quiet Sunday.

Scheduled checks: ≥ 60s · never backfilled
Classifier: classifier-v2 · 5 families
Alerts: score · conversion · streak
Compared with: 7-day mean

The need

A passing gate says the code you shipped was fine. It says nothing about the rule someone changed at the edge this morning.

Most of what breaks agent journeys in production never goes through your pipeline: a bot-management rule, a CDN configuration, a third-party script, a payment provider tightening a check. Your deploys stay green and your agent traffic quietly stops converting.

Monitoring closes that gap from two directions — synthetic journeys you control, and the real agent traffic already arriving at your origin.

Agent traffic is already there. Most merchants cannot say how much, or what it does.
Assistants and crawlers get counted together, so the ones that would have bought disappear into a scraping number.
A conversion drop for one agent family is invisible in an aggregate that humans dominate.

Uptime monitoring proves your site answers. It cannot tell you that an assistant reached checkout and was turned away.

How it works

Two sources, one funnel.

Scheduled checks
The same test plans your release check uses, re-run on an interval against production. Sixty seconds is the floor, and a missed slot is skipped rather than backfilled — a scheduler down for six hours must not enqueue six hours of runs when it returns.
Classified traffic
Batched events are classified into assistant_agent, declared_crawler, protocol_client, unknown_automation or human, reading signed identity first, then a declared user agent, then behaviour — in descending order of how much each can be trusted.
Funnels per family
Daily stage-by-stage conversion for each agent family, so a drop that only affects assistants is visible instead of being averaged into human traffic.
Alerts on a trend
Score drop, conversion drop and scheduled-check failure streaks. Conversion is measured against a seven-day mean rather than yesterday, and repeats are suppressed for a cooldown.

Checkout success rate · assistant-style agents7d

94%Mon

91%Tue

67%Wed

Fell from 94% to 67% after release 9be77d1

Checkout conversion for assistant-style agents, by day. The alert names the deploy, not just the number.

What it brings

The first person to notice should be you, not a customer's assistant.

Assistants counted separately from crawlers
An assistant may transact; a crawler will not. Merging them hides the exact conversion problem worth paying to fix.
Alerts that survive a quiet weekend
Conversion is compared to a seven-day mean, and days with no resolved outcomes are excluded — so a shipper that stops sending an outcome field does not look like a collapse.
A regression tied to a deploy
When a scheduled check's score drops, the alert names the release it followed, which is the difference between an incident and a number.
Each project, its own channels
Every project sends alerts to its own Slack webhook and up to ten email addresses, so staging noise never lands in the production channel. A rule can instead page PagerDuty or post to a signed webhook of its own. Alerts are recorded and queryable whether or not delivery is configured, and a delivery failure never fails the evaluation.

Boundaries

What it does not do.

Stated here so you do not have to discover it in a pilot.

Classification is a judgement, not an identity check.
A signed agent identity is trustworthy; a declared user agent is a claim anyone can copy. Confidence travels with every classification, and the rules are versioned (classifier-v2 today) so history stays comparable.
Ingest needs a feed.
Funnels from real traffic require events from your edge, CDN or origin. Without them, monitoring still runs scheduled checks — but it only sees the journeys it performs itself.
Scheduled checks cost what a test costs.
A five-minute interval is twelve journeys an hour against production. The floor is sixty seconds, and the sensible setting is usually much slower.

See it against your own site. A scan takes about ninety seconds and needs no account.