CDN and server logs, read locally
See the traffic analytics cannot: AI agents, private shares, unmeasured readers.
agenttraffic reads the logs your CDN already writes and reports which AI agents fetched your pages, verified against the IP ranges their operators publish, plus the link previews that reveal private sharing and the visitors your analytics script never counted. Open source, local, no telemetry.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ EDGE · EXAMPLE.COM 7 days ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2026-08-18T04:00:00.000Z → 2026-08-25T04:00:00.000Z (America/New_York) IP ranges shipped, 5 days old AI CRAWLERS ··········································································· ChatGPT-User OpenAI live-fetch 90 14p 20 spoofed, 10… PerplexityBot Perplexity search 10 12p 30 unchecked live-fetch = a person just asked an assistant about the page primary number = requests verified against published provider IP ranges unchecked = no range data (rDNS-only provider) or no usable client IP 1 training / ops crawler hidden (--show-training) SEARCH CRAWLERS ······································································· Googlebot 30 5 spoofed, 25 unche… SPOOFED UA MATCHES recognised bot UA, IP outside the published range ················· ChatGPT-User 20 ip-range Googlebot 5 ip-range PAGES ASSISTANTS FETCHED live-fetch + search ········································· / 35 /app 27 MEASURED AT THE EDGE ·································································· requests 1200 browser requests 800 500 page fetches, 210 distinct IPs bot ua matches 300 130 verified spoofed 25 bot UA, IP outside published range unclassified 45 neither browser- nor bot-shaped cache hit 75.0% 4xx / 5xx 110 / 5 scanner probes 60 PHP/CMS probes, all 4xx asset probes 9 nested image 404s, CMS logo scans devtools opened 3 distinct IPs (7 source-map 404s) REFERRERS external, browser page fetches ············································· news.ycombinator.com 40 www.google.com 40 t.co 10 350 no referrer · 55 internal navigation · 5 unparseable of the no-referrer fetches, 100 (29%) are shared links SHARED LINKS ?preset= page fetches, not shares ······································ https://example.com/app?preset=abc123 70 https://example.com/app?preset=def456 50 LINK PREVIEWS links pasted into chat apps (floor) ···································· Facebook / iMessage 30 12 urls Slack 15 6 urls Discord 5 3 urls BROKEN LINKS 404s, excluding probes ·················································· /old-page 12 /feed 4 OTHER BOTS unrecognised agents, worth a look ········································· AhrefsBot 25 python-requests 5 UNCLASSIFIED neither browser- nor bot-shaped ········································· curl 30 Go-http-client 15 excluded from the split: UptimeRobot 30 NOTES ················································································· Perplexity has been documented crawling with undeclared user agents; verified count is a floor. iMessage unfurls arrive as facebookexternalhit and Twitterbot; they are counted under Facebook / iMessage. Gemini training traffic arrives as Googlebot and is inside this row. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Log sources
Reads the logs your stack already writes
No agent to install and nothing to change at the edge. Point it at the files you already have.
- CloudFrontstandard logs, v2 and legacy
- S3fetch a window straight from the bucket
- NGINXcombined format
- Apachecombined format
- Any CLF logcommon or combined, gzip too
- CloudflaresoonLogpush
- VercelsoonLog Drains
- NetlifysoonLog Drains
- Fastlysoonreal-time logging
- AkamaisoonDataStream
- CaddysoonJSON access logs
The gap
Analytics counts browsers that run its script. Logs count everything.
AI assistants, crawlers, and link unfurlers never execute JavaScript, and a growing share of readers block it. The edge log is the only record that sees them all.
Analytics script sees
- Browsers that loaded the tag
- Referrers the browser chose to send
- Sessions, if consent was given
- Nothing from a bot, an assistant, or an unfurler
The edge log sees
- Every request: ChatGPT-User, Claude, Perplexity, Googlebot
- Which agents are real, verified against published IP ranges
- Link previews from Slack, iMessage and Discord: private shares
- Readers who blocked the script, scanners, broken links, cache hits
How it works
Three steps, nothing installed, nothing sent
STEP 01
Point it at your logs
CloudFront standard logs (v2 or legacy), nginx or Apache combined format. Files, directories, globs, gzip. Or fetch straight from S3.
npx agenttraffic ./logs --days 7STEP 02
Every bot UA gets checked
Each recognised agent is matched against the IP ranges its operator publishes. Verified, spoofed and unchecked are never added together.
ChatGPT-User 90 verified · 20 spoofedSTEP 03
Read the report
One screen: which agents fetched what, where links were shared, and what your analytics script never counted. JSON output for anything else.
npx agenttraffic ./logs --json
01
Floors, not totals
Every count that can be spoofed, stealth-crawled or fanned out is labelled a floor. The report says "fetch events", never "people asked".
02
Verified, unverified, spoofed
A bot UA is checked against the IP ranges its operator publishes. The three outcomes stay separate; a range that was never published is not the same as a spoof.
03
Local by default
Client IPs are used in memory for verification and distinct estimates and never written out. Network access only in the commands that ask for it.
Registry
Which agents read your site
One page per user agent: the exact UA strings, how to verify it, and what its hits mean.
- agents in the registry
- 54
- operators
- 33
- with published IP ranges
- 15
- IP range snapshot
- 2026-08-28
Guides
Telling the agents apart
GPTBot vs ChatGPT-User vs OAI-SearchBotThree OpenAI user agents, three different meanings. Which one is a reader, which is the index, which is training, and what to block.All agentsEvery user agent, by operator and kind.
By kind: Live fetch, AI search, Training, Search engine, Operational, Link unfurl
FAQ
Straight answers
How do I know if ChatGPT, Claude or Perplexity is reading my site?
Run npx agenttraffic on your CDN or server logs. The report lists every recognised AI agent by name (ChatGPT-User, Claude-User, PerplexityBot and the rest), how many requests each made, which pages they fetched, and whether the requests came from the IP ranges the operator publishes. Analytics scripts cannot see any of this because assistants never run JavaScript.
What does GPTBot in my logs mean, and how is it different from ChatGPT-User?
GPTBot is OpenAI's training crawler; ChatGPT-User is the live fetch made when a person asks ChatGPT about a page; OAI-SearchBot builds the search index behind ChatGPT search. They are three different user agents with three different meanings for your site, and blocking one does not block the others. Read the comparison.
Is a bot user agent in my logs actually that bot?
Not necessarily. Anyone can send any user agent string. agenttraffic checks each bot request against the IP ranges its operator publishes and keeps three outcomes separate: verified (inside the range), spoofed (range published, IP outside it) and unverified (the operator publishes no range). Requests with no usable client IP are marked not checked and say why.
Which log formats does it read?
CloudFront standard logs (v2 with the header line parsed, and legacy), plus common and combined log format from nginx, Apache or anything else that writes CLF. Files, directories and globs, gzip read transparently, and a fetch command that syncs a date window straight from an S3 bucket. Other CDNs and hosts are planned.
Is any of my data sent anywhere?
No. Reports are computed locally and there is no telemetry. Client IPs are used in memory for verification and distinct-visitor estimates and are never written to output. Only two commands touch the network, and only when you run them: update, which fetches fresh IP ranges, and fetch, which downloads logs from your own bucket.
Why does the report say "fetch events" and "floor" instead of visitors?
Because the log cannot prove a person. A single question to an assistant can fan out into several fetches, some crawlers use undeclared user agents, and spoofed requests exist. Every number that could be inflated or under-counted is labelled a floor, and the report says what was measured rather than what it might imply.
How is this different from Google Analytics or Plausible?
Those tools count browsers that load their script and send an event. Logs count every request the edge answered: AI assistants, search crawlers, link unfurlers from Slack and iMessage, readers with script blockers, scanners and broken links. agenttraffic is not a replacement for a web analytics tool; it reports the traffic that tool structurally cannot see.
Can I get the data as JSON?
Yes. Pass --json (or --json=report.json) to write the versioned schema v1 report instead of the terminal view. The schema ships in @agenttraffic/core, so the output is stable to build on. See the docs.
Roadmap
Weekly: which agents read your site
A hosted weekly report is on the roadmap. Leave an email to hear when it exists; nothing else is sent.