FocusWeb AI Crawler Monitor is an observation-only plugin that helps you understand how AI crawlers interact with your WordPress site. It never blocks or alters requests — it simply logs matching visits and gives you a dashboard to review them.
Detected bots (built in, configurable):
- GPTBot (OpenAI)
- ChatGPT-User (OpenAI, on-demand)
- ClaudeBot (Anthropic)
- Claude-Web (Anthropic)
- PerplexityBot
- Google-Extended
- CCBot (Common Crawl)
- Bytespider (ByteDance)
- Amazonbot
- Applebot-Extended
Features:
- Lightweight
init-hook based detection, no server log file access required. - Custom database table with indexes for fast dashboard queries.
- Dashboard with summary cards, a Chart.js time-series chart, a most-crawled URL table, and a paginated raw log with CSV export.
- Configurable log retention with a daily cleanup cron job.
- Optional IP anonymization (masks the last octet/segment) for GDPR-friendly logging, enabled by default.
- Excludable URL path patterns (e.g.
/wp-admin/*,/checkout/*). - AI Referral Traffic tracking: detects human click-throughs from AI answer engines (ChatGPT, Perplexity, Microsoft Copilot, Google Gemini, Claude, Meta AI, You.com) via the Referer header or matching utm_source parameters, and captures the source query when the referring platform exposes one.
- Built-in llms.txt generator: produces suggested content from your site title, tagline, pages, and posts for manual placement at your site root.
- Fully functional on shared/managed hosting — no filesystem writes outside the database, no special server permissions needed.