Bots and web crawlers coming to your site check the content of the full user agent string, and those that are not on the whitelist will be blocked. AI crawlers are also blocked.
In addition, you can manually blacklist the IP address. If you enable the automatic IP blocking option, IP addresses that make more than 5 requests in 1 minute will be blocked for 240 minutes (4 hours) by default. You can change these values here.
Blocked bots include:
- GPTBot
- ClaudeBot
- Amazonbot
- SemrushBot
- MJ12bot
- and more bots…