TechDex Search Scrape Guard
TechDex Search Scrape Guard protects the public WordPress search feature from spam, automated abuse, deep pagination, and other excessive search activity. It blocks bots and abusive requests that can consume database and server resources, slow down your site, or use your search results to scrape and rip content from it.
I built Search Scrape Guard after a client’s live WordPress site was hammered with thousands of automated searches in a short period of time. The traffic appeared designed to scrape and rip content, and it was consuming database and server resources as a result.
This plugin is designed to stop that activity before WordPress performs the resource-intensive work behind a search. Normal searches continue working within the limits you configure.
Search Protection
Search Scrape Guard can:
- Reject empty and whitespace-only searches
- Limit search length and word count
- Limit how deeply visitors can paginate through search results
- Set different pagination limits for anonymous and logged-in users
- Block known automation signatures
- Rate-limit repeated searches
- Optionally require Cloudflare Turnstile verification
Rejected searches receive lightweight responses without loading the complete WordPress theme.
The rate limiter uses a fixed 256-row database table instead of creating a new transient or cache entry for every IP address. This prevents the rate-limiting storage from becoming another source of resource bloat.
Activity Log
The searchable activity log shows:
- Why a search was rejected
- The latest search sample
- The requested results page
- The reported browser or user agent
- A masked network
- How often the same pattern occurred
Repeated activity is grouped into a maximum of 250 signatures. The log is paginated and can refresh automatically. Raw IP addresses are not retained.
Optional Cloudflare Turnstile
Cloudflare Turnstile can be used when a visitor reaches the search rate limit or when verification is required before searching.
Visitors who complete the verification receive a short-lived signed cookie so they can continue searching without being challenged on every request.
Turnstile is optional and is not used unless a site administrator configures it.
Scope
Search Scrape Guard applies its protections to the public WordPress search feature. It does not apply those restrictions to administration, REST, AJAX, cron, or WP-CLI requests.
The plugin can also add noindex directives to search-result pages and search exclusions to WordPress’s virtual robots.txt file.
Robots directives are guidance for compliant crawlers. They are not a security control.
External Services
Cloudflare Turnstile is optional and no data is sent unless a site administrator configures Turnstile keys. Once configured, Turnstile may be used when required verification is enabled or after a search rate limit is reached. The verification page loads Cloudflare’s Turnstile JavaScript and submits the visitor’s verification token and connection IP address to Cloudflare for validation. This data is sent only when Turnstile is configured and a search request requires verification.
Cloudflare privacy policy: https://www.cloudflare.com/privacypolicy/
Cloudflare terms: https://www.cloudflare.com/website-terms/
The plugin does not otherwise send search queries, log records, usage statistics, or site data to TechDex or any other external service.
Privacy
Search Scrape Guard stores bounded operational security data locally in the WordPress database. Raw IP addresses are not retained. Search samples and reported user-agent strings may contain information supplied by a requester, so administrators should treat the log as security data and limit dashboard access accordingly.