Investigating “Site Quality Monitoring” Crawl Errors on Sub-directory URLs
-
Hi everyone,
I’m currently managing a shop site on cPanel version 132.0.x and I’ve run into a specific issue with the new <b data-path-to-node=”6″ data-index-in-node=”109″>Site Quality Monitoring</b> (koality) integration.While my homepage returns a 100% health score, the crawler seems to be flagging “Broken Link” or “MIME Type Mismatch” errors specifically on my deeper category pages. For example, on this URL: <b data-path-to-node=”7″ data-index-in-node=”193″><link-block _nghost-ng-c156078443=”” class=”ng-star-inserted”>https://de.roots24.shop/c/lithium</link-block>The problem:</b>
When I run a manual crawl via the Metrics > Site Quality Monitoring interface, the page loads fine in a browser, but the cPanel tool reports a <code data-path-to-node=”8″ data-index-in-node=”156″>403 or sometimes a <code data-path-to-node=”8″ data-index-in-node=”175″>timeout specifically for the assets on that path.<b data-path-to-node=”9″ data-index-in-node=”0″>Server Specs:</b><b data-path-to-node=”10,0,0″ data-index-in-node=”0″>OS:</b> AlmaLinux 9.x<b data-path-to-node=”10,1,0″ data-index-in-node=”0″>Web Server:</b> LiteSpeed with LSCACHE enabled<b data-path-to-node=”10,2,0″ data-index-in-node=”0″>PHP:</b> 8.3 (PHP-FPM)
<b data-path-to-node=”11″ data-index-in-node=”0″>What I’ve checked:</b>
Mod_Security logs (nothing being blocked for the cPanel monitoring IP).<code data-path-to-node=”12,1,0″ data-index-in-node=”0″>.htaccess rules (Standard Shopware/WordPress-style redirects).Permissions are set to 755 for the <code data-path-to-node=”12,2,0″ data-index-in-node=”35″>/c/ directory.
Is it possible that the Site Quality Monitoring bot is being rate-limited by the server-side firewall (CSF), or is there a known issue with how it handles sub-directory paths in the latest EasyApache 4 update?Any pointers on where to look in the logs would be greatly appreciated!
The blog I need help with is: (visible only to logged in users)
-
Since you have already ruled out Mod_Security and verified directory permissions, the 403 and timeout errors on your deeper category paths suggest that LiteSpeed’s anti-throttling logic or CSF (ConfigServer Security & Firewall) is flagging the Site Quality Monitoring (koality) crawler as a bot attack. Because category pages often require more PHP processing than a static homepage, LiteSpeed may trigger a “Soft Limit” or “Hard Limit” when the crawler requests multiple assets in quick succession, resulting in a Forbidden response to protect server resources. To resolve this, you should check your CSF Deny logs at /etc/csf/csf.deny for any blocked monitoring IPs and add them to your allowlist, while also reviewing the LiteSpeed Web Server console under Server → Security → Flood Protection to ensure the connection limits aren’t too restrictive for automated tools. Additionally, verify your .htaccess file for any “Hotlink Protection” or User-Agent rewrites that might be blocking the bot’s access to CSS or JS files, which would explain the “MIME Type Mismatch” error when the crawler receives an HTML 403 page instead of the expected asset.