SEO tool (Screaming Frog) can't connect to my site

• 

  sershink · Member · Aug 12, 2019 at 6:25 pm

  • Copy link
  • Add topic to favorites

  I’m trying to use SEO tool, recommended by WordPress in recent Newsletter, but getting connection error – Connection Refused. It works well with other sites, hosted by other provider.
  I’m using url https://mx5things.com/

  The blog I need help with is: (visible only to logged in users)

• 

  supernovia · Staff · Aug 15, 2019 at 11:09 pm

  • Copy link

  Hi @sershink, can you let us know what SEO tool you’re referring to? If WordPress.com recommended it, it’ll likely work here, but may need to be configured differently than you would on a self-managed site.

• 

  sershink · Member · Aug 16, 2019 at 5:49 pm

  • Copy link

  SEO tool Screaming Frog

• 

  staff-zinnia · Staff · Aug 16, 2019 at 6:15 pm

  • Copy link

  Thanks. To confirm, this is the tool that you’re unable to connect to your WordPress.com account? https://www.screamingfrog.co.uk/seo-spider/

  It won’t be possible to connect that service unless you are on our business plan. The business plan has a different site infrastructure that permits plugins to be used. Those details are at https://wordpress.com/pricing/

  If you decide to upgrade you’ll only pay the difference between your premium plan and the business plan.

  Write back if you have more questions.

• 

  sershink · Member · Aug 16, 2019 at 6:55 pm

  • Copy link

  thanks

• 

  justjennifer · Member · Aug 19, 2019 at 5:22 am

  • Copy link

  Hi folks, Screaming Frog is a utility program downloaded and run from one’s computer rather than installed on site as a plugin, like Yoast, which is also recommended in that same newsletter

  I pinged Screaming Frog support and they replied

  WordPress.com servers respond with a connection refused to a request from the Screaming Frog SEO Spider user-agent.

  This appears to be some kind of ‘bot’ related check the server obviously performs, to ensure only real people visit etc.

  You can still crawl the websites by changing user-agent (Config > User-agent) and switching to Googlebot, or Chrome etc. But this feature is only available in the licenced version of our software.

  So unless WordPress.com approves their user agent, it seems it may not be possible to use the free version of their utility on WordPress.com. (Personally I’d rather been hoping it was possible since A. it was mentioned in the marketing newsletter and B. I haven’t found a good checker since Xenu, which was last updated for Windows XP.)