Site blocked from Google crawlers

  • Unknown's avatar
    adirondackfamilybookfest · Member ·

    Dear WordPress, It appears my website is being blocked from being indexed by Google due to some robots.txt code. However, no fix I have attempted works as everything requires an upgrade to a paid account. Is this correct? Please help.

    WP.com: Yes
    Correct account: Yes

    The blog I need help with is: (visible only to logged in users)

  • Unknown's avatar
    helper-heroponriki · Member ·

    Hi there!

    By default, every website hosted on WordPress.com has its own robots.txt file that search engines will pick up and you shouldn’t need to upgrade to fix this. That error usually indicates you have “Discourage search engines from indexing this site.” selected in your Settings > Reading section or the site is set to be Private.

    Looking at your site logs, it looks like you did have those settings enabled at one point, but you switched it to Public around two months ago.

    It’s quite possible Google’s cache of your site was made back when you had those privacy settings enabled. You can submit a request to Google to recrawl your site which would involve setting up your site in Google’s Search Console. You can read more about this here:

    https://wordpress.com/support/seo/#why-isnt-my-site-showing-up-on-google

    I hope that helps! Let us know if you have any questions.

  • Unknown's avatar
    adirondackfamilybookfest · Member ·

    Ok, great! however, I seem to be unable to verify site ownership. Will Filezilla FTP work with the free WordPress.com site to upload the verification code needed to request the site be re-indexed?
    Thanks.

  • Unknown's avatar
    helper-heroponriki · Member ·

    We make it even easier! :)

    While most site verification tools will ask you to add a “meta tag” to your site’s header, we’ve simplified the process by providing specific tools for adding the meta tag to your site, even on the free plan.

    Visit My Site → Tools → Marketing → Traffic and scroll down to Site verification services to access these tools.

    Here’s the link to the specific Google instructions that will help: https://wordpress.com/support/site-verification-services/#google-search-console

  • The topic ‘Site blocked from Google crawlers’ is closed to new replies.