Indexed, though blocked by robots.txt

  • Unknown's avatar
    myhearinglossstory · Member ·

    Hello,
    I recently received the following email:

    New issue detected on https://myhearinglossstoryforum.wordpress.com/
    To owner of https://myhearinglossstoryforum.wordpress.com/,
    Search Console has identified that your site is affected by 1 new issue of type Coverage.

    Top warnings (5 maximum)
    Warnings are suggestions for improvement. Some warnings can enhance your appearance on Search; some warnings can become errors in the future. The following warnings were found on your site:

    Indexed, though blocked by robots.txt

    We recommend that you find out more about these issues and fix them when possible to enable your site to have the best possible experience and coverage in Google Search.

    I followed the link, but can’t find a way to fix this problem – I am using a free hosted WordPress account. It seems that my blog visitors can no longer login to this page…

    Any suggestions would be very helpful! Thank you!
    Best wishes

    The blog I need help with is: (visible only to logged in users)

  • Unknown's avatar
    staartmees · Member ·

    your site is up and running for me and it seems Google has indexed your site. Who send you tha e-mail and what link did you follow?

  • Unknown's avatar
    myhearinglossstory · Member ·

    Hello
    Thank you for your reply.

    The blog address is:
    https://myhearinglossstoryforum.wordpress.com/

    Is this the one that is working for you?

    I was concerned because when I typed “https://myhearinglossstoryforum.wordpress.com/wp-login.php”
    into Google, it said that there was no information available. It also worried me because it is e search result in its own right…

    The Email was sent to me by Google Search Console Team.
    This is the page the ‘fix’ button took me to, but I think it might be protected:
    https://search.google.com/u/1/search-console/index/drilldown?resource_id=https://myhearinglossstoryforum.wordpress.com/&item_key=CAMYBCAD&utm_source=wnc_10030322&utm_medium=gamma&utm_campaign=wnc_10030322&utm_content=msg_100058679&hl=en-GB

  • Unknown's avatar
    garysixtyeight · Member ·

    I was concerned because when I typed “https://myhearinglossstoryforum.wordpress.com/wp-login.php”
    into Google, it said that there was no information available. It also worried me because it is e search result in its own right…

    It seems that my blog visitors can no longer login to this page

    Hi there,

    This is the intended behavior, it’s not something that has to be fixed.

    WordPress.com uses robots.txt files to block search engines from including certain URL’s in its index, the login URL is one of them

    This does doesn’t prevent you or anybody else from being ale to log in, it only prevents the page from appearing in search results.

  • kokkieh · Staff ·

    Hi there,

    The info provided by @garysixtyeight is spot on. A site’s login page should not ever show up in search results, so robots.txt is doing what it should by blocking search engine crawlers access to it.

    An important thing to understand about Google’s Search Console and Pagespeed Insights tools is that, while they’re very useful, they’re very dumb. They compare sites to a set of arbitrary rules set by Google, and if a site does not conform to those rules, they start shouting warnings.

    But they’re completely ignorant of the context why certain things on a website is set up the way it is, and aren’t able to check whether something is actually broken/not working.

    Sometimes they’re just plain wrong – a few weeks ago I got an email from search console claiming that my personal site isn’t mobile responsive. Meanwhile my site is using a theme that was designed for mobile screens first, and that looks and works beautifully on mobile. But it didn’t match a new rule that Google added to the tool, so it told me my site is broken when it was actually completely fine.

    So you should always take these warnings with a grain of salt. If you actually see a problem on your site, or in this case with search results, these tools can provide you with info to help you find the cause. But their pro-active “warnings” are more often just red herrings.

    I hop that clarifies things a bit for you.

  • Unknown's avatar
    myhearinglossstory · Member ·

    Hello

    Thank you for your comment.

    That’s good to know to take the Google Search Console results with a pinch of salt – it did worry me. With having a basic account, it obviously limits how much control I have over the account, which makes fixing things often really difficult.

    Thanks again and best wishes
    Carly

  • Unknown's avatar
    garysixtyeight · Member ·

    You’re welcome, glad we could help :)

  • The topic ‘Indexed, though blocked by robots.txt’ is closed to new replies.