robo txt

• 

  skeyboard · Member · Apr 5, 2017 at 4:11 pm

  • Copy link
  • Add topic to favorites

  Hi- I have a site on wordpress , with two mirror sites. They are not being indexed by google because it says that there is robo.txt
  if you type the site name in, it will work, but if you search for it, it will not show up.
  I have made no changes on the site,the only thing ever happening there is when you automatically update the site. Please help.
  I contacted the host and they say it has nothing to do with them. they say the issue is with google or with the code on the site itself (which has not changed, and used to work perfectly.
  Thanks.
  Please help

  The blog I need help with is: (visible only to logged in users)

• 

  justjennifer · Member · Apr 5, 2017 at 4:21 pm

  • Copy link

  Hi there, you’ve posted your question in the forums for sites hosted on WordPressdotcom, but it sounds like you need help for your WP (org) sites.

  Standalone WP (org) and WordPressdotcom (where we are) are different entities; more information about the differences can be found here: https://en.support.wordpress.com/com-vs-org/

  Help for self-hosted sites using the standalone WP (org) software can be found over at https://codex.wordpress.org/Main_Page and at https://wordpress.org/support/ If you don’t have a username on the ORG forums, you can register one on that page.

  Best wishes

• 

  skeyboard · Member · Apr 5, 2017 at 7:59 pm

  • Copy link

  Actually, the blog it says this is related to is not the site I was looking for help with, and I can’t seem to log in anywhere or reach anyone who can help me. The logo on my sites (not blogs- websites) look like the blue logo used here on wordpress.com.

• 

  justjennifer · Member · Apr 6, 2017 at 4:20 am

  • Copy link

  The best thing to do then would be to provide us the link(s) to the sites you need help with. That way we can tell you if we can help you or you need to post in the WP (org) forums.

  All sites have a robots.txt file that tells search engines if they can crawl the site and what folders should be left out (more about robots.txt).

  On WordPressdotcom, the robots.txt is set by the Privacy settings you’ve given your site https://en.support.wordpress.com/settings/privacy-settings/ but we cannot change the robots.txt directly.