robots.txt

• 

  pressword1111 · Member · Sep 22, 2021 at 8:59 am

  • Copy link
  • Add topic to favorites

  I want to add/edit a robots.txt file to disallow in order to prevent any search engine crawling and to remove the site from the Internet Archive (Wayback Machine).

  How do I do this when I have a free account?

  Site: (email visible only to moderators and staff).
  WP.com: Yes
  Correct account: Yes

• 

  justjennifer · Member · Sep 22, 2021 at 4:58 pm

  • Copy link

  Hi there, here on WordPress.com the only way to completely block access to search engines is by changing your site visibility to Private. Here’s more on that:
  https://wordpress.com/support/settings/privacy-settings/

  Post back here if you have any other questions.

• 

  pressword1111 · Member · Sep 22, 2021 at 11:55 pm

  • Copy link

  That may do it from search engines, but archive.org specifically say you must have robots.txt in the root of your username.Wordpress.com site.

  This is something that WordPress should be able to offer.

• 

  justjennifer · Member · Sep 23, 2021 at 6:10 am

  • Copy link

  Hi there, Sorry if I wasn’t clear.

  Every WordPress.com website has a robots.txt file. You can see yours by adding “/robots.txt” (without the quote marks) to your WordPress.com site URL. The settings in that file are determined by what you set as your site’s Privacy settings. We are not able to access or manipulate that file directly.

  You can learn more about robots.txt at https://moz.com/learn/seo/robotstxt

  Hope that is clearer, but let me know if you have other questions about this.