My robots.txt does not change when i change…

freddiemaize · Member · Jun 19, 2009 at 3:18 pm
Copy link

Add topic to favorites
My robots.txt does not change when i change from

I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archivers

to

I would like my blog to be visible only to users I choose

Nor it changes when I change the other way

Do anyone have a clue on this?? Please share it!

The blog I need help with is: (visible only to logged in users)
freddiemaize · Member · Jun 19, 2009 at 3:19 pm
Copy link
Oh sorry sorry.. It dose change.. After few mints..
thesacredpath · Member · Jun 19, 2009 at 3:24 pm
Copy link
This is what is in your robots.txt file right now which is an public with search engines allowed:
```
Sitemap: http://freddiemaize.wordpress.com/sitemap.xml

User-agent: IRLbot
Crawl-delay: 3600

User-agent: *
Disallow: /next/

# har har
User-agent: *
Disallow: /activate/

User-agent: *
Disallow: /signup/

User-agent: *
Disallow:
```
This is what it would look like with the blog set to “private” or to block search engines since in both cases you would be blocking search engines:
```
User-agent: *
Disallow: /
```
thesacredpath · Member · Jun 19, 2009 at 3:26 pm
Copy link
It may be that it will take a few minutes to an hour for the change to make it though to all the datacenters and to all the servers, so my suggestion would be to wait for an hour and then check it again, and make sure and either clear your browser cache before doing so, or force-refresh/reload that page so you can make sure the browser is not pulling from its cache.

No items found.