Privacy Settings robots.txt not immediate?

• 

  tedthetrumpet · Member · Aug 4, 2011 at 10:07 am

  • Copy link
  • Add topic to favorites

  Are changes to Privacy Settings not immediate? I’ve changed my site visibility to ‘I would like my site to be visible to everyone, including search engines (like Google, Bing, Technorati) and archivers’, but my http://jsimonvanderwalt.com/robots.txt still says:

  User-agent: *
  Disallow:

  I’ve flushed my cache, it’s not that…

  The blog I need help with is: (visible only to logged in users)

• 

  thesacredpath · Member · Aug 4, 2011 at 10:18 am

  • Copy link

  Go back to settings > privacy and make sure it is still set for full access. If it is, then check in an hour or so. Perhaps the servers are just busy and backed up.

• 

  wpgaurav · Member · Aug 4, 2011 at 10:22 am

  • Copy link

  Yes. It takes time. When I did so, it took about 6hours.

• 

  tedthetrumpet · Member · Aug 4, 2011 at 10:32 am

  • Copy link

  Ok, thanks, the setting is definitely correct, I logged back out and in again to check for sure… I’ll give it some time.

• 

  tedthetrumpet · Member · Aug 5, 2011 at 8:34 am

  • Copy link

  Getting tired and frustrated. Getting near to the 24 hour mark here, still locked out of the search engines, and the WP team is on a jolly to St Louis?!? Why should a change like this not be instant? I don’t get it.

• 

  thesacredpath · Member · Aug 5, 2011 at 8:41 am

  • Copy link

  I’ll tag this thread for staff attention. They are monitoring the forums, so hopefully someone will respond in a few hours.

• 

  tedthetrumpet · Member · Aug 5, 2011 at 9:06 am

  • Copy link

  Thanks, I appreciate it.

• 

  thesacredpath · Member · Aug 5, 2011 at 9:12 am

  • Copy link

  You are welcome and someone from staff should (hopefully) wander through in the next few hours or so.

• 

  auxclass · Member · Aug 5, 2011 at 9:21 am

  • Copy link

  http://ismyblogworking.com/jsimonvanderwalt.com

  Shows that search engines can crawl your site. It also shows the same two lines you don’t like as letting in robots. Looking at my site the robots.txt has something in the Disallow line to block some access, so seems to me that a blank entry does not block access, but it is late so my memory might be a bit fuzzy.

• 

  tedthetrumpet · Member · Aug 5, 2011 at 9:34 am

  • Copy link

  Aha! I’ve not been looking closely enough. I’ve been glancing at the page, and not noticing the difference between what it says now:

  User-agent: *
  Disallow:

  And what it said when I first changed my settings, which was

  User-agent: *
  Disallow: /

  That wee slash is rather crucial, and I didn’t notice it before. Although… I found before some examples of wp robots.txt files, which I can’t see now: shouldn’t the xml sitemap be in there also?

  Anyway, it looks like google at least has crawled my blog now, off to try to wake up a few of the other search engines.

  Thanks for the support.

• 

  auxclass · Member · Aug 5, 2011 at 9:45 am

  • Copy link

  You be welcome

• 

  thesacredpath · Member · Aug 5, 2011 at 9:56 am

  • Copy link

  @auxclass, good catch. I must remember to put brain in gear.