Search engines not crawling

• 

  acupunctuurpraktijk · Member · Jun 19, 2013 at 1:19 pm

  • Copy link
  • Add topic to favorites

  Hi,

  For some reason search engines don’t seem to be crawling my webiste. I changed the setting to allow crawling on the 22th of may but still no search engines have crawled the site. What could the problem be? Or does it take more time than I expected? Below the robot.txt

  # This file was generated on Wed, 22 May 2013 21:55:52 +0000
  # If you are regularly crawling WordPress.com sites, please use our firehose to receive real-time push updates instead.
  # Please see http://en.wordpress.com/firehose/ for more details.

  Sitemap: http://acupunctuurbaidu.nl/sitemap.xml
  Sitemap: http://acupunctuurbaidu.nl/news-sitemap.xml

  User-agent: IRLbot
  Crawl-delay: 3600

  User-agent: *
  Disallow: /next/

  User-agent: *
  Disallow: /mshots/v1/

  # har har
  User-agent: *
  Disallow: /activate/

  User-agent: *
  Disallow: /wp-login.php

  User-agent: *
  Disallow: /signup/

  User-agent: *
  Disallow: /related-tags.php

  User-agent: *
  Disallow: /public-api/

  # MT refugees
  User-agent: *
  Disallow: /cgi-bin/

  User-agent: *
  Disallow: /wp-admin/
  Disallow: /wp-includes/

  Thanks for your help!
  Daniek

  The blog I need help with is: (visible only to logged in users)