robot.txt disallows search engines
-
Hi
I’m a newbie and this world and I had to face the robot.txt vs Google search engine.
I tried going to dashboard – settings – reading – “Allow search engines to index this site”, but the same problem persists. Please help?
# This file was generated on Sun, 11 Oct 2015 01:31:23 +0000
# If you are regularly crawling WordPress.com sites, please use our firehose to receive real-time push updates instead.
# Please see http://en.wordpress.com/firehose/ for more details.Sitemap: https://travellingclassynista.wordpress.com/sitemap.xml
Sitemap: https://travellingclassynista.wordpress.com/news-sitemap.xmlUser-agent: IRLbot
Crawl-delay: 3600User-agent: *
Disallow: /next/User-agent: *
Disallow: /mshots/v1/# har har
User-agent: *
Disallow: /activate/User-agent: *
Disallow: /public.api/# MT refugees
User-agent: *
Disallow: /cgi-bin/User-agent: *
Disallow: /wp-login.phpUser-agent: *
Disallow: /wp-admin/The blog I need help with is: (visible only to logged in users)
-
Search engines can index your site
http://ismyblogworking.com/travellingclassynista.wordpress.com
What you see is normal – it is good practice to keep search engines out of areas of your site that have no content worth reading (wp-admin for instance)
-
-
- The topic ‘robot.txt disallows search engines’ is closed to new replies.