Robots.txt blocking Twitterbot. Don't know why
-
Hello, my robots.txt file now displays the following additional lines of code which are preventing Twitter from creating a cool looking card:
User-agent: Twitterbot
Disallow: /When I plug a post (eg: https://themetrorailguy.com/2020/12/22/ardanuy-awarded-surat-ahmedabad-metros-ddc-traction-contracts/) into Twitter’s card validator (https://cards-dev.twitter.com/validator) it says:
“ERROR: Fetching the page failed because it’s denied by robots.txt.”
This problem just started occurring. I do not have a robots.txt file on my site’s file manager. I have checked All in One SEO’s robots.txt section and I have that part disabled as well. So what exactly could have inserted those 2 lines of code? How can I remove them?
Thanks!
The blog I need help with is: (visible only to logged in users)
-
Hi there,
You’re in the wrong forum. Your site is using the open-source WordPress software and therefore you need to seek help at these forums:
https://wordpress.org/support/forums/
These forums are for WordPress.com hosted sites only. If you want to know more about the differences between WordPress.com and the WordPress software you can read this document:
- The topic ‘Robots.txt blocking Twitterbot. Don't know why’ is closed to new replies.
WordPress.com Forums 