robots.txt

• 

  honliong92 · Member · Dec 30, 2015 at 10:04 am

  • Copy link
  • Add topic to favorites

  I tried to search for my own URL on google search. However, the description of the link shows ‘A description for this result is not available because of this site’s robots.txt’.

  Do you know how I can configure it?
  For your information, I am using the web-based interface to set up my page, not the html format. Does it affect the outcome of the page?

• 

  honliong92 · Member · Dec 30, 2015 at 10:09 am

  • Copy link

  I have already ticked the option of “allow search engine to index my site”. I am able to see my link in google but the issue is there is no description below the link. Please, does anyone know about this issue?

• 

  n3wjack · Member · Dec 30, 2015 at 10:48 am

  • Copy link

  I checked the robots.txt but I don’t see why Google has a problem with it.
  Maybe try this tool to figure out why it doesn’t want to crawl the site:
  https://support.google.com/webmasters/answer/6062598

• 

  honliong92 · Member · Dec 30, 2015 at 11:26 am

  • Copy link

  Thanks for the suggestion

  I wanted to follow the guide. However, I have no idea on where to upload the tester file in wordpress as I am using the web version, not html version. Can you guide me?