Site made public, but Google still isn't indexing it – robots.txt issue?
-
So last week, I did a Google search to see how my site was doing for a search by my name. It came up on the 2nd page of results, but instead of giving my site name and description, it just gave the URL as the site name, and said “A description for this result is not available because of this site’s robots.txt”.
I checked, and lo and behold, in making my site, I had accidentally checked the option to keep search engines from indexing it. Or maybe this was the default setting, because I don’t know why I would have selected that. Anyway, I switched the setting to make the site public and index-able.
A week later, I did another search and the way Google describes my site is unchanged. It still says the thing about the robots.txt file instead of a description. Is this something that takes a while to update, or is there something else I’m missing? I decided to try to update my site’s robots.txt file directly and was unable to find where to access it on my dashboard.
Any thoughts? Has anyone else run into this problem?
The blog I need help with is: (visible only to logged in users)
-
The result of making a site private which I never do is a robots.txt file.
You can Ask Google to recrawl your URLs
https://support.google.com/webmasters/answer/6065812?hl=enre: search engine discoverability
It can take weeks for search engines to index your site content. Please read this support doc https://en.support.wordpress.com/search-engines/ so you know we have no control over search engines positioning or appearance in the SERPs (search engine page results).
To verify blog ownership of a WordPress.com hosted blog with the major search engines you must use this process > http://en.support.wordpress.com/webmaster-tools/
Note: Even if you do not verify the blog the content will be indexed by search engines so don’t feel panicky about this please.WordPress.com automatically supplies sitemaps for our blogs to search engines – we do nothing. http://en.support.wordpress.com/sitemaps/#xml-sitemaps-for-search-engines
Search engines aren’t attracted to sites that are page based. Sites that are post based where publication of posts, not pages is frequent are more attractive to them. To gain search engine attention, I recommend that you start publishing posts (not pages) https://en.support.wordpress.com/post-vs-page/ frequently.That’s because it can take weeks for search engines to index content in a new blog and/or to re-index content under a new URL.
WordPress.COM SEO resources that you will want to consult are:
https://en.support.wordpress.com/search-engines/
https://en.support.wordpress.com/webmaster-tools/
http://en.blog.wordpress.com/2013/03/22/seo-on-wordpress-com/
https://dailypost.wordpress.com/2016/03/10/six-seo-factors-you-should-know/
http://en.support.wordpress.com/getting-more-views-and-traffic/
https://dailypost.wordpress.com/postaday/ebook-grow-traffic/Also, note that posts do not display in the Reader forever. Posts only display in the WordPress.COM Reader for 60 days.
It’s important that you read this support doc closely http://en.support.wordpress.com/topics/#missing-posts
The rule of thumb is to assign to your posts the least, not the most, combined number of only relevant categories and tags that accurately describe the individual post content. And, you never assign the same keyword or keyword phrase as both a category and a tag.
- The topic ‘Site made public, but Google still isn't indexing it – robots.txt issue?’ is closed to new replies.
WordPress.com Forums 