iRobots.txt SEO

iRobots.txt SEO (IRSEO) is a fully customizable robots.txt virtual file generator. IRSEO creates a highly optimized and secure robots.txt file straight out of the box. Users may choose to enable or disable specific user agents, directories or files using intuitive options all of which include detailed instructions.

The robots.txt file is a text file located in the root directory of a website. It’s purpose is to direct user-agents (AKA bots) away from or towards specific files or directories. Inhibiting a bot from indexing specific pages will ensure your website remains keyword optimized and all indexed pages are relevant to your potential customers.

IRSEO also inhibits several WordPress system directories and files by default. Doing this ensures that the search bots do not include security sensitive pages within search results. For example, searching for « inurl:wp-content name size description » in Google will produce a list of sites with indexed and open content directories.

Note that IRSEO creates a virtual robots.txt file. This robots.txt file is displayed whenever access to the robots.txt file is requested.