sly : marketing

Make and Check Your robots.txt

We are all interested in helping the [tag]search engines[/tag] to crawl our sites. For this reason you should have a file called [tag]robots.txt[/tag], this tells the [tag]robots[/tag] if they are allowed to search your site and what parts of your site they shouldn´t search.

For example, you could tell [tag]Google[/tag] that they are not allowed to crawl your site at all, or you could tell them that they are allowed to crawl your site, but not the folders with personal pictures.

To learn how to make your robots.txt file, just visit 123promotion, they have an excellent robots txt generator.

When you are done, just save the file as robots.txt and upload it to your server. Finally, you should check if the file validates. This can be done here.

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe: rss | email | twitter | +