SiteMap and Robots File

Sitemap and Robots

One of the best things you can do to get the search engines to notice you, is to install a sitemap and robots file on your blog or website.  Now I know that may sound a bit trivial for some and all, but it is really really simple.

First of all, you can get a sitemap very easily from XML-Sitemaps.com.  You simply follow four simple steps on their site and in no time at all, you will have the file you need for your site.  Very simply put: upload that file to your root directory (the place that is the bottom most tier of your website - example: c:/ would be the root of a PC, and so the same would apply to a website or blog, only the root is where the website is being hosted).

Once you have done so, it is a great idea to sign up with Google Webmaster Tools so you can tell Google exactly what you want crawled on your site.  You will be able to designate what to crawl and what not to.  Although some crawling robots out there, ignore the request and crawl everything anyway (usually these are spam search robots). 

SiteMap XML

A sitemap basically tells Google and other search engine crawling robots, what your site looks like in terms of a map.  It says to the crawling spiders what is okay to index and what is not.  It is important to have a sitemap on your site, because even though the internet looks for content all the time, they won’t always know you are there, unless you make an effort to tell them.

Along with a sitemap, it is a good idea to install a robots.txt file so that it can work in conjunction with the sitemap in that it will tell any other crawling search "spider", what to search and what to leave alone.  And although a sitemap and robots file sound the same, they are not.  The sitemap tells the search engines what the landscape looks, like and the robots file tells them what to index and crawl, and what to leave alone.

Robots.txt

You can easily get a robots.txt file the same way you got a sitemap from XML-Sitemaps.  Just simply upload this .txt file to your root directory also and you are set to go.  If you want to find a sample robots.txt file to upload to you site, just search robots.txt on the net and you will see a sample come up.  Save it as a txt file and upload it to the root directory.

The internet is so much friendlier to your site or blog when you have these things in place.  You can visit Google Webmaster Tools for more information or XML-Sitemaps too.