Robots meta tag and X-Robots-Tag HTTP header specifications - Controlling Crawling and Indexing - Google Code

code.google.com/web/controlcrawlindex/docs/robots_meta_tag.html

Saved 21 times on del.icio.us, most recently in 2012.

Tags: google seo robots.txt http indexing robots crawling header reference meta

More links tagged google

More links tagged seo

More links tagged robots.txt