Showing posts with label robots. Show all posts
Showing posts with label robots. Show all posts

Tuesday, May 10, 2011

Matt Cutts: Using 'NonIndex, Follow' in robot meta tags

Imagine a scenario in which you have an HTML sitemap but for one reason or another you do not want Google to return the sitemap itself within the search results.

Thursday, October 14, 2010

Serious Robots.txt Misuse & High Impact Solutions


Some of the Internet's most important pages from many of the most linked-to domains, are blocked by a robots.txt file. Does your website misuse the robots.txt file, too? By Lindsey

Friday, February 5, 2010

Testing How Crawl Priority Works

Typical Site's Link Earning Potential by Content Section

We all know that the search engine robots more frequently visit popular pages, i.e. those that have the largest number of incoming links, both internal and external ones. By M. Galecki