Robots.txt Search Improvements

Submitted by Bryan on June 5, 2008 - 7:50pm

Gadgetopia: "Google, Yahoo, and Microsoft have gotten together and actually agreed on extensions to the REP — the Robots Exclusion Protocol, otherwise known as your robots.txt file."

Complete Story

»