Controlling search engine spiders

All web sites should have a robots.txt file to direct incoming search engine spiders. This site offers a good free tutorial covering robot.txt syntax. http://www.robotstxt.org/wc/exclusion-admin.html

Related posts:

  1. PDA search engine
  2. Nice Search Engine
  3. “Search Engine Optimization” Scams
  4. New Search Engine Front End
  5. A search engine that understands
= Paid content

All Windows Secrets articles posted on 2004-01-21: