# Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name; "*" is for everyone # Disallow: if this matches first part of requested path, # forget it User-agent: Googlebot # Google Disallow: /cgi-bin/ Disallow: /images/ User-agent: Googlebot-Image # Google's Image Search Disallow: /cgi-bin/ User-agent: msnbot # MSN Disallow: /cgi-bin/ Disallow: /images/ User-agent: Slurp # Inktomi Disallow: /cgi-bin/ Disallow: /images/ User-agent: fast # AllTheWeb Disallow: /cgi-bin/ Disallow: /images/ User-agent: teomaagent1 # AskJeeves Disallow: /cgi-bin/ Disallow: /images/ User-agent: directhit # AskJeeves Disallow: /cgi-bin/ Disallow: /images/ User-agent: lycos # Lycos Disallow: /cgi-bin/ Disallow: /images/