# # MANIFEST: Robot exclusion control file # # This file controls robot access to http://www.catalogovirtual.com/ # # The Standard for Robot Exclusion may be found at # http://info.webcrawler.com/mak/projects/robots/norobots.html # # A list of known spiders may be found at # http://info.webcrawler.com/mak/projects/robots/active.html # User-agent: Lycos* Allow: / Disallow: /cgi-win/ Disallow: /cgi-bin/ Disallow: /cgi-shl/ #Others allowed (for now!) User-agent: * Allow: / Disallow: /cgi-win/ Disallow: /cgi-bin/ Disallow: /cgi-shl/