# robot.txt User-agent: Googlebot Disallow: /*.psn$ Disallow: /*? Disallow: /quakes/ Disallow: /cgi-dos/ Crawl-delay: 5 User-agent: * Disallow: /quakes/ Disallow: /cgi-dos/ Crawl-delay: 5