# Robots.txt for dmoz.at # Last update: 20030725-tschild # # NOTE: # Please do not crawl us faster than 1 hit/second # # purpose: deny everything except http://dmoz.at/ main page # User-agent: * Disallow: /a Disallow: /f Disallow: /i Disallow: /l Disallow: /m Disallow: /robots.txt. Disallow: /s Disallow: /w