User-Agent: * Disallow: /lib/ Disallow: /asc2009/ Disallow: /asc2010/ Disallow: /asc2008/ Disallow: /admin/ Disallow: /admin/ Disallow: /pdf/ Allow: / # See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file