User-agent: * Allow: / User-agent: JobSpider Disallow: / User-agent: JobSpider_BA Disallow: / #User-agent: * #Crawl-delay: 5 #Disallow: /lib