#stop these files / directories being crawled by any spider User-agent: * Disallow: /styles/ Disallow: /amie Disallow: /css/ Disallow: /javascript/ Disallow: /commentarticle #Google images crawler User-agent: Googlebot-Image Disallow: / Allow: /images/ #specific rules for Google #User-agent: Googlebot Sitemap: http://www.thebeautybiz.com/sitemap.xml