#Robots.txt www.davidcarralon.com/robots.txt User-agent: * Disallow: # Don't allow spambot to crawl any pages User-agent: spambot Disallow: / # Disallow access to CLIENTS User-agent: * Disallow: /CLIENTS # Allow Majestics bot to crawl User-Agent: MJ12bot Disallow: sitemap:www.davidcarralon.com/sitemap.xml