# /robots.txt file for http://webcrawler.com/ # mail webmaster@webcrawler.com for constructive criticism # For help see http://www.robotstxt.org/wc/exclusion-admin.html User-agent: * Disallow: /test/ Disallow: /ts/demo/ # BEGIN XML-SITEMAP-PLUGIN Sitemap: http://www.digitalcream.com.au/sitemap.xml.gz # END XML-SITEMAP-PLUGIN