# All robots will spider the domain
User-agent: *

User-agent: *
Disallow: /cgi-bin/

User-agent: *
Disallow: /linkout/
Disallow: /go/
Disallow: /page/
Disallow: /help-us-improve/
Disallow: /contact/
Disallow: /refresh/


Sitemap: http://www.whoishostingthis.com/sitemap.xml