User-agent: * Disallow: /cgi-bin/ Disallow: /gloaming/tmp/ Disallow: /gloaming/cache/ Disallow: /gloaming/class/ Disallow: /gloaming/images/ Disallow: /gloaming/include/ Disallow: /gloaming/install/ Disallow: /gloaming/kernel/ Disallow: /gloaming/language/ Disallow: /gloaming/templates_c/ Disallow: /gloaming/themes/ Disallow: /gloaming/uploads/ Disallow: /gloaming/modules/piCal/ # Referrals is disallowed to prevent referral spam from having an # impact on search engine rankings # Stop Google from indexing the category pages, as they contain # information that is duplicated from the (more useful) archives User-agent: Googlebot Disallow: /gloaming/categories Disallow: /gloaming/redirect Disallow: /gloaming/referrals Disallow: /gloaming/modules/piCal/ # Links in comments are redirected via this script - make sure # search engine spiders ignore all redirected links # See http://simon.incutio.com/archive/2003/10/13/linkRedirects User-agent: * Disallow: /redirect Disallow: /referrals