# This file is used to allow crawlers to index our site. # It is a good idea to include this file otherwise a 404 is generated in the server logs as crawlwers request the page # List of all web robots: http://www.robotstxt.org/wc/active/html/index.html # # Check robots.txt at: # http://www.searchengineworld.com/cgi-bin/robotcheck.cgi # # Details about Googlebot available at: http://www.google.com/bot.html User-agent: * Disallow: /includes/ Disallow: /email/ Disallow: /images/ Disallow: /media/ Disallow: /apps/surveys/ Disallow: /apps/login/