WeRelate:Bot

You're probably reading this page because we crawled your website. WeRelate is a vertical search engine and Wiki for genealogy. It is developed by the Foundation for On-Line Genealogy, a not-for-profit organization dedicated to helping people discover and share information about their ancestors. Our goals include providing a freely-searchable, comprehensive index of genealogical information as well as making it easy for people to share genealogical information on-line. We make all of the source code and content available via open-source. Our crawler is based upon Nutch, an open-source web crawler.

How to contact us

If you have problems or questions about our crawler, please send an email to dallan@werelate.org.

Sysadmins/robots.txt

Our crawler obeys the robots.txt exclusion standard, described at http://www.robotstxt.org/wc/exclusion.html#robotstxt. If for some reason you want to ban our crawler from your site, you can place the following in your robots.txt file:

User-agent: WeRelate

Disallow: /

Webmasters/Robots META

Our crawler also obeys robot META tags in HTML. If you do not have permission to edit the /robots.txt file on your server, you can still tell robots not to index your pages or follow your links. The standard mechanism for this is the robots META tag, as described at http://www.robotstxt.org/wc/meta-user.html.