Projects/Lanvalia

Lanvalia is a small Web Crawler for research purposes (data mining). It is programmed in Ruby using the excellent sqlite and Hpricot libraries.

User-Agents:

Blocking

Lanvalia obeys the robots.txt exlusion standard (only robots.txt file, no meta tags).

Example robots.txt entry

User-Agent: Lanvalia

Disallow: /

Problems

If the bot is misbehaving, please contact me (don't forget to include access log or at least time, ip and url).