detected browser was archiver

[daddywolfe]

YaBB SE Version: 1.5.2 to 1.5.4
PHP Version: 4.3.4
MySQL Version: 3.23.x
Server Platform: Unix, Linux, or BSD
Link to Forum: http://www.ivoryangels.com/yabbse2

Problem Description:
One of the boards on a site we host had an unusual browser show up in the logs.  The detected browser was *****ia_archiver  (1)********

This has never been seen before in the logs, and I do not recall having read or seen anything even remotely like this.  Does anyone know anything about this.  Here's the entire list which was displayed, as reported to me:

Total Browsers: 8

Mozilla/4.0 - MSIE 6.0  (696)
Mozilla/5.0  (198)
Mozilla/4.0 - MSIE 5.0  (44)
Mozilla/4.0 - MSIE 5.23  (12)
Mozilla/4.0 WebTV/2.6 - MSIE 4.0)  (1)
Mozilla/4.0 - MSIE 5.5  (28)
Mozilla/4.0 - MSIE 5.01  (1)
*****ia_archiver  (1)********

[daddywolfe]

Some more information on the event.  This is from the logs:

209.237.238.173 - - [16/Feb/2004:05:28:54 -0700] "GET /robots.txt HTTP/1.0" 404 276 "-" "ia_archiver"
209.237.238.173 - - [16/Feb/2004:05:28:55 -0700] "GET /yabbse2/index.php HTTP/1.0" 200 191301 "-" "ia_archiver"

[[Unknown]]

Block it via robots.txt.  For mroe information, search for it on google .

-[Unknown]

[Peter Duggan]

FWIW, I've been blocking that for years because it thinks it's providing a service by stealing and preserving all the old site versions you'd rather forget!

robots.txt:

Code Select Expand

User-agent: ia_archiver
Disallow: /

[daddywolfe]

LOL... gotcha.  I fixed it now, so they won't be back.  First time I've seen a web crawler show up like that.  We knew the webcrawlers were scanning, but not that one.  I just changed the robot.txt to all inclusive, and moved it to where it needed to be for their site.