Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
baiduspider
new IP for baiduspider, fwiw
5 kpaul 10:24 pm Sep 15, 2003
37.104 and 39.104
what are these, please?
8 beezee 9:04 pm Sep 15, 2003
Beginner needs spider monitoring software
4 nicheman 4:20 pm Sep 15, 2003
64.156.198.78
from Windows to Linux within one minute
5 coyote 1:14 pm Sep 15, 2003
IBM_Planetwide
Getting hit very hard by this spider
4 requiem 3:46 am Sep 15, 2003
Keebler elf?
bot from Keebler fell in trap
5 coyote 2:11 am Sep 15, 2003
Utse/0.04?
8 keyplyr 6:24 am Sep 14, 2003
more 131.107.
nonsense
4 wilderness 6:12 am Sep 14, 2003

3 pendanticist 5:47 am Sep 14, 2003
Under_the_Rainbow_2.2 from 80.58.13.107
3 ServMe 7:49 pm Sep 12, 2003
I'm sure you've heard this before... list of all of 'em...
5 andrewrab 5:09 pm Sep 11, 2003
What's this?
not a spider but it's "pulling" everything
17 SuzyUK 3:32 pm Sep 11, 2003
New Fast/Overture UA
Yet another new one from FAST
7 jdMorgan 10:09 am Sep 11, 2003
Sleipnir
what is this?
11 keyplyr 7:17 am Sep 11, 2003
Need help with IP block range RegExp for RewriteCond
I want to block 211.104.0.0 - 211.119.255.255
9 Wizcrafts 8:16 pm Sep 10, 2003
Googlebot's schedule?
When does Googlebot visit?
3 Widestrides 11:11 pm Sep 9, 2003
Spiders
Identification
5 irvseykat 9:13 pm Sep 8, 2003
More Bad Bots?
3 BlueSky 8:39 pm Sep 7, 2003
AOLserver-Tcl/3.5.6
what is it?
4 coyote 7:24 pm Sep 7, 2003
64.40.105.197
Look.com
4 wilderness 6:03 pm Sep 7, 2003
PlantyNet_WebRobot_V1.9
a bot and its two humans
2 coyote 2:17 am Sep 6, 2003
Exalead -- it comes for your .mov files
Why should I let this sucker on my system?
4 berli 12:36 am Sep 6, 2003
68.166.235.57
changed UA
5 coyote 3:51 am Sep 5, 2003
Illinois State Tech Labs
from a french IP
3 coyote 2:28 am Sep 5, 2003
Feedster Crawler
2 frontpage 10:52 pm Sep 4, 2003