Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
spider question
Ssi
6 ganderla 6:51 am Sep 27, 2003
Strange Request in my Logs
3 guillermo5000 3:10 am Sep 27, 2003
24.196.191.232
2 chance 7:05 pm Sep 26, 2003
Python-urllib
from google?
7 silverbytes 1:48 pm Sep 26, 2003
"Image Collector V1.0"
2 seindal 1:42 pm Sep 26, 2003
Colocation
66.114.67.120
8 wilderness 9:02 pm Sep 25, 2003
cj.com spider?
5 jp_19 4:05 pm Sep 25, 2003
Been hit hard lately!
3 creative_craig 12:28 pm Sep 25, 2003
205.223.216.6
6 coyote 2:20 am Sep 25, 2003
Host name: j3174.inktomisearch.com
All Over My forum's for Hours
3 epic1231 3:32 pm Sep 24, 2003
FAST-WebCrawler/3.8/Fresh
Is this new?
2 mipapage 11:33 pm Sep 23, 2003
Overture-WebCrawler
6 Tor 6:29 pm Sep 23, 2003
Internetseer
How do I kill this parasite?
14 epanastatis 11:50 pm Sep 22, 2003
Can anyone imagine/knows what Googlebot's Interface is like?
Just wondering
4 sidyadav 8:56 pm Sep 22, 2003
203.24.143.197
bot used to steal?
4 coyote 6:47 pm Sep 21, 2003
Intelliseek spybot / snoopbot / copyrightbot
any other IP ranges for this pest
2 nativenewyorker 6:23 pm Sep 21, 2003
Name Protect bot a big pest
How to ban it?
13 mayor 6:05 pm Sep 21, 2003
Program Shareware 1.0.3
What is this?
9 andrewrab 1:15 am Sep 20, 2003
RPT-HTTPClient/0.3-3
New IBM bot or just odd browser?
2 claus 11:35 pm Sep 19, 2003
New User_Agent "PHP/4.1.1"
Who knows about this UA?
3 Wizcrafts 4:27 pm Sep 19, 2003
New ZyBorg UA strings - again
Two of them - this time featuring "Beta"
2 claus 3:56 pm Sep 19, 2003
27 Googlebot's on my site
...all at the same time
5 esllou 6:29 pm Sep 18, 2003
MSIE Crawler, visits daily, follows overture listing
A nasty little 'spider' that is costing me money
4 werty 4:12 pm Sep 18, 2003
EgotoBot/4.8
spiders dmoz listing, "caching" pages
2 stevenha 11:40 am Sep 16, 2003
Yahoo is up to something
207.126.224.174
4 fiestagirl 6:20 am Sep 16, 2003