Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Bot Database
8 digitaleus 2:34 pm Feb 16, 2003
63.148.99.247
[2] ( 1 2 )
39 palmpal 6:45 am Feb 15, 2003
Anyone knows this bot: "Mozilla/4.0 efp@gmx.net"?
question about new bot efp@gmx.net
14 aleksl 12:10 am Feb 14, 2003
Domanova
2 wilderness 11:47 am Feb 13, 2003
Arin Whois
net searches
4 wilderness 4:14 am Feb 12, 2003
Slow Spy Bot
Not sure from Where... or why...
5 carfac 3:53 am Feb 12, 2003
Who is this?
Took a lot of bandwidth
4 spikedo55 7:30 pm Feb 11, 2003
Dart+Communications+PowerTCP
7 paul_london 12:06 pm Feb 11, 2003
why would a satellite proxy/cache trip spider trap
can't figure out if this was a harmless user or bad bot
3 amznVibe 10:17 pm Feb 10, 2003
12.148.209.196
Who is this rude character?
10 Powdork 10:05 pm Feb 10, 2003
EasyWebPromotion bot
How would I block this spider?
3 robjones 2:54 pm Feb 9, 2003
anyone familiar with CHTML
chtml
3 coolcreep 8:12 am Feb 9, 2003
Web Magnet - badly written downloader?
2 jmccormac 7:42 pm Feb 8, 2003
New Inktomi IP
Its a new one for me...
5 Josk 3:20 pm Feb 7, 2003
New Inktomi Spider...
2 Mtlinfo 3:09 pm Feb 7, 2003
AOL bot?
198.81.26.* all over my site
2 Giacomo 11:05 am Feb 6, 2003
Research Machines plc
3 wilderness 3:55 am Feb 6, 2003
Strange UA's
Can't figure out what they really are!
11 carfac 5:31 pm Feb 5, 2003
New to the Game
need some update
4 fashezee 4:28 am Feb 5, 2003
GoogleSynth
2 wilderness 4:22 am Feb 5, 2003
Timing out during crawling
Why does it happen?
4 tkarade 9:58 pm Feb 4, 2003
Strange Activity From Toronto DSL Dialups
2 jmccormac 2:06 am Feb 4, 2003
SplatSearch
6 volatilegx 10:35 pm Feb 3, 2003
167.21.1.227
3 NeedScripts 1:06 pm Feb 2, 2003
Testbot
Hitting way to hard from Dk.
2 eljefe3 4:10 am Feb 2, 2003