Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Netriplex
5 keyplyr 1:33 am Dec 10, 2011
Nigma.ru
2 Pfui 10:02 pm Dec 3, 2011
Kindle
7 Pfui 7:38 pm Dec 3, 2011
Opera Night at Amazon Web Services
2 incrediBILL 6:17 pm Dec 3, 2011
ContextAd Bot
2 keyplyr 7:15 pm Nov 30, 2011
spxyric1.bankofamerica.com
3 keyplyr 10:20 pm Nov 28, 2011
Lycosa - anyone know what this is?
15 motorhaven 1:59 pm Nov 27, 2011
80legs IPs
to post or not to post
8 incrediBILL 7:07 am Nov 27, 2011
Mozenda
2 incrediBILL 9:02 pm Nov 26, 2011
sem rush bot and Amazon IPs
Is this an Amazon bot or a bot using Amazon?
3 grandma_genie 7:51 pm Nov 26, 2011
SearchBot
3 keyplyr 12:47 am Nov 24, 2011
Host-Spy
2 Pfui 8:18 am Nov 22, 2011
Pixray? Ixnay .
2 Pfui 1:58 am Nov 20, 2011
Hostpro Ltd, Ukraine
2 keyplyr 8:46 pm Nov 19, 2011
Baiduspider - does it obey robots.txt ?
[2] ( 1 2 )
34 1script 10:37 pm Nov 18, 2011
Microsoft bot 157 ranges updated
DNS for IP ranges 157.55 and 157.56 updated
21 dstiles 6:03 pm Nov 18, 2011
about.com
4 keyplyr 8:50 pm Nov 16, 2011
184.105.140.
Hurricane Electric -- a.k.a. Google Quality Rater?
9 Pfui 8:48 pm Nov 16, 2011
WASALive
4 Pfui 6:33 pm Nov 16, 2011
207.38.118.
BAT's Bad Bot(s)
4 Pfui 9:54 pm Nov 15, 2011
Spider Tracking Links - Examining 2 Methods
13 incrediBILL 10:09 pm Nov 14, 2011
ColoSpace, Inc.
4 keyplyr 9:06 pm Nov 14, 2011
siteintel.net
Bot-running bad Opera
8 Pfui 8:44 pm Nov 14, 2011
Monthly Robots.txt Request From securityspace.com
5 incrediBILL 7:29 pm Nov 11, 2011
followthatpage
Not on my site!
8 Mokita 7:27 am Nov 9, 2011