Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Had an interesting visitor today 173.194.73.xxx
Google? Looks like their hosting MarkMonitor
3 Bewenched 2:15 am Oct 26, 2012
Amazon bot and friends?
Possible botnet visit?
6 grandma_genie 1:32 am Oct 26, 2012
The return of Choopa
173.199.116.xxx and 173.199.114.xxx
9 Bewenched 12:41 am Oct 26, 2012
Nobis
9 keyplyr 10:16 pm Oct 25, 2012
Hetzner Online AG Data Centers
176.9.31.xx
5 Bewenched 8:13 pm Oct 25, 2012
EGIHosting
7 keyplyr 2:42 pm Oct 25, 2012
Hurrican Electric
who are these guys
4 Bewenched 2:36 pm Oct 25, 2012
A comprehensive list of EC2 IP ranges
17 bakedjake 2:35 pm Oct 25, 2012
strange refer
6 wilderness 10:45 am Oct 24, 2012
possible botnet
15 wilderness 9:47 pm Oct 20, 2012
the return of the ahrefsbot
8 lucy24 5:10 pm Oct 20, 2012
eSyndiCat Bot
2 keyplyr 4:10 pm Oct 20, 2012
Facebook ?
69.171.xxx.x
12 Bewenched 2:59 pm Oct 20, 2012
Google Web Preview
4 wilderness 9:33 pm Oct 18, 2012
Brand Affinity Technologies
207.38.20.0 - 207.38.20.255
3 Bewenched 9:32 pm Oct 14, 2012
CareerBot
3 keyplyr 8:40 am Oct 14, 2012
item-db bot
3 keyplyr 2:19 am Oct 13, 2012
MsnBot
Cloaking - perhaps running a check on page content
5 Ken_S 10:47 pm Oct 11, 2012
ya.com?
4 incrediBILL 7:48 pm Oct 11, 2012
Yahoo Mobile access with Safari
Every sign of being a bot not a proxy
8 dstiles 1:14 am Oct 11, 2012
another for the profilers
5 lucy24 12:46 am Oct 11, 2012
47 USC § 230 - Bot Blockers Policy?
Protection for “Good Samaritan” blocking
3 incrediBILL 8:15 am Oct 10, 2012
New Amazon range?
Amazonaws
10 grandma_genie 8:32 pm Oct 9, 2012
Server under HEAD attack
2 incrediBILL 7:37 am Oct 6, 2012
ShenZhen Sunrise Technology Co
Some new bot?
5 iwillgetu 1:56 pm Oct 5, 2012