Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
what's wrong with UA
suspicious UA
6 dupres01 2:48 pm Nov 18, 2012
HiChina
5 keyplyr 3:37 am Nov 18, 2012
BeetleBot
188.207.200.116
6 Bewenched 1:47 am Nov 18, 2012
Bingbot with one domain and two IPs
What's the point of that?
10 moTi 3:18 am Nov 16, 2012
CompSpyBot
68.47.129.55
8 Bewenched 9:51 pm Nov 15, 2012
adbeat.com
174.129.111.51
2 Bewenched 10:48 pm Nov 14, 2012
Educational Unit
took 52 files - one image only - the contact image
13 Ken_S 5:17 am Nov 14, 2012
www.integromedb.org/Crawler
2 Bewenched 12:43 am Nov 14, 2012
the art-loving iPad
8 lucy24 6:36 pm Nov 12, 2012
Colocation
4 Ken_S 12:57 pm Nov 12, 2012
Mythic Beasts Test Netblock
6 keyplyr 7:57 pm Nov 11, 2012
Remote host
3 wilderness 10:53 am Nov 11, 2012
Jbot
3 keyplyr 3:03 am Nov 11, 2012
Korea KRNIC
4 Bewenched 9:23 pm Nov 10, 2012
Image thief
176.62.49.50
3 Bewenched 2:38 pm Nov 10, 2012
Proactive vs. Reactive IP Range Blocking
General Policy or Behavior Based
7 incrediBILL 9:36 pm Nov 9, 2012
Amazon motherload
4 Bewenched 11:48 pm Nov 8, 2012
MaxPointCrawler / maxpointinteractive.com
208.123.79.68
4 Bewenched 3:17 pm Nov 7, 2012
Wowrack
10 keyplyr 9:11 am Nov 7, 2012
EasouSpider
4 keyplyr 6:58 am Nov 7, 2012
odd behavior
216.38.216.101
5 Bewenched 6:34 am Nov 7, 2012
Strange Inktomi Corporation IP
11 SEOPTI 1:44 pm Nov 6, 2012
orchestr8
66.35.60.158
5 Bewenched 7:32 am Nov 6, 2012
Does amazon have a crawler or are all Amazon AWS IPs fair game?
9 bigtoga 6:57 am Nov 6, 2012
New google IP range
173.194/16
19 dstiles 1:03 pm Nov 5, 2012