Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
findlinks bot
130.83.167.153
4 Bewenched 8:00 am Dec 4, 2012
Yasni
10 incrediBILL 3:03 pm Dec 3, 2012
Site Rating System
3 Ken_S 2:22 am Dec 3, 2012
ChinaCache North America
17 incrediBILL 8:57 pm Dec 1, 2012
Definitive Sources of IP Lists
9 incrediBILL 8:42 pm Dec 1, 2012
More virtual servers
Linode Cloud
7 Bewenched 9:31 pm Nov 30, 2012
Interesting visit from Google
Merchant Center?
3 Bewenched 8:41 pm Nov 29, 2012
Rogue .pdf grabber
85.73.112.xx
5 Bewenched 3:44 pm Nov 27, 2012
Big Fluffy Cloud
cloud servers that is
5 Bewenched 2:25 am Nov 27, 2012
More Chinanet
119.128.0.0 [2] ( 1 2 )
36 not2easy 7:29 pm Nov 26, 2012
GET Request
6 Ken_S 10:03 pm Nov 25, 2012
76.74.153.xx
no robots.txt , blank user agent
10 Bewenched 8:21 pm Nov 25, 2012
Ongoing ia archiver Abuse From China
4 incrediBILL 6:52 pm Nov 25, 2012
filterdb.iss.net/crawler/
194.153.113.35
6 Bewenched 11:38 pm Nov 24, 2012
Fake Google or code.google.com ?
89.164.163.72
8 Bewenched 10:35 pm Nov 24, 2012
Spinn3r+3.1
174.36.228.xxx
2 Bewenched 9:14 pm Nov 24, 2012
Mailru-net
stopped by snagging images only.
8 Bewenched 9:48 pm Nov 22, 2012
Data Foundry
2 keyplyr 6:40 am Nov 21, 2012
Request for same image via various wireless networks
Mozilla/4.0, No-Referrer, Repeated 1-3 image requests
5 mslina2002 3:14 am Nov 21, 2012
headers check
10 wilderness 12:56 am Nov 21, 2012
Nobis/Ubiquity
4 keyplyr 12:12 am Nov 21, 2012
what's wrong with UA
suspicious UA
6 dupres01 2:48 pm Nov 18, 2012
HiChina
5 keyplyr 3:37 am Nov 18, 2012
BeetleBot
188.207.200.116
6 Bewenched 1:47 am Nov 18, 2012
Bingbot with one domain and two IPs
What's the point of that?
10 moTi 3:18 am Nov 16, 2012