Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
botnet with same user agent targeting specific pages
29 wilderness 10:52 pm Jun 5, 2012
Forum and Form spammer.
2 g1smd 1:38 am Jun 4, 2012
covert Microsoft crawl
6 keyplyr 12:46 am Jun 3, 2012
Nokia bad behaviour
Is this a cloud, a bot or a proxy?
3 dstiles 6:36 pm Jun 2, 2012
New Blekkobot User Agent String
scoutjet / blekkobot
11 dstiles 9:04 pm May 30, 2012
Bing plain clothes
obnoxious pest
2 Staffa 12:57 am May 30, 2012
Spider Forum Thread Titles
16 incrediBILL 10:17 pm May 29, 2012
Bad Spiders and Proxy Servers
Using Opera Proxy
19 grandma_genie 7:25 am May 27, 2012
HEAD request for favicon
"Your favicon? What would it want with me?"
2 lucy24 2:44 am May 25, 2012
url include
12 wilderness 8:40 pm May 23, 2012
"Capture Extractor" Botnet Crawling Observed
2 incrediBILL 3:48 pm May 23, 2012
Injection Attempt From Microlink IPs
4 keyplyr 8:59 pm May 22, 2012
Scrapers Using Peak Web Hosting and Generic Browser User Agents
4 keyplyr 10:53 pm May 19, 2012
deny all
15 wilderness 3:50 pm May 14, 2012
Yahoo SE
2 wilderness 1:46 pm May 14, 2012
Bing blows a gasket
9 wilderness 7:57 pm May 13, 2012
PycURL
4 wilderness 7:00 pm May 13, 2012
Cookie based spider catcher
6 motorhaven 11:16 pm May 12, 2012
Been getting 200+ page loads per day from same IP
recorded by js based analytics
10 scooterdude 7:01 pm May 12, 2012
Digital AlphaServer User Agent
3 wilderness 6:26 pm May 12, 2012
Bright House Networks
3 wilderness 11:21 pm May 10, 2012
HopOne
10 keyplyr 9:17 pm May 10, 2012
MediaTemple
3 keyplyr 8:58 pm May 10, 2012
Multiple User Agents, same IP
Is this a bot?
17 grandma_genie 8:49 pm May 10, 2012
User Agent "-"
missing User Agent
8 dupres01 8:01 pm May 9, 2012