Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Google Keyword Suggestion
2 Pfui 7:15 pm Oct 1, 2011
swish-e
2 Pfui 4:20 am Oct 1, 2011
AndroidDownloadManger
...and other iterations
4 tangor 1:53 am Oct 1, 2011
MSN bot hitting hard without a referrer
65.52.104.27
10 SEOPTI 12:00 am Sep 29, 2011
vBSEO User Agent
Question about referer
4 grandma_genie 6:11 pm Sep 28, 2011
Why I'm Somewhat MIA
[2] ( 1 2 )
31 incrediBILL 5:56 pm Sep 28, 2011
FlipboardProxy
At least 2 versions; 2 cases (2 much)
3 Pfui 12:20 pm Sep 27, 2011
Kavende Iranian Crawler
4 g1smd 2:33 pm Sep 25, 2011
today's head-scratcher
8 lucy24 2:05 am Sep 25, 2011
Gootkit auto-rooter scanner
Malicious virus implanter from dynamic IPs
6 dstiles 10:15 pm Sep 24, 2011
DIY Botnet Kit
It gets worse!
3 dstiles 4:58 pm Sep 24, 2011
amazonaws.com plays host to wide variety of bad bots
Most recently seen: Gnomit[10] ( 1 2 3 4 5 6 7 8 9 10 )
278 Pfui 10:30 pm Sep 23, 2011
.htaccess file for blocking Scraper/Spam Bots
Htaccess File to Help with blocking Scraper/Spam Bots
5 sandman22 7:59 pm Sep 22, 2011
Feedfetcher every 40 minutes for two days
feedfetcher persistently looking for non existent item
17 revrob 9:22 pm Sep 21, 2011
Grafix-is
6 keyplyr 8:05 pm Sep 20, 2011
Level 3 redux
6 Pfui 3:25 pm Sep 19, 2011
Resolving "urlresolver"
Google IPs repeat no-robots runs
30 Pfui 4:55 am Sep 19, 2011
Netscape 5.0 is driving me nuts
Netscape 5.0 showing on logs
9 walkman 9:00 pm Sep 16, 2011
Does everyone use bespoke scripts for spider identification
6 scooterdude 9:47 pm Sep 14, 2011
Sitebuilder name in results-spammers thru a backdoor?
Sitebuilder in search results-block "backdoor"
5 KaraScene1 4:24 pm Sep 14, 2011
Traffic from Malicious IPs and Adwords ads for my website
I get visitors from adwords when I never started a campaign
5 Makis77 8:45 pm Sep 13, 2011
isc header collector
2 Pfui 9:30 pm Sep 9, 2011
Iparadigms
8 keyplyr 7:01 pm Sep 3, 2011
Why is this agent fetching my images?
7 aristotle 4:58 pm Sep 3, 2011
PostPost
2 Pfui 1:52 pm Sep 3, 2011