Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Voxel Cloud
3 keyplyr 11:44 pm Nov 8, 2011
Adhost
3 keyplyr 11:41 pm Nov 8, 2011
domainsoutlook.com good, you may rely on it
see magic 8 ball for more information
3 incrediBILL 4:29 am Nov 7, 2011
robot97, robot99, please report to statshow.com
oh goody it's on godaddy!
6 incrediBILL 1:48 am Nov 7, 2011
zitetrendz.com izt vatching you!
3 incrediBILL 12:26 am Nov 7, 2011
hostnology.in's domain scraper site
9 incrediBILL 12:24 am Nov 7, 2011
dawhois.com and their qwest for my site data
2 incrediBILL 11:26 pm Nov 6, 2011
tweetmeme scraping update
2 incrediBILL 11:14 pm Nov 6, 2011
clearwebstats.com and mostly sunny
2 incrediBILL 11:11 pm Nov 6, 2011
whoare.us who who, who who!
we know who you are too!
3 incrediBILL 10:43 pm Nov 6, 2011
Influencebot
2 tangor 10:24 pm Nov 6, 2011
w3who.net's hidden crawler, crouching user agent
7 incrediBILL 10:21 pm Nov 6, 2011
PagePeeker
2 Pfui 9:43 pm Nov 5, 2011
unister-bot
4 keyplyr 9:41 pm Nov 5, 2011
domain tools, surveybot, IP update
3 incrediBILL 2:16 am Nov 5, 2011
shell.com
2 Pfui 7:23 pm Nov 3, 2011
GlueText
7 Pfui 7:10 pm Nov 3, 2011
Iptcbot
5 JAB_Creations 9:39 pm Nov 2, 2011
webhost.com
5 keyplyr 9:34 pm Nov 2, 2011
LinkedInBot
7 Pfui 10:42 pm Nov 1, 2011
206.16.163.38
a.k.a. canonresourcecenter.com
5 Pfui 3:03 am Nov 1, 2011
Mail.RU/2.0
9 keyplyr 10:43 pm Oct 31, 2011
Cisco bot
4 Pfui 12:04 am Oct 31, 2011
Mozilla/5.0 (tarakas)
Kaspersky bot-running from Russian Fed.
10 Pfui 3:19 am Oct 30, 2011
seorank.com
You name it, they fake it
2 Pfui 10:54 pm Oct 29, 2011