Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Findshare bot
Found this Russian? bot hitting my logs today.
12 grandma_genie 12:14 am Oct 12, 2010
Yahoo Nokia Symbian crawler
Is this a Yahoo crawler for mobile aps?
3 grandma_genie 1:33 am Oct 9, 2010
Catchbot
Fetches robots.txt now
3 Dijkgraaf 8:44 am Oct 7, 2010
sixapart using libwww-perl
5 Pfui 5:34 pm Oct 5, 2010
Visitor pretending to be Google bot?
Who is gandalf.volutionmedia.com?
16 grandma_genie 1:48 am Oct 3, 2010
Sosospider
Update on this spider
5 Dijkgraaf 2:41 pm Oct 1, 2010
123People
4 Dijkgraaf 11:20 am Oct 1, 2010
Is this a real MSN bot or a fake?
IP seems OK, but is overly zealous.
12 grandma_genie 6:59 pm Sep 29, 2010
TwengaBot
19 topr8 12:44 am Sep 29, 2010
Fake Slurp
but referring from Google?
8 Dijkgraaf 10:04 pm Sep 28, 2010
C4pc
12 Pfui 9:32 pm Sep 28, 2010
Windows Phone OS 7.0 handset
3 GaryK 1:33 am Sep 28, 2010
Vnetus? Is this a bot?
5 geekdogfl 10:55 am Sep 27, 2010
casper bot search attempts to infect sites
user agent changing almost daily[2] ( 1 2 )
41 dstiles 11:30 pm Sep 25, 2010
Imperia-LinkSpider
Another experimental web bot from Germany
3 Dijkgraaf 2:31 am Sep 24, 2010
bigfinder
Bad German bot
2 Dijkgraaf 11:54 am Sep 23, 2010
MSN Bot Spoofing UA as Browser
3 Web_Savvy 3:19 am Sep 23, 2010
Opera
3 Dijkgraaf 2:13 am Sep 23, 2010
SolomonoBot/1.01
4 MxAngel 10:00 pm Sep 22, 2010
BT-ctrlaltdel and Creative ZENcast
Any info on this user agent?
8 grandma_genie 12:06 am Sep 21, 2010
LinkedInBot
8 Pfui 8:23 pm Sep 20, 2010
What IPs will Googlebot access my site from?
2 DiscoStu 2:32 pm Sep 20, 2010
AppEngine-Google: proxy1china
11 Umbra 1:14 pm Sep 18, 2010
What is wmtips.com?
Possible hacker using this site?
3 grandma_genie 8:04 pm Sep 17, 2010
Website Explorer
Website Explorer/0.9.9.9
4 Dijkgraaf 5:05 pm Sep 17, 2010