Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
DuckDuckBot
9 keyplyr 4:54 pm Dec 18, 2009
NSmith / NSmitm / NutSmith / Jane Smith
2 Pfui 9:28 pm Dec 17, 2009
Spinn3r
Still ban-worthy
2 Pfui 9:06 am Dec 17, 2009
Mozilla/5.0 (compatible; LegalAnalysisAgent/1.0; http://www.#*$!
4 GaryK 5:14 pm Dec 14, 2009
baypup/colbert (Baypup; http://sf.baypup.com/webmasters; jason@baypup.
2 GaryK 4:46 pm Dec 14, 2009
Netvibes favicons proxy
3 GaryK 4:45 pm Dec 14, 2009
Stealth bot?
Same-site referers lack post-suffix slash
12 Pfui 4:40 pm Dec 14, 2009
208.43.205.234:80:::pscan
3 GaryK 4:30 pm Dec 14, 2009
On Dasher?
4 Megaclinium 3:10 am Dec 14, 2009
Facebook share follower
3 dstiles 8:45 pm Dec 12, 2009
Made by ZmEu @ WhiteHat v0.3 (www.WhiteHat.ro)
3 GaryK 7:17 am Dec 12, 2009
buddybuzz
4 Pfui 6:20 am Dec 9, 2009
YLC Test/1.0
2 GaryK 9:16 pm Dec 5, 2009
Moreoverbot/5.00 ( http://www.moreover.com; webmaster@moreover.com)
6 GaryK 8:05 pm Dec 5, 2009
AppEngine-Google; ( http://code.google.com/appengine; appid: mapthisli
8 GaryK 5:18 pm Dec 5, 2009
Wells Fargo
2 keyplyr 7:05 pm Dec 4, 2009
http://ppl.blastoffnetwork.com/investor (Blastoff Network)
6 GaryK 6:10 pm Dec 4, 2009
SheenBot
9 Pfui 3:21 pm Dec 4, 2009
ClamAV 0.95.3
7 GaryK 2:54 pm Dec 4, 2009
Mozilla/5.0 (compatible; FinancialAnalysisAgent/1.0; http://www.ifsdb.
2 GaryK 7:59 am Dec 4, 2009
Google News Spider Agent
2 engine 2:16 pm Dec 3, 2009
Multiple-IP hits
botnet activity?
4 dstiles 3:06 pm Dec 2, 2009
Blank referrer and UA with <wbr> in query
Is this some buggy software or search engine
2 enigma1 11:32 pm Dec 1, 2009
Inutile si tu respectes les standards du web
2 GaryK 8:49 pm Nov 30, 2009
New Droid UA?
5 GaryK 3:06 pm Nov 30, 2009