Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
fake google bot?
6 pp46 11:19 am Jan 2, 2007
LinkWalker/2.0
Anyone knows what this bot does?
3 ibnuasad 9:29 pm Jan 1, 2007
bot/1.0 (bot; http://; bot@bot.bot)
7 Mokita 7:41 am Dec 29, 2006
Yodaobot 1.0 is coming
Do you notice the Outfoxbot 0.x? Last weekend it released as Yodaobot 1.0
16 chedong 6:04 pm Dec 28, 2006
Exabot-Test/1.0
2 GaryK 6:35 am Dec 25, 2006
Yahoo invents browser that doesn't need an OS!
Y! sitemap checker
2 jdMorgan 5:05 pm Dec 22, 2006
IE7 Save Site UA?
7 keyplyr 9:58 am Dec 21, 2006
User-Agent begins with "User Agent::"
4 Umbra 12:51 am Dec 19, 2006
WorQmada/1.0
6 bobothecat 6:02 am Dec 18, 2006
Bigsearch.ca
new Canadian search engine currently Beta
3 Mokita 2:58 am Dec 16, 2006
MSNBot from APNIC?
Is this a spoofed user-agent?
5 Kadence 7:53 pm Dec 15, 2006
MSNPTC/1.0 Bot Gone Wild
MSN's AdCenter Bot
4 speakSEM 7:31 pm Dec 14, 2006
new disguise of rippers?
8 the_nerd 3:37 am Dec 14, 2006
Analyzing requests on every pageview vs. Every 1 minute
Pros & cons...
8 MichaelBluejay 2:34 am Dec 14, 2006
HMSE Robot
Not a clue what it is.
3 GaryK 1:02 am Dec 14, 2006
New Search Engine Spider IPs
4 volatilegx 3:31 pm Dec 13, 2006
Nokia6600
14 wilderness 10:37 pm Dec 10, 2006
Yahoo Bots
8 sem4u 11:02 am Dec 6, 2006
Amazon
3 wilderness 3:32 pm Dec 2, 2006
Is Ebay attempting wholesale site downloads?
Regular attempts from the EBay Network to download my entire site.
24 AlexK 12:32 am Nov 30, 2006
masidani bot v0.6(www.masidani.com)
Holiday shopping bot
3 GaryK 12:18 am Nov 27, 2006
Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/delete ma
New UA for Naver
3 GaryK 12:15 am Nov 27, 2006
ICRA label generator
ICRA_Semantic_spider
6 GaryK 8:34 pm Nov 26, 2006
SirSeek
badly malformed UA
2 GaryK 8:07 pm Nov 26, 2006
"CMS crawler"
their URL gives "Access Denied"
6 Mokita 7:04 pm Nov 26, 2006