Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Unvoluntary banning googles sitemap crawler?
What the bot name or ip is?
3 silverbytes 7:52 pm Mar 2, 2007
Possible Bot or Spammer?
Or is this Live bot?
11 EarleyGirl 9:49 pm Feb 28, 2007
HoneyNet White Paper : Know your Enemy: Web Application Threats
3 Brett_Tabke 7:36 pm Feb 28, 2007
OT: convoluted UA
2 Mokita 6:17 pm Feb 26, 2007
Java/1.x.x
Ban them or ignore them?
9 bouncybunny 9:29 pm Feb 18, 2007
Web Accelerators
Characteristics?
2 vortech 1:44 am Feb 15, 2007
WinkBot
2 wilderness 11:27 pm Feb 12, 2007
"CiteBite" - will duplicate your content for you
No UA, no robots.txt
9 zCat 7:07 am Feb 12, 2007
Amazon
13 wilderness 10:43 am Feb 11, 2007
Botnet 'pandemic' threatens internet
[2] ( 1 2 )
37 Brett_Tabke 3:02 am Feb 9, 2007
SnapPreviewBot
10 keyplyr 2:48 am Feb 9, 2007
VSynCrawler
2 keyplyr 12:24 am Feb 6, 2007
DRT-ResolveBot-Ignore
3 wilderness 12:06 am Feb 6, 2007
Updated Ask UA
Has anyone else seen this?
11 GaryK 7:37 pm Feb 5, 2007
UbiCrawler/v0.5beta
3 EarWig 10:18 pm Feb 4, 2007
Up-to-date bot identification?
21 Reno 8:20 pm Feb 3, 2007
bot or inline advertising?
6 incrediBILL 8:16 pm Feb 3, 2007
Jakarta HTTP Client/1.0
5 keyplyr 4:01 pm Feb 3, 2007
These google things in my bot trap
Spambots or real ones?
6 silverbytes 9:29 pm Feb 2, 2007
niXXiebot
Takes robots.txt before every page.
4 GaryK 5:15 am Feb 1, 2007
Honeynets: Trapping attackers and naming names
18 Brett_Tabke 2:25 am Jan 31, 2007
Bot with blank referrer and UA - Microsoft IP
It's back..
2 LunaC 12:52 am Jan 30, 2007
The IRS Bots
15 Brett_Tabke 3:10 pm Jan 29, 2007
Pingdom
How to stop them
7 james96 1:29 pm Jan 29, 2007
gugli-bot
4 keyplyr 1:49 am Jan 28, 2007