Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| VSynCrawler |
2 | keyplyr | 12:24 am Feb 6, 2007 | ||||
| DRT-ResolveBot-Ignore |
3 | wilderness | 12:06 am Feb 6, 2007 | ||||
| Updated Ask UA Has anyone else seen this? |
11 | GaryK | 7:37 pm Feb 5, 2007 | ||||
| UbiCrawler/v0.5beta |
3 | EarWig | 10:18 pm Feb 4, 2007 | ||||
| Up-to-date bot identification? |
21 | Reno | 8:20 pm Feb 3, 2007 | ||||
| bot or inline advertising? |
6 | incrediBILL | 8:16 pm Feb 3, 2007 | ||||
| Jakarta HTTP Client/1.0 |
5 | keyplyr | 4:01 pm Feb 3, 2007 | ||||
| These google things in my bot trap Spambots or real ones? |
6 | silverbytes | 9:29 pm Feb 2, 2007 | ||||
| niXXiebot Takes robots.txt before every page. |
4 | GaryK | 5:15 am Feb 1, 2007 | ||||
| Honeynets: Trapping attackers and naming names |
18 | Brett_Tabke | 2:25 am Jan 31, 2007 | ||||
| Bot with blank referrer and UA - Microsoft IP It's back.. |
2 | LunaC | 12:52 am Jan 30, 2007 | ||||
| The IRS Bots |
15 | Brett_Tabke | 3:10 pm Jan 29, 2007 | ||||
| Pingdom How to stop them |
7 | james96 | 1:29 pm Jan 29, 2007 | ||||
| gugli-bot |
4 | keyplyr | 1:49 am Jan 28, 2007 | ||||
| Is this some sort of IE downloader? |
5 | SEOPTI | 11:01 pm Jan 27, 2007 | ||||
| AppleSyndication/54 Is this from Apple? |
2 | bouncybunny | 2:17 am Jan 27, 2007 | ||||
| IP ranges for Good Bots? Avoiding ban good bots, ip ranges |
6 | silverbytes | 3:15 pm Jan 26, 2007 | ||||
| Taiga web spider |
17 | wilderness | 8:51 pm Jan 25, 2007 | ||||
| Bot coming from Microsoft IPs |
5 | Mokita | 8:34 pm Jan 25, 2007 | ||||
| Podtech Network crawler |
6 | Mokita | 7:39 pm Jan 24, 2007 | ||||
| Dealing with the rogue bots and botnets |
10 | incrediBILL | 2:09 am Jan 24, 2007 | ||||
| Morfeus #*$!ing Scanner Spam scanner or referral bomber? |
5 | Wemic | 1:07 am Jan 23, 2007 | ||||
| HTTP::Lite/2.1.6 |
2 | wilderness | 7:58 am Jan 22, 2007 | ||||
| Is there a way to combat this junk? Mozilla/4.0 (compatible ; MSIE 6.0; Windows NT 5.1) |
2 | SEOPTI | 2:00 am Jan 22, 2007 | ||||
| Garbage user agent srings and changing IP addreses Impossible to block? |
12 | bouncybunny | 10:59 pm Jan 21, 2007 |