Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| CFNetwork/1.1 useragent |
2 | JAB_Creations | 4:07 pm Sep 14, 2005 | ||||
| Weirdly formated UA requested robots.txt |
2 | Dijkgraaf | 4:06 am Sep 14, 2005 | ||||
| Vision Research Lab |
8 | wilderness | 5:46 pm Sep 8, 2005 | ||||
| Help me with my .htaccess file please |
2 | a_angova | 3:07 am Sep 8, 2005 | ||||
| 200 (return error code 0) - What does it means? |
2 | Shaundm | 1:56 pm Sep 7, 2005 | ||||
| envolk[ITS]spider |
4 | wilderness | 1:51 pm Sep 7, 2005 | ||||
| JRSocketEngine? |
4 | teasers | 3:33 am Sep 7, 2005 | ||||
| SlurpConfirm404 Is this really Slurp? |
4 | fish_eye | 6:34 pm Sep 6, 2005 | ||||
| Strange UA |
4 | wilderness | 1:31 am Sep 5, 2005 | ||||
| Mozilla/4.0 (compatible; prgrabber 1.0) Is this a Google PageRank checker? |
2 | GaryK | 6:17 pm Sep 4, 2005 | ||||
| Amazon Crawling aws-alexa libwww-perl/5.65 |
9 | The_Contractor | 5:37 pm Sep 4, 2005 | ||||
| msnbot/1.0-MM (+http://search.msn.com/msnbot.htm) Comming from 65.54.164.### |
4 | caspita | 2:39 pm Sep 2, 2005 | ||||
| Yahoo Spider consuming large amounts of bandwidth |
3 | servoweb | 3:09 pm Sep 1, 2005 | ||||
| Checkbot and LWP University uses to check links |
2 | guitaristinus | 3:05 pm Aug 31, 2005 | ||||
| InfociousBot (+http://corp.infocious.com/tech_crawler.php) and another bad bot |
3 | DoppyNL | 1:32 pm Aug 30, 2005 | ||||
| Ocp |
4 | wilderness | 2:08 am Aug 30, 2005 | ||||
| Yahoo spidering from new Class C subnet |
5 | volatilegx | 1:44 pm Aug 29, 2005 | ||||
| Unknown spiders visiting |
3 | Dave_A | 10:49 pm Aug 28, 2005 | ||||
| double slashes |
2 | wilderness | 7:01 pm Aug 28, 2005 | ||||
| Can you help me with this IP is it spider |
2 | tomasz | 1:34 am Aug 28, 2005 | ||||
| Jakarta Commons-HttpClient/3.0-rc1 Anyone seeing this? |
18 | The_Contractor | 5:41 pm Aug 27, 2005 | ||||
| DynaWeb? Chinese proxy service |
4 | keyplyr | 5:15 pm Aug 27, 2005 | ||||
| SquigglebotBot/1.0 Is this related to Relevare? |
3 | GaryK | 5:06 pm Aug 27, 2005 | ||||
| Jayde |
3 | wilderness | 11:06 am Aug 26, 2005 | ||||
| YahooSeeker-Testing/v3.9 testing what ... I wonder? |
6 | bobothecat | 1:24 pm Aug 25, 2005 |
