Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| msnbot/1.0-MM (+http://search.msn.com/msnbot.htm) Comming from 65.54.164.### |
4 | caspita | 2:39 pm Sep 2, 2005 | ||||
| Yahoo Spider consuming large amounts of bandwidth |
3 | servoweb | 3:09 pm Sep 1, 2005 | ||||
| Checkbot and LWP University uses to check links |
2 | guitaristinus | 3:05 pm Aug 31, 2005 | ||||
| InfociousBot (+http://corp.infocious.com/tech_crawler.php) and another bad bot |
3 | DoppyNL | 1:32 pm Aug 30, 2005 | ||||
| Ocp |
4 | wilderness | 2:08 am Aug 30, 2005 | ||||
| Yahoo spidering from new Class C subnet |
5 | volatilegx | 1:44 pm Aug 29, 2005 | ||||
| Unknown spiders visiting |
3 | Dave_A | 10:49 pm Aug 28, 2005 | ||||
| double slashes |
2 | wilderness | 7:01 pm Aug 28, 2005 | ||||
| Can you help me with this IP is it spider |
2 | tomasz | 1:34 am Aug 28, 2005 | ||||
| Jakarta Commons-HttpClient/3.0-rc1 Anyone seeing this? |
18 | The_Contractor | 5:41 pm Aug 27, 2005 | ||||
| DynaWeb? Chinese proxy service |
4 | keyplyr | 5:15 pm Aug 27, 2005 | ||||
| SquigglebotBot/1.0 Is this related to Relevare? |
3 | GaryK | 5:06 pm Aug 27, 2005 | ||||
| Jayde |
3 | wilderness | 11:06 am Aug 26, 2005 | ||||
| YahooSeeker-Testing/v3.9 testing what ... I wonder? |
6 | bobothecat | 1:24 pm Aug 25, 2005 | ||||
| Oracle Ultra Search no robots |
3 | bull | 2:43 pm Aug 23, 2005 | ||||
| Becomebot |
4 | reaper | 3:27 pm Aug 22, 2005 | ||||
| DTAAgent |
3 | bumpaw | 2:26 am Aug 22, 2005 | ||||
| Thank you Yahoo SlurpConfirm404 |
17 | fiestagirl | 2:06 am Aug 22, 2005 | ||||
| Odd UA? |
3 | mcneely | 12:33 am Aug 22, 2005 | ||||
| Multiple UA's |
2 | wilderness | 7:44 pm Aug 21, 2005 | ||||
| Helix/1.2 |
3 | bull | 12:16 pm Aug 21, 2005 | ||||
| Poodle_predictor_1.0 For the dogs or what? |
6 | mcneely | 12:30 pm Aug 20, 2005 | ||||
| OmniExplorer Crawler Should I continue to let this spider hammer my sites? |
3 | FlipFlops | 6:10 am Aug 20, 2005 | ||||
| What's your spider traffic look like? |
4 | walnut99 | 6:03 am Aug 20, 2005 | ||||
| What is agent::_detectURL? |
2 | Joey123 | 7:09 pm Aug 19, 2005 |