Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Anyone heard of Compaq-NSIS-COP/1.0? |
4 | tkroll | 4:54 pm Jul 18, 2005 | ||||
| WebIndexer/1-dev Seems to be trying to grab DMOZ listed sites.... |
4 | bobothecat | 1:03 pm Jul 18, 2005 | ||||
| Possible Yahoo bot. No user agent or HTTP_REFERER header |
4 | volatilegx | 4:47 am Jul 18, 2005 | ||||
| MS Search 4.0 Robot |
4 | bobothecat | 2:58 pm Jul 15, 2005 | ||||
| Yahoo Seeker Going Wild - Is it worth it? 3 GB a day of bandwidth for this bot? |
2 | otc_cmnn | 10:22 pm Jul 13, 2005 | ||||
| Missigua Locator |
2 | mcneely | 9:26 pm Jul 13, 2005 | ||||
| JobSpider No jobs here! |
3 | bobothecat | 3:35 am Jul 13, 2005 | ||||
| Spoofed Googlebot They didn't quite get the UA and other things right |
2 | Dijkgraaf | 3:08 am Jul 13, 2005 | ||||
| Yahoo!'s Hotjobs bot |
2 | volatilegx | 5:46 pm Jul 11, 2005 | ||||
| ccubee |
7 | wilderness | 1:45 pm Jul 11, 2005 | ||||
| Problems with Voila Can't stop it, hundreds of thousands of hits per day. |
4 | motorhaven | 2:15 pm Jul 8, 2005 | ||||
| Yadows Crawler here's a new one |
4 | volatilegx | 8:20 am Jul 7, 2005 | ||||
| Microsoft-WebDAV-MiniRedir/5.1.2600 help please |
13 | thehittmann | 10:14 am Jul 3, 2005 | ||||
| nicebot Not so nice... |
13 | bobothecat | 7:23 pm Jun 30, 2005 | ||||
| YahooSeeker-Testing/v3.9 |
3 | MaxM | 2:11 pm Jun 30, 2005 | ||||
| FAST MetaWeb Crawler Question I've never seen it before. What's your experience with it? |
6 | GaryK | 3:03 am Jun 29, 2005 | ||||
| New Name For Yahoo Bot? YahooSeeker/CafeKelsa |
4 | mmr82 | 3:23 pm Jun 27, 2005 | ||||
| nicebot? opinions, please :S New bot? calls itself 'nicebot'... |
2 | blueninja | 3:04 am Jun 27, 2005 | ||||
| tpiol.com we were hit this week by spider |
2 | TomWaits | 4:39 am Jun 26, 2005 | ||||
| Anyone familiar with brandimensions? Visitors from their site |
7 | aleksl | 4:16 am Jun 26, 2005 | ||||
| www.archive.org |
6 | OZmike | 9:00 pm Jun 24, 2005 | ||||
| Msn |
4 | wilderness | 2:02 am Jun 24, 2005 | ||||
| Convera Crawler is Posting links. |
10 | Jabba | 4:46 am Jun 18, 2005 | ||||
| Msnptc/1.0 What is this bot? |
4 | delphian | 12:37 pm Jun 16, 2005 | ||||
| Web Trends - Bots file update? |
2 | jaime1985 | 9:14 pm Jun 15, 2005 |