Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| "Pita" spider Has anyone seen this one? |
6 | nafmo | 12:35 am Jun 22, 2003 | ||||
| The economics of spidering There has to be a mutual benefit here |
2 | webdevsf | 11:20 pm Jun 21, 2003 | ||||
| Msnbot/0.1 First time I've seen this one. |
11 | volatilegx | 3:27 pm Jun 21, 2003 | ||||
| ua: Hello Now this one is nasty |
4 | FineWare | 12:05 pm Jun 21, 2003 | ||||
| Automatic identification/blocking |
2 | ephka | 5:40 pm Jun 20, 2003 | ||||
| InternetSeer.com should i block this |
4 | sajjid | 12:13 am Jun 20, 2003 | ||||
| False agent-name and False IP A real pain in the....... |
15 | sanuk | 11:00 am Jun 19, 2003 | ||||
| pleasdo as user agent? any ideas? |
2 | Clark | 9:34 pm Jun 18, 2003 | ||||
| Canadian Crawler Hammered my site Bell Nexxia? |
5 | kewlbeezer | 10:00 pm Jun 17, 2003 | ||||
| interseer or InterSeer using NC and OR is ok? |
4 | sanuk | 8:57 pm Jun 17, 2003 | ||||
| 13 | DavidT | 8:45 pm Jun 17, 2003 | |||||
| Checkbot/1.71 New bot? I couldn't find it in WW site search |
9 | berli | 1:37 am Jun 17, 2003 | ||||
| Another New Crawler - kljwdw3gfdrfdsfdsdfd Looks like someone was lazy when they made this name... |
3 | kewlbeezer | 10:29 pm Jun 16, 2003 | ||||
| trapping spiders - Google ban result? making a spider trap |
8 | sanuk | 10:27 pm Jun 16, 2003 | ||||
| Missouri College Browse |
4 | ned911 | 10:20 pm Jun 16, 2003 | ||||
| Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3a) Gecko/20021212 Is this Alta Vista? |
6 | GaryK | 10:35 am Jun 15, 2003 | ||||
| what's this: "GET /watch-info HTTP/1.0" 403 |
6 | meinereiner | 10:29 am Jun 15, 2003 | ||||
| Anyone Know Which Spider this is? Got hit hard by them this morning |
16 | cmjohnston | 10:05 am Jun 14, 2003 | ||||
| Singingfish extractor not obeying robots.txt sound files hosted at different domain than website |
3 | Finder | 12:51 am Jun 14, 2003 | ||||
| YahooBB220048191035.bbtec.net whats yahoo doin on my site lol |
3 | penfold25 | 5:11 pm Jun 13, 2003 | ||||
| toCrawl/UrlDispatcher another one |
8 | sanuk | 11:55 am Jun 13, 2003 | ||||
| Two CJ spiders Two CJ Spiders |
3 | yogsco | 5:06 am Jun 13, 2003 | ||||
| Searchalot |
6 | wilderness | 5:21 pm Jun 12, 2003 | ||||
| Genie What is this? |
6 | Oaf357 | 3:53 am Jun 12, 2003 | ||||
| Vivante ain't coming to *my* lifestyle, I guarantee. |
2 | SeanL | 3:23 am Jun 12, 2003 |
