Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Session IDs Session IDs or not |
2 | chavezr | 4:08 pm Oct 24, 2003 | ||||
| Msnia New? |
4 | ganderla | 5:00 am Oct 24, 2003 | ||||
| voilabot beta 1.2 need to ban, but don't see reques for robots.txt |
4 | macrost | 9:02 pm Oct 23, 2003 | ||||
| kuloko-bot crawling hard |
6 | creative_craig | 2:46 pm Oct 23, 2003 | ||||
| Fbot/1.1 http*//www.fetchbook.info/bot.jsp |
2 | pendanticist | 12:01 pm Oct 23, 2003 | ||||
| dakkastats strange behavior |
2 | SeanL | 9:08 pm Oct 22, 2003 | ||||
| Research Crawler appears to be ok |
6 | coyote | 7:38 pm Oct 22, 2003 | ||||
| CHIP Explorer HU what's this? |
3 | coyote | 4:24 am Oct 21, 2003 | ||||
| Microsoft Internet Explorer 5.5 Java/? what's this? |
3 | keyplyr | 4:20 am Oct 21, 2003 | ||||
| icsbot-0.1 Any Clues |
3 | xcandyman | 7:01 pm Oct 20, 2003 | ||||
| Slammed by 12.148.209.198 |
4 | King_of_Bling | 1:21 pm Oct 20, 2003 | ||||
| Exodus (CW) spoofing Google Exodus via Cable & Wireless spoofing User Agent |
2 | wormdrive | 8:47 am Oct 20, 2003 | ||||
| No IP logged - just dot: . - - hidden IP - kuloko-bot/0.2 |
2 | krzys | 1:31 am Oct 20, 2003 | ||||
| Mediapartners-Google/2.1grabbing a 3.8 MB MP3 file? |
2 | rfgdxm1 | 10:07 pm Oct 19, 2003 | ||||
| Http4u by Ph. Jounin and L. Le Bras |
2 | Audun | 11:22 am Oct 19, 2003 | ||||
| WebFilter from a different IP |
2 | MarieC | 7:51 pm Oct 18, 2003 | ||||
| Someone on .gov using larbin unsuccessfully trying to crawl |
2 | MarieC | 3:58 pm Oct 18, 2003 | ||||
| www.bravespider.com referral from 209.152.137.9 |
6 | Peeress | 11:50 am Oct 18, 2003 | ||||
| Looksmart using spider with no user-agent for MSN? A Looksmart spider came that had an IE user-agent... |
10 | WileE | 5:33 am Oct 18, 2003 | ||||
| CerberianDrtrs/Version-3.0-Build-30 65.245.128.82 - Colocate America |
4 | jim_w | 3:22 pm Oct 17, 2003 | ||||
| WebSauger 1.20b? Couldn't find any info on the web... |
3 | abates | 3:49 am Oct 17, 2003 | ||||
| Mediapartners-Google? MimeLive Client? and others What do you say about that spiders? |
7 | silverbytes | 11:20 pm Oct 16, 2003 | ||||
| Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 ) spider comes from MS IP-addresses |
4 | RonPK | 5:00 pm Oct 16, 2003 | ||||
| vscooter |
5 | wilderness | 3:58 pm Oct 16, 2003 | ||||
| SE bots |
2 | tmftobes | 11:05 am Oct 16, 2003 |
