Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Exodus (CW) spoofing Google Exodus via Cable & Wireless spoofing User Agent |
2 | wormdrive | 8:47 am Oct 20, 2003 | ||||
| No IP logged - just dot: . - - hidden IP - kuloko-bot/0.2 |
2 | krzys | 1:31 am Oct 20, 2003 | ||||
| Mediapartners-Google/2.1grabbing a 3.8 MB MP3 file? |
2 | rfgdxm1 | 10:07 pm Oct 19, 2003 | ||||
| Http4u by Ph. Jounin and L. Le Bras |
2 | Audun | 11:22 am Oct 19, 2003 | ||||
| WebFilter from a different IP |
2 | MarieC | 7:51 pm Oct 18, 2003 | ||||
| Someone on .gov using larbin unsuccessfully trying to crawl |
2 | MarieC | 3:58 pm Oct 18, 2003 | ||||
| www.bravespider.com referral from 209.152.137.9 |
6 | Peeress | 11:50 am Oct 18, 2003 | ||||
| Looksmart using spider with no user-agent for MSN? A Looksmart spider came that had an IE user-agent... |
10 | WileE | 5:33 am Oct 18, 2003 | ||||
| CerberianDrtrs/Version-3.0-Build-30 65.245.128.82 - Colocate America |
4 | jim_w | 3:22 pm Oct 17, 2003 | ||||
| WebSauger 1.20b? Couldn't find any info on the web... |
3 | abates | 3:49 am Oct 17, 2003 | ||||
| Mediapartners-Google? MimeLive Client? and others What do you say about that spiders? |
7 | silverbytes | 11:20 pm Oct 16, 2003 | ||||
| Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 ) spider comes from MS IP-addresses |
4 | RonPK | 5:00 pm Oct 16, 2003 | ||||
| vscooter |
5 | wilderness | 3:58 pm Oct 16, 2003 | ||||
| SE bots |
2 | tmftobes | 11:05 am Oct 16, 2003 | ||||
| Mozilla/5.0 (Version: 1334 Type:216) seemingly random numerals |
6 | bird | 5:24 am Oct 16, 2003 | ||||
| Harmfulness Robot |
3 | BlueSky | 3:24 am Oct 16, 2003 | ||||
| Crawl_Application (from IBM?) What's the latest? |
4 | pendanticist | 2:53 am Oct 16, 2003 | ||||
| Can spiders crawl members only pages? |
5 | AjiNIMC | 9:29 pm Oct 15, 2003 | ||||
| Mozilla/4.0 hhjhj@yahoo.com it's back w/new IP |
3 | coyote | 6:54 pm Oct 15, 2003 | ||||
| 80.177.2.139 - MSIECrawler Is this legit? |
3 | rooshine | 8:00 am Oct 15, 2003 | ||||
| Space Bison/0.02 [fu] (Win67; X; SK) What's this beast |
4 | seindal | 11:45 pm Oct 14, 2003 | ||||
| How do i make a simple bot trap |
3 | hafnius | 11:29 pm Oct 14, 2003 | ||||
| Bot hitting alot |
13 | Rivethed | 3:35 am Oct 14, 2003 | ||||
| Help explain unusual GET requests from spider/downloader GET /' and GET /MSOffice/cltreq.asp? |
10 | Wizcrafts | 1:22 am Oct 14, 2003 | ||||
| New bot came to see me -- WebSurvey Java ? |
2 | guillermo5000 | 1:08 am Oct 14, 2003 |