Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| MediaPartners spider Ban this one |
2 | Digger | 7:42 pm Jul 23, 2003 | ||||
| Gaisbot/3.0 info? |
5 | coyote | 10:43 am Jul 23, 2003 | ||||
| he is back look UE - Mediapartners-Google/2.1 he is back look UE - Mediapartners-Google/2.1 |
2 | StopSpam | 10:22 pm Jul 22, 2003 | ||||
| An aggressive spider - does anyone know it? |
6 | mrdch | 6:15 pm Jul 22, 2003 | ||||
| MSIE Crawler More information requested |
3 | loanuniverse | 4:08 pm Jul 22, 2003 | ||||
| Google and WAP |
6 | wilderness | 3:05 am Jul 22, 2003 | ||||
| who/what does mozilla stand for |
12 | Blelisa | 8:08 pm Jul 21, 2003 | ||||
| LWP::Simple heads up |
9 | wilderness | 7:23 pm Jul 21, 2003 | ||||
| New UA: CopyHunter/1.54 Has anybody else seen it and know its purpose? |
3 | Wizcrafts | 10:42 am Jul 21, 2003 | ||||
| SetEnvIf ... keep_out (Apache directive) |
3 | Made_In_Sheffield | 10:18 pm Jul 19, 2003 | ||||
| Cyveillance IP Ranges What are the IP Ranges for Cyveillance |
9 | kewlbeezer | 7:30 pm Jul 18, 2003 | ||||
| Willow Internet Crawler twotrees.com |
2 | bull | 9:03 pm Jul 17, 2003 | ||||
| Analyze your log: how to exclude yourself? How to avoid being counted? Your own visits to your website... |
6 | silverbytes | 9:33 pm Jul 16, 2003 | ||||
| Whois this guy?- 12.119.251.194 GOT index 670 times so far today |
16 | Powdork | 12:43 am Jul 16, 2003 | ||||
| A Simple Beginning with htaccess |
3 | wilderness | 11:58 pm Jul 15, 2003 | ||||
| Mediapartners-Google/2.1 But no adsense/adwords or similar |
3 | pixel_juice | 9:34 pm Jul 15, 2003 | ||||
| Chinese bot from 61.144.218.161 Anyone've heard of this Chinese bot? |
2 | aleksl | 1:11 pm Jul 15, 2003 | ||||
| guestbook spammers webbots that are sniffing around guestbook pages |
23 | Jaf | 12:21 pm Jul 15, 2003 | ||||
| Some Google hints 216.33.229.163 and Python again |
8 | pontifex | 8:56 pm Jul 14, 2003 | ||||
| 65.110.131.177 one hungry bot |
2 | coyote | 11:48 am Jul 14, 2003 | ||||
| Look out GoogleBot! Look out MSNBot! Here comes GalaxyBot! New bot from galaxy.com |
7 | Made_In_Sheffield | 11:11 am Jul 14, 2003 | ||||
| WebmasterWorldWebBot on 193.95.242.56 |
7 | jimmykav | 11:35 am Jul 12, 2003 | ||||
| 209.189.108.196 oingo.com Requested disallowed files |
5 | upside | 10:28 pm Jul 11, 2003 | ||||
| 210.196.42.200 Asia Pacific Network Information Centre |
4 | photoman | 6:20 pm Jul 11, 2003 | ||||
| inktomi slurp robot. si1006.inktomisearch.com |
2 | jackobar | 3:07 pm Jul 10, 2003 |
