Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| 216.88.0.0-216.91.255.255 Savvis Communications - why spidering? |
6 | trillianjedi | 5:55 pm Jul 28, 2003 | ||||
| Another one banned |
7 | cyberkat | 2:40 pm Jul 28, 2003 | ||||
| Typical spider activity beginner's question |
2 | Mr_Busby | 3:26 am Jul 28, 2003 | ||||
| WebTV and Verio |
5 | wilderness | 4:10 am Jul 27, 2003 | ||||
| why are spiders bad? sorry if this is a rookie question |
12 | SammiGirl | 12:33 am Jul 27, 2003 | ||||
| Fake GoogleBot? Korean NaverRobot bot masking as "GoogleBot" |
6 | Romeo | 3:57 pm Jul 26, 2003 | ||||
| Fake Dynamic IP changing continusly Robot with Dynamic changing IP |
3 | nilloc | 3:13 pm Jul 26, 2003 | ||||
| Any ideas: arirang_check What is it? |
3 | animator | 5:13 pm Jul 25, 2003 | ||||
| Flash Processor Anyone has seen this spider? |
2 | bilalak | 5:42 am Jul 25, 2003 | ||||
| New COMBINE spider detected Ban this spider |
2 | Digger | 2:52 pm Jul 24, 2003 | ||||
| MediaPartners spider Ban this one |
2 | Digger | 7:42 pm Jul 23, 2003 | ||||
| Gaisbot/3.0 info? |
5 | coyote | 10:43 am Jul 23, 2003 | ||||
| he is back look UE - Mediapartners-Google/2.1 he is back look UE - Mediapartners-Google/2.1 |
2 | StopSpam | 10:22 pm Jul 22, 2003 | ||||
| An aggressive spider - does anyone know it? |
6 | mrdch | 6:15 pm Jul 22, 2003 | ||||
| MSIE Crawler More information requested |
3 | loanuniverse | 4:08 pm Jul 22, 2003 | ||||
| Google and WAP |
6 | wilderness | 3:05 am Jul 22, 2003 | ||||
| who/what does mozilla stand for |
12 | Blelisa | 8:08 pm Jul 21, 2003 | ||||
| LWP::Simple heads up |
9 | wilderness | 7:23 pm Jul 21, 2003 | ||||
| New UA: CopyHunter/1.54 Has anybody else seen it and know its purpose? |
3 | Wizcrafts | 10:42 am Jul 21, 2003 | ||||
| SetEnvIf ... keep_out (Apache directive) |
3 | Made_In_Sheffield | 10:18 pm Jul 19, 2003 | ||||
| Cyveillance IP Ranges What are the IP Ranges for Cyveillance |
9 | kewlbeezer | 7:30 pm Jul 18, 2003 | ||||
| Willow Internet Crawler twotrees.com |
2 | bull | 9:03 pm Jul 17, 2003 | ||||
| Analyze your log: how to exclude yourself? How to avoid being counted? Your own visits to your website... |
6 | silverbytes | 9:33 pm Jul 16, 2003 | ||||
| Whois this guy?- 12.119.251.194 GOT index 670 times so far today |
16 | Powdork | 12:43 am Jul 16, 2003 | ||||
| A Simple Beginning with htaccess |
3 | wilderness | 11:58 pm Jul 15, 2003 |