Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| wysiwyg possible to ban? |
12 | keyplyr | 8:57 am Jul 6, 2002 | ||||
| mylinea? |
2 | Sinner_G | 9:28 pm Jul 5, 2002 | ||||
| SansarnRobot and S@... Originating from 203.185.132.67 |
2 | hanuman | 8:12 am Jul 5, 2002 | ||||
| Fast (alltheweb) - "FirstPage" - noticed this for the first time today |
4 | engine | 11:24 pm Jul 4, 2002 | ||||
| I need a list of Google IPs |
2 | tomasz | 12:17 pm Jul 4, 2002 | ||||
| Dual Proxy what is it? |
7 | keyplyr | 5:58 pm Jul 3, 2002 | ||||
| Spider Bait - anyone heard of it? apparently it hides the ? in database driven pages |
3 | charon | 10:43 am Jul 3, 2002 | ||||
| the NEW Hotbot/Lycos alliance Which spiders now? No more inktomi |
5 | Friday | 6:57 am Jul 3, 2002 | ||||
| Lachesis anyone heard of it? |
5 | incywincy | 10:45 pm Jul 2, 2002 | ||||
| Inktomi bot? |
3 | Weblamer | 8:56 pm Jul 2, 2002 | ||||
| Pompos/1.0 pompos@iliad.fr Anybody seen this? |
5 | volatilegx | 9:55 pm Jul 1, 2002 | ||||
| Who's Libby and why does she want to be my friend? |
3 | idiotgirl | 8:09 pm Jul 1, 2002 | ||||
| Is this valid robots.txt? valid robots.txt * asterisk disallow |
3 | pwc21 | 3:36 pm Jul 1, 2002 | ||||
| display error pages to banned users .htaccess too aggressive |
6 | idiotgirl | 7:04 pm Jun 30, 2002 | ||||
| Spider? choking on javascript Huge increase in 404's |
2 | Robert_Scott | 10:20 am Jun 29, 2002 | ||||
| Possible Spambot wih user agent "MFC Library" |
2 | Bluestreak | 10:39 pm Jun 27, 2002 | ||||
| Annoying and unusual bot activity from other countries |
3 | Bluestreak | 2:43 pm Jun 27, 2002 | ||||
| List of recent spiders Unknown to me |
10 | Thierry_Zoller | 9:24 pm Jun 26, 2002 | ||||
| VB OpenUrl Anyone seen this one? |
3 | jdMorgan | 8:04 pm Jun 25, 2002 | ||||
| IBM Almaden Research Center mystery |
5 | wilderness | 1:15 am Jun 25, 2002 | ||||
| cc0.inktomi.com ua = libwww-perl/5.49 |
10 | misosoph | 10:06 pm Jun 24, 2002 | ||||
| Is this AltaVista's image robot? qascoot1.sv.av.com |
2 | misosoph | 4:52 pm Jun 21, 2002 | ||||
| IP Lists [2] ( 1 2 ) |
36 | Brett_Tabke | 4:51 pm Jun 20, 2002 | ||||
| FirstGov.gov Spider |
7 | msgraph | 10:26 am Jun 20, 2002 | ||||
| Teradex Mapper from 216.129.82.203 Grabbed my ODP-listed pages yesterday. |
4 | mbauser2 | 10:19 am Jun 20, 2002 |