Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| ipd/1.0 Has anyone else seen this one? |
10 | Pushycat | 2:01 pm Jul 11, 2002 | ||||
| bad spider, no UA, no read of robots.txt where's my newspaper.... |
6 | dcheney | 5:55 am Jul 11, 2002 | ||||
| bigip1a-snat.sv.av.com who is from altavista ? |
2 | berno | 7:32 pm Jul 10, 2002 | ||||
| Pioneer User Agent |
3 | frontpage | 7:28 pm Jul 10, 2002 | ||||
| Internet Explore 5.x - **mumble mumble** hyperactive chinese bot **mumble mumble** |
6 | Dreamquick | 7:50 am Jul 10, 2002 | ||||
| AltaVista NOIMAGEINDEX name="robots" content="noimageindex" Is this new? |
16 | misosoph | 11:24 pm Jul 9, 2002 | ||||
| Spider Trap needs GOOD Spider list Having setup a good spider trap I need to exclude the good ones... |
15 | scottspence | 4:44 am Jul 9, 2002 | ||||
| Heartbeat page requests... .. could be a little off topic... |
2 | scottspence | 10:04 pm Jul 8, 2002 | ||||
| Should a spider take more than a few pages at a time? whole site not been indexed |
4 | nickc001 | 8:10 am Jul 8, 2002 | ||||
| DTS Agent Anyone know what this one is? |
6 | jdMorgan | 12:55 am Jul 8, 2002 | ||||
| Steeler from 157.82.157.91 One of those research bots you guys hate. |
7 | mbauser2 | 10:09 am Jul 7, 2002 | ||||
| wysiwyg possible to ban? |
12 | keyplyr | 8:57 am Jul 6, 2002 | ||||
| mylinea? |
2 | Sinner_G | 9:28 pm Jul 5, 2002 | ||||
| SansarnRobot and S@... Originating from 203.185.132.67 |
2 | hanuman | 8:12 am Jul 5, 2002 | ||||
| Fast (alltheweb) - "FirstPage" - noticed this for the first time today |
4 | engine | 11:24 pm Jul 4, 2002 | ||||
| I need a list of Google IPs |
2 | tomasz | 12:17 pm Jul 4, 2002 | ||||
| Dual Proxy what is it? |
7 | keyplyr | 5:58 pm Jul 3, 2002 | ||||
| Spider Bait - anyone heard of it? apparently it hides the ? in database driven pages |
3 | charon | 10:43 am Jul 3, 2002 | ||||
| the NEW Hotbot/Lycos alliance Which spiders now? No more inktomi |
5 | Friday | 6:57 am Jul 3, 2002 | ||||
| Lachesis anyone heard of it? |
5 | incywincy | 10:45 pm Jul 2, 2002 | ||||
| Inktomi bot? |
3 | Weblamer | 8:56 pm Jul 2, 2002 | ||||
| Pompos/1.0 pompos@iliad.fr Anybody seen this? |
5 | volatilegx | 9:55 pm Jul 1, 2002 | ||||
| Who's Libby and why does she want to be my friend? |
3 | idiotgirl | 8:09 pm Jul 1, 2002 | ||||
| Is this valid robots.txt? valid robots.txt * asterisk disallow |
3 | pwc21 | 3:36 pm Jul 1, 2002 | ||||
| display error pages to banned users .htaccess too aggressive |
6 | idiotgirl | 7:04 pm Jun 30, 2002 |
