Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Web Download Tool Something else to ban |
12 | carfac | 6:39 pm Oct 2, 2002 | ||||
| We weirdest UA Ever! |
5 | carfac | 7:23 pm Oct 1, 2002 | ||||
| New Fast IP |
3 | volatilegx | 5:12 pm Oct 1, 2002 | ||||
| Another new one! Lickity_Split |
6 | carfac | 3:23 am Sep 30, 2002 | ||||
| What is w.x.y.z.nombres.ttd.es What engine is this a spider from? |
3 | bnhall | 1:21 pm Sep 28, 2002 | ||||
| Scooter/3.2.SF0 |
2 | Scooter24 | 8:13 pm Sep 27, 2002 | ||||
| www.go2-yellowpages.com Deep crawling for awhile now, and incorrectly |
4 | savvy1 | 8:11 pm Sep 27, 2002 | ||||
| Spiders/Bots and Javascript Spiders without javascript indexing our "javascript required" page |
6 | Advantex | 8:46 pm Sep 25, 2002 | ||||
| Caught another "Trademark Search" robot |
3 | carfac | 6:44 pm Sep 25, 2002 | ||||
| antibot-V1.1.5/i586-linux-2.2 belongs to AntiSearch |
8 | volatilegx | 5:32 pm Sep 25, 2002 | ||||
| NetMechanic V3.0 Seems to be spidering? |
2 | Nick_W | 8:56 am Sep 24, 2002 | ||||
| I am getting hit really hard... ia_archiver |
8 | mack | 4:27 pm Sep 23, 2002 | ||||
| sherlock/1.3 httpget/1.3 |
2 | frontpage | 6:57 pm Sep 22, 2002 | ||||
| Scrubby 2.2 Is this a new spider? |
2 | Nick_W | 4:37 pm Sep 22, 2002 | ||||
| V5 HTTP/S Component Who is that? |
3 | frontpage | 3:57 pm Sep 22, 2002 | ||||
| Spider from multiple IP's all at once |
7 | carfac | 2:07 am Sep 22, 2002 | ||||
| 63.215.240.27 (unknown.level3.net) Scooter/2.0 |
6 | Key_Master | 3:45 pm Sep 20, 2002 | ||||
| av |
4 | PeteU | 7:21 pm Sep 19, 2002 | ||||
| 128.242.197.101 I am paranoid but . . . |
4 | rogerRabbit | 10:54 am Sep 19, 2002 | ||||
| MSIE 5.01 requesting robots.txt? anyone who knows this? |
3 | bull | 3:37 pm Sep 18, 2002 | ||||
| searching for formmail another candidate to ban? |
2 | bull | 9:43 am Sep 18, 2002 | ||||
| The number of googles spiders The number of googles spiders what does it stands for |
2 | Ove | 7:14 am Sep 18, 2002 | ||||
| minibot any ideas? |
2 | Finder | 11:10 pm Sep 17, 2002 | ||||
| Mysterious Spider |
5 | tesla | 7:44 pm Sep 17, 2002 | ||||
| Ask Jeeves Ask Jeeves disregarding robots.txt |
16 | Scooter24 | 5:00 pm Sep 17, 2002 |
