Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Bot list HTTrack or httrack |
2 | wilderness | 10:06 pm Nov 13, 2004 | ||||
| Geona Looks nice.... |
2 | CodeJockey | 9:18 pm Nov 13, 2004 | ||||
| gazz/5.0 Need to update .htaccess file |
6 | guitaristinus | 3:38 pm Nov 9, 2004 | ||||
| \"SeznamBot/1.0\" yes, the correct UA |
3 | bull | 8:59 am Nov 9, 2004 | ||||
| PrassoSunner 1.00 |
2 | bull | 4:42 pm Nov 8, 2004 | ||||
| Nutch from Looksmart |
3 | volatilegx | 10:28 pm Nov 7, 2004 | ||||
| Y!oasis from Yahoo IP |
2 | bull | 7:40 am Nov 7, 2004 | ||||
| Intel Multiple IP's |
2 | wilderness | 4:25 am Nov 7, 2004 | ||||
| bangalore.corp.yahoo.com? Is this a legitimate yahoo spider? global-pix1.bangalore.corp.yahoo.com |
2 | Osian | 12:52 am Nov 6, 2004 | ||||
| 17 | mattie | 2:39 pm Nov 5, 2004 | |||||
| Offline Explorer Not sure where to post this |
2 | chobo321321 | 12:48 am Nov 3, 2004 | ||||
| ExoticCrawler ban it? |
4 | guitaristinus | 10:44 pm Nov 2, 2004 | ||||
| new private search engine? jeteye.com[2] ( 1 2 ) |
43 | macrost | 5:27 am Nov 2, 2004 | ||||
| innerprise.net? |
2 | blaketar | 4:05 am Nov 2, 2004 | ||||
| Amazon |
2 | wilderness | 1:10 am Oct 29, 2004 | ||||
| Jakarta Commons-HttpClient/2.0rc2 Anyone banning it? |
2 | guitaristinus | 2:10 pm Oct 28, 2004 | ||||
| WebRescuer v0.2.4 Anyone know what this is? |
3 | Busynut | 11:45 am Oct 28, 2004 | ||||
| spider took website down |
4 | ScottM | 2:39 am Oct 28, 2004 | ||||
| SightQuest Bot/1.2 Went right for links page |
3 | idoc | 1:53 am Oct 27, 2004 | ||||
| Definitive list of spiders. |
5 | lloyd | 4:14 pm Oct 26, 2004 | ||||
| Cn-dot CN-DOT Spider |
2 | Hollywood | 11:08 pm Oct 25, 2004 | ||||
| Woodland Data Center OH |
3 | wilderness | 11:10 am Oct 24, 2004 | ||||
| Gridzoom Throws Down the Gauntlet gridBot/0.3alpha[2] ( 1 2 ) |
41 | pendanticist | 8:05 pm Oct 23, 2004 | ||||
| msnbot/0.3 hits the street UA version apparently skipped from 0.11 to 0.3 |
6 | jdMorgan | 1:38 am Oct 22, 2004 | ||||
| DiamondBot Apparently run by gator |
6 | seindal | 11:51 pm Oct 21, 2004 |