Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| core-project/1.0 Seems to be looking for FrontPage vulnerabilities |
6 | GaryK | 6:38 am Oct 11, 2005 | ||||
| yacy german P2P SE |
2 | bull | 1:52 pm Oct 10, 2005 | ||||
| yahoo crawler submit your site for free? |
3 | lightning2004 | 12:31 am Oct 10, 2005 | ||||
| Is this spider useful? |
20 | joker197cinque | 1:24 am Oct 9, 2005 | ||||
| EverbeeCrawler? |
4 | keyplyr | 8:26 pm Oct 7, 2005 | ||||
| A tough one: spambot disguised as IE Problem with a spambot disguised as IE |
4 | tnlnyc | 8:01 pm Oct 7, 2005 | ||||
| YahooSeeker/bsv3.9 A new day, a new YahooSeeker |
4 | MaxM | 4:45 pm Oct 6, 2005 | ||||
| Anyone know what AuditBot/0.1 is Being hit everyday with this |
10 | dramstore | 11:36 am Oct 4, 2005 | ||||
| LocalcomBot/1.2 Claims to index local websites. |
10 | GaryK | 4:07 am Oct 3, 2005 | ||||
| ShunixBot New French search engine |
4 | GaryK | 7:20 pm Oct 2, 2005 | ||||
| Metaspinner |
2 | wilderness | 6:33 am Oct 2, 2005 | ||||
| PrivacyFinder Cache Bot |
3 | keyplyr | 12:26 am Sep 29, 2005 | ||||
| Yahoo requesting GET vars that don't exist It appears that slurp is auto-incrementing a get variable |
3 | stargeek | 12:36 am Sep 27, 2005 | ||||
| Lycos |
4 | volatilegx | 8:46 pm Sep 26, 2005 | ||||
| Longest User Agent Ever? Pure Spam |
6 | volatilegx | 7:18 pm Sep 25, 2005 | ||||
| iCCrawler - any info? claims to come from iccenter.net - purpose? |
2 | thetrasher | 7:42 pm Sep 24, 2005 | ||||
| zzrobots.txt .yeah human error my [bleep]! |
6 | JAB_Creations | 8:38 pm Sep 21, 2005 | ||||
| Anyone read Finnish? Virgo/0.7 |
9 | keyplyr | 8:03 pm Sep 21, 2005 | ||||
| PreviewGenerator.org |
2 | MLHmptn | 2:01 pm Sep 20, 2005 | ||||
| Project Kolinka Forum Search Another bot that ignores robots.txt |
8 | Moparx | 4:18 am Sep 20, 2005 | ||||
| SBIder/0.7 What do you think about this bot? |
8 | GaryK | 2:49 am Sep 20, 2005 | ||||
| Clushbot Again This time in the Laomedon flavor |
3 | GaryK | 12:47 am Sep 20, 2005 | ||||
| Reputable Spider lists? Trying to find a reputable spider list... what do you recommend? |
3 | profitpuppy | 12:41 am Sep 20, 2005 | ||||
| Omnipelagos.com RSS Spider 0.3 Claims to find shortest path between any two things. |
2 | GaryK | 12:39 am Sep 20, 2005 | ||||
| Feedfetcher-Google; (+http://www.google.com/feedfetcher.html) Just noticed this one today... |
2 | BillyS | 11:32 pm Sep 19, 2005 |
