Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| identifying user views resulting from spiders |
8 | newbie6 | 2:28 pm Apr 2, 2008 | ||||
| Snoopy v1.2.3 Source Identified PHP Class |
3 | incrediBILL | 5:46 pm Mar 28, 2008 | ||||
| CCBot another Nutch Variant |
7 | Ocean10000 | 1:39 pm Mar 28, 2008 | ||||
| Sap |
6 | wilderness | 11:48 pm Mar 27, 2008 | ||||
| User Agent: "WordPress/2.1.1" coming from Russia and UK Is this an RSS aggragator, or a scraper bot? |
6 | Wizcrafts | 2:32 am Mar 27, 2008 | ||||
| Readrun tag crawl showing up lately annoyance |
4 | mrjones | 9:52 am Mar 26, 2008 | ||||
| Slashers wondering about site slashers |
3 | Megaclinium | 6:22 am Mar 26, 2008 | ||||
| what is this bot? 72.244.103.zz |
3 | Megaclinium | 11:42 pm Mar 25, 2008 | ||||
| ConveraCrawler Is ConveraCrawler a malicious robot? |
17 | mattie | 1:05 pm Mar 25, 2008 | ||||
| Boitho ignoring robots.txt |
7 | Megaclinium | 8:17 pm Mar 24, 2008 | ||||
| Strange UA |
4 | wilderness | 1:14 am Mar 21, 2008 | ||||
| Pete-Spider Light |
2 | keyplyr | 11:24 pm Mar 20, 2008 | ||||
| TekSavvy |
8 | wilderness | 12:20 pm Mar 20, 2008 | ||||
| gnu-classpath Cute |
3 | wilderness | 9:27 pm Mar 19, 2008 | ||||
| zermelo |
3 | zerillos | 8:34 pm Mar 19, 2008 | ||||
| odd access log entry access log get form entry |
2 | esme | 10:03 pm Mar 18, 2008 | ||||
| anyone know this UA? all the spaces are +'s :? |
11 | wkitty42 | 4:07 pm Mar 18, 2008 | ||||
| agodar |
3 | Mokita | 10:28 pm Mar 17, 2008 | ||||
| User Agent "Firefox" |
8 | marodhum | 10:17 pm Mar 17, 2008 | ||||
| Ban spider by connection: close Header, (idea) I got an idea by ban spider by connection: close header |
15 | Eric | 9:04 am Mar 17, 2008 | ||||
| libwww-perl/5.53 archive.org |
10 | react | 7:32 am Mar 17, 2008 | ||||
| Quintura-Crw/0.1 Quintura-Crw/0.1 from IP 208.82.204.zz |
4 | Eric | 1:46 am Mar 17, 2008 | ||||
| New RSS Reader - Spider? Asa/1.0.1 |
2 | Ocean10000 | 1:07 am Mar 17, 2008 | ||||
| Jeanie 2008 bot No robots request, fell straight into spider trap |
2 | Receptional_Andy | 9:57 pm Mar 16, 2008 | ||||
| Yahoo Slurp China htaccess block Slurp China htaccess block |
13 | cyberdyne | 9:02 am Mar 8, 2008 |
