Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Bot or malware?
I'm seeing strange 'msoffice' access to my system
4 Megaclinium 4:48 am Apr 4, 2008
Please welcome our new Moderator incrediBILL
29 engine 4:41 am Apr 4, 2008
Bravo MSN
2 wilderness 9:57 pm Apr 2, 2008
Yahoo image grabber
2 newbie6 8:08 pm Apr 2, 2008
identifying user views resulting from spiders
8 newbie6 2:28 pm Apr 2, 2008
Snoopy v1.2.3 Source Identified
PHP Class
3 incrediBILL 5:46 pm Mar 28, 2008
CCBot
another Nutch Variant
7 Ocean10000 1:39 pm Mar 28, 2008
Sap
6 wilderness 11:48 pm Mar 27, 2008
User Agent: "WordPress/2.1.1" coming from Russia and UK
Is this an RSS aggragator, or a scraper bot?
6 Wizcrafts 2:32 am Mar 27, 2008
Readrun
tag crawl showing up lately annoyance
4 mrjones 9:52 am Mar 26, 2008
Slashers
wondering about site slashers
3 Megaclinium 6:22 am Mar 26, 2008
what is this bot?
72.244.103.zz
3 Megaclinium 11:42 pm Mar 25, 2008
ConveraCrawler
Is ConveraCrawler a malicious robot?
17 mattie 1:05 pm Mar 25, 2008
Boitho ignoring robots.txt
7 Megaclinium 8:17 pm Mar 24, 2008
Strange UA
4 wilderness 1:14 am Mar 21, 2008
Pete-Spider Light
2 keyplyr 11:24 pm Mar 20, 2008
TekSavvy
8 wilderness 12:20 pm Mar 20, 2008
gnu-classpath
Cute
3 wilderness 9:27 pm Mar 19, 2008
zermelo
3 zerillos 8:34 pm Mar 19, 2008
odd access log entry
access log get form entry
2 esme 10:03 pm Mar 18, 2008
anyone know this UA?
all the spaces are +'s :?
11 wkitty42 4:07 pm Mar 18, 2008
agodar
3 Mokita 10:28 pm Mar 17, 2008
User Agent "Firefox"
8 marodhum 10:17 pm Mar 17, 2008
Ban spider by connection: close Header, (idea)
I got an idea by ban spider by connection: close header
15 Eric 9:04 am Mar 17, 2008
libwww-perl/5.53
archive.org
10 react 7:32 am Mar 17, 2008