Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Banning Googlebot Spoofers?
5 keyplyr 4:25 am Jul 19, 2007
Suspicious computer from Microsoft IP range
pretends to come from Search Live results - what's this?
3 radix 5:04 am Jul 11, 2007
Is this really googlebot feedfetcher?
agent feedfetcher
2 Clark 1:02 am Jul 7, 2007
Gigabot IP address?
8 keyplyr 11:43 pm Jul 4, 2007
BoardReader-Image-Fetcher
4 wilderness 5:15 am Jun 25, 2007
facebookexternalhit/1.0
2 MaxM 3:13 am Jun 25, 2007
shelob v1.0 - No robots.txt
From Juniper Networks
4 jdMorgan 7:00 pm Jun 17, 2007
IEAutoDiscovery User-Agent
DeskShare AutoDiscovery Active Web Reader RSS feed reader
2 VisitingPaul 4:53 pm Jun 14, 2007
Is this really ia archiver?
What is the true IA user agent?
4 bouncybunny 4:15 pm Jun 14, 2007
worio+bot+heritrix/1.10.0
worio+bot+heritrix/1.10.0
6 blend27 2:25 am Jun 13, 2007
Intelix/0.7
New bot from Amazon?
3 GaryK 8:27 pm Jun 12, 2007
Spoofing ASK?
2 keyplyr 1:54 am Jun 7, 2007
YahooSeeker/M1A1-R2D2
3 MaxM 5:09 am Jun 6, 2007
Busy id-search.org/bot.html
2 longen 7:55 pm Jun 1, 2007
Claims to be Ask
IP registered to MCI
7 volatilegx 12:21 am May 27, 2007
MySpace News bot
improper identification
3 MaxM 1:32 pm May 22, 2007
Is this really Yahoo's slurp?
2 botslist 12:56 am May 22, 2007
More on tide526.microsoft.com
30 helleborine 4:33 pm May 19, 2007
Secure Sites From Botnet Vulnerability Probes
15 incrediBILL 1:15 pm May 14, 2007
mqbot@cazoodle.com
4 SEOPTI 12:10 am May 9, 2007
Anyone know what VB Project is?
This user agent appears followed by normal browser
8 bouncybunny 6:09 am May 6, 2007
Major Intel.com Spidering Activity - Multiple IP's
3 Brett_Tabke 11:04 pm May 5, 2007
Project Honey Pot files $1B+ Lawsuit Against Spammers and Spambots
spam harvesters about to be harvested in court
24 incrediBILL 2:06 pm May 1, 2007
Open-I-Search Unveiled
Interview with the BotMaster
2 yodokame 4:48 pm Apr 29, 2007
Referer is Google with no search string
http://www.google.com/ - that's the whole URL
5 Umbra 8:06 pm Apr 24, 2007