Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
where's the browser?
19 lucy24 7:36 pm Jul 27, 2011
Bender the web crawler
anyone know what this new googlebot is about?
12 Kelowna 1:57 pm Jul 26, 2011
UAs that asked for robots.txt
11 incrediBILL 6:02 am Jul 18, 2011
Yellow Pages bot impersonating Googlebot?
11 mrtonyg 9:57 pm Jul 15, 2011
m160.cs.uwaterloo.ca
block?
4 santapaws 7:40 pm Jul 15, 2011
evrinid
6 Pfui 8:37 pm Jul 13, 2011
Garlik? A UK outfit
5 tangor 2:52 am Jul 13, 2011
back in the Ukraine
if the User Agent fits, wear it
23 lucy24 8:22 pm Jul 9, 2011
a legitimate blank?
26 lucy24 3:52 pm Jul 9, 2011
Re-visiting Data URI
Unexpected data:image URI calls to web sites
6 dstiles 9:24 pm Jul 8, 2011
the faviconbot's new clothes
3 lucy24 11:45 pm Jul 7, 2011
SiteSpeedBot and misinformation, bad bot
5 incrediBILL 5:22 am Jul 2, 2011
q1
2 Pfui 10:03 pm Jun 29, 2011
where old browsers go to die
Netscape hits bottom
2 lucy24 9:53 am Jun 28, 2011
User agent chomping bandwidth
Syntax error?
8 didibreakit 9:30 pm Jun 26, 2011
Is it Googlebot or a user coming from Google search?
4 jojy 1:02 pm Jun 25, 2011
WordPad and htaccess
4 wilderness 10:04 pm Jun 24, 2011
V20xRmRRPT0K
Anyone seen this in stats? Strange Bot?
17 didibreakit 11:48 pm Jun 15, 2011
Pfui
Finally heard from her.
6 GaryK 4:28 pm Jun 13, 2011
Is this a botnet attack?
admin/file_manager.php/login.php
23 grandma_genie 9:17 pm Jun 7, 2011
Need some advices and clarifications
6 macas 12:44 pm Jun 2, 2011
How Often To Check Robots.txt?
2 devitnow 7:28 pm Jun 1, 2011
Yahoo crawler(s) requesting mangled URLs
2 Jonesy 8:54 pm May 28, 2011
upload.php
8 wilderness 9:56 pm May 27, 2011
cmsworldmap dot com
4 tangor 9:19 pm May 26, 2011