Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Fast? |
3 | wilderness | 1:01 am Sep 15, 2004 | ||||
| Unexplained visits no referrer - endless visits..no javascript |
2 | websitegal | 8:39 pm Sep 13, 2004 | ||||
| Cerberian Drtrs Version-3.1-Build-16 Does not fetch robots.txt |
13 | jdMorgan | 1:43 pm Sep 13, 2004 | ||||
| automatic logfile spam generator how to stop automated robot queries with false headers |
6 | cayleyv | 2:00 am Sep 12, 2004 | ||||
| Alexa bot intent on harvesting email addresses Why does Alexa want email addresses? |
6 | surfin2u | 12:45 am Sep 12, 2004 | ||||
| bob102.inktomisearch.com requesting /?marcoz_bs=my+favorite+keyword |
2 | bnhall | 5:50 pm Sep 8, 2004 | ||||
| Googlebot is Mozilla 5.0 compatable now seem new robots still under deploy |
2 | chedong | 4:10 am Sep 7, 2004 | ||||
| Spambots in PDFs Can spambots harvest e-mail addresses from PDF files? |
3 | cdarling | 3:28 am Sep 7, 2004 | ||||
| MSIE/6.0 libwww-perl/5.76 Looking for advice on this robot. |
3 | Tony_L | 4:00 am Sep 6, 2004 | ||||
| ShowLinks/1.0+libwww/5.4.0 anyone seen this one |
5 | aswaine | 10:49 pm Sep 2, 2004 | ||||
| BruinBot |
5 | volatilegx | 8:05 pm Sep 2, 2004 | ||||
| Has anyone had this one? Being spidered for 2 days |
9 | rainman359 | 6:31 pm Sep 2, 2004 | ||||
| Exava (exabot@exava.com) New shopping search engine in Mountain View, CA |
20 | jamesa | 1:46 pm Sep 2, 2004 | ||||
| New Unfriendly U-A and Request URI Is this an attempted server exploit? |
8 | Wizcrafts | 4:06 am Sep 2, 2004 | ||||
| ArtfaceBot? |
3 | keyplyr | 6:14 pm Aug 30, 2004 | ||||
| Several new spiders New to me at least |
19 | volatilegx | 1:41 pm Aug 29, 2004 | ||||
| Netscape 3 |
3 | wilderness | 4:13 pm Aug 28, 2004 | ||||
| Zipp revisited |
3 | wilderness | 8:29 pm Aug 27, 2004 | ||||
| New msn robot spotted today MSNPTC user-agent |
17 | jdMorgan | 7:53 pm Aug 27, 2004 | ||||
| Gecko user agent Language property missing |
10 | GaryK | 7:32 pm Aug 23, 2004 | ||||
| WebmasterWorld forum11 Updated and Collated Bot List |
21 | bull | 6:27 pm Aug 22, 2004 | ||||
| spiders moving too quickly 41 queries per second |
5 | JonR28 | 10:24 am Aug 21, 2004 | ||||
| LookBot can't find anything on it |
7 | volatilegx | 8:30 pm Aug 20, 2004 | ||||
| Java/1.4.2_01 user agent What is it? |
5 | KeithBoynton | 5:00 pm Aug 20, 2004 | ||||
| SKIZZLE! Distributed Internet Spider v1.0 |
2 | DoingItWell | 4:54 pm Aug 20, 2004 |