Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Is Yahoo's Spider Inktomi? |
2 | sirkei | 8:53 pm Oct 10, 2004 | ||||
| Wotbox/alpha0.6 |
5 | wilderness | 2:00 am Oct 8, 2004 | ||||
| stat statcrawler@gmail.com just noticed this today... |
8 | BillyS | 11:50 pm Oct 7, 2004 | ||||
| New Spider? statcrawler (at) gmail.com no url given |
2 | uncle_bob | 12:57 am Oct 3, 2004 | ||||
| Strange UA and Activity from Google has the gbot become sentient? |
2 | uncle_bob | 12:55 am Oct 3, 2004 | ||||
| Ocelli/globalspec.com Who is this? |
3 | PCInk | 9:56 am Oct 2, 2004 | ||||
| Echelon? NSA? What?! Is anyone else seeing this, and being creeped out? |
3 | Enhasa | 9:45 pm Sep 26, 2004 | ||||
| How do I track the spiders? Novice here... |
2 | Phil_AM | 7:39 pm Sep 24, 2004 | ||||
| Seekon new spider |
2 | volatilegx | 5:14 pm Sep 24, 2004 | ||||
| findlinks/0.87 |
3 | wilderness | 6:22 pm Sep 22, 2004 | ||||
| Rogue Yahoo-MMAudVid crawler Yahoo crawler hammers website, ignores robots.txt |
22 | tangent | 9:07 am Sep 21, 2004 | ||||
| Microsoft Data Access Internet Publishing Provider DAV 1.1 |
2 | wilderness | 3:22 pm Sep 18, 2004 | ||||
| JetEye Spider? |
2 | joeychgo | 4:25 am Sep 18, 2004 | ||||
| Googlebot IP 66.249.64.n |
6 | Just_Guessing | 9:35 pm Sep 16, 2004 | ||||
| Is it Google or not? I would expect any Ip used by Google would DNS resolve?! |
4 | MaxGrenk | 11:32 pm Sep 15, 2004 | ||||
| Fast? |
3 | wilderness | 1:01 am Sep 15, 2004 | ||||
| Unexplained visits no referrer - endless visits..no javascript |
2 | websitegal | 8:39 pm Sep 13, 2004 | ||||
| Cerberian Drtrs Version-3.1-Build-16 Does not fetch robots.txt |
13 | jdMorgan | 1:43 pm Sep 13, 2004 | ||||
| automatic logfile spam generator how to stop automated robot queries with false headers |
6 | cayleyv | 2:00 am Sep 12, 2004 | ||||
| Alexa bot intent on harvesting email addresses Why does Alexa want email addresses? |
6 | surfin2u | 12:45 am Sep 12, 2004 | ||||
| bob102.inktomisearch.com requesting /?marcoz_bs=my+favorite+keyword |
2 | bnhall | 5:50 pm Sep 8, 2004 | ||||
| Googlebot is Mozilla 5.0 compatable now seem new robots still under deploy |
2 | chedong | 4:10 am Sep 7, 2004 | ||||
| Spambots in PDFs Can spambots harvest e-mail addresses from PDF files? |
3 | cdarling | 3:28 am Sep 7, 2004 | ||||
| MSIE/6.0 libwww-perl/5.76 Looking for advice on this robot. |
3 | Tony_L | 4:00 am Sep 6, 2004 | ||||
| ShowLinks/1.0+libwww/5.4.0 anyone seen this one |
5 | aswaine | 10:49 pm Sep 2, 2004 |
