Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Naughty Yahoo User Agents Please post them here[2] ( 1 2 ) |
45 | GaryK | 5:04 pm Jul 14, 2006 | ||||
| New UA: Mozilla/4.0 (compatible; mark.blonin.bot;) has a weird Referer |
6 | Mokita | 6:44 pm Jul 13, 2006 | ||||
| sna-0.0.1 mikeelliott@hotmail.com (nee mikemuzio@msn.com) Spider as Snoopy Spammer |
2 | Pfui | 1:13 am Jul 13, 2006 | ||||
| NASA Search 1.0? |
6 | keyplyr | 6:00 pm Jul 12, 2006 | ||||
| Feed43 Proxy/1.0 (www.feed43.com) |
2 | zCat | 2:35 pm Jul 10, 2006 | ||||
| A Python-urllib hit 8000+ times yesterday |
2 | bigreat | 8:16 pm Jul 9, 2006 | ||||
| SBIder/0.8-dev Anyone getting this? |
15 | youfoundjake | 3:21 pm Jul 8, 2006 | ||||
| Hadrinka Tumaj Al-Kahal WebWasher 3.4 |
9 | Pfui | 6:18 am Jul 7, 2006 | ||||
| Yahoo Mindset No exclamation mark! |
3 | GaryK | 5:35 am Jul 5, 2006 | ||||
| page_verifier Scanning for malware |
5 | GaryK | 10:19 pm Jul 3, 2006 | ||||
| Redirect Test/0.1 from Amazon.com |
5 | GaryK | 10:10 pm Jul 3, 2006 | ||||
| lanshanbot/1.0 ( http://search.msn.com/msnbot.htm) This one looks familiar |
3 | GaryK | 9:52 pm Jul 3, 2006 | ||||
| New Google IPs GSA Crawler |
4 | fiestagirl | 3:29 pm Jul 3, 2006 | ||||
| Same-second hits from nec-labs using Java and -- Ken If at first you don't succeed, do a switcheroo. |
6 | Pfui | 3:20 am Jul 2, 2006 | ||||
| gsa-crawler Whois = Google? |
6 | coconutz | 11:21 pm Jul 1, 2006 | ||||
| /a1b2c3d4e5f6g7h8i9/nonexistentfile.php |
11 | Umbra | 3:32 am Jul 1, 2006 | ||||
| Snapbot Anyone know what it is?[2] ( 1 2 ) |
33 | Mokita | 3:18 am Jul 1, 2006 | ||||
| Psycheclone Anyone heard of this one? |
20 | malachite | 8:51 pm Jun 30, 2006 | ||||
| Excluding all UA's with spider, bot, and crawl in them Is this too broad a brush to paint with? |
5 | cfx211 | 8:16 pm Jun 29, 2006 | ||||
| Mozilla/4.0 (compatible; Google Desktop) New user-agent? |
3 | referer | 6:32 pm Jun 29, 2006 | ||||
| OutfoxBot disobeys robots.txt |
2 | keyplyr | 5:25 pm Jun 27, 2006 | ||||
| KBeeBot |
2 | keyplyr | 5:17 pm Jun 27, 2006 | ||||
| Charlotte the spider anyone seen this one yet? |
22 | innocbystr | 8:08 pm Jun 26, 2006 | ||||
| WebarooBot was RufusBot |
2 | keyplyr | 6:04 pm Jun 25, 2006 | ||||
| bot/1.0 claims to be Microsoft |
5 | 4string | 4:42 pm Jun 25, 2006 |
