Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Profane user agents Do you get any/many? |
16 | GaryK | 9:50 am May 17, 2009 | ||||
| New MSN Mobile Crawler UA Launched without fanfare |
12 | Samizdata | 8:54 pm May 12, 2009 | ||||
| ms research robot MSN strikes again. Again. |
3 | Pfui | 8:28 pm May 10, 2009 | ||||
| Why is Tumblr crawling from ThePlanet? |
5 | GaryK | 7:45 pm May 6, 2009 | ||||
| Google Wireless Transcoder 74.125.74.nnn acting badly |
8 | Umbra | 4:31 pm May 6, 2009 | ||||
| Scraper thread on WebmasterWorld |
3 | incrediBILL | 6:30 pm Apr 30, 2009 | ||||
| MSN stealth 'bot? MSIE 7 User-agent |
5 | jdMorgan | 10:52 pm Apr 29, 2009 | ||||
| New junk from Yahoo |
7 | GaryK | 1:41 am Apr 26, 2009 | ||||
| Foxy in User-Agent |
10 | dstiles | 11:05 pm Apr 24, 2009 | ||||
| Question about an IE UA |
3 | GaryK | 8:15 pm Apr 24, 2009 | ||||
| bnf.fr bot Digital National Library of France |
9 | keyplyr | 9:34 am Apr 23, 2009 | ||||
| Diffbot I sent this packing a few months ago. Now it's back with a new UA |
5 | GaryK | 5:06 pm Apr 22, 2009 | ||||
| Chat Catcher |
3 | GaryK | 3:40 pm Apr 21, 2009 | ||||
| Some Opera Mobile user agents |
10 | GaryK | 2:39 am Apr 21, 2009 | ||||
| fake Googlebot? |
24 | zerillos | 2:10 am Apr 21, 2009 | ||||
| GoogleToolbar and favicon asking for it everywhere |
12 | smallcompany | 10:59 am Apr 20, 2009 | ||||
| Daumoa, Korean bot disobedient and weird UA |
4 | Mokita | 3:52 am Apr 19, 2009 | ||||
| Bad-Neighborhood Link Analyzer |
26 | GaryK | 9:27 pm Apr 16, 2009 | ||||
| Unreal |
11 | wilderness | 11:38 pm Apr 14, 2009 | ||||
| Next Generation Search Engine I'm seeing a lot of these lately |
3 | GaryK | 5:10 pm Apr 13, 2009 | ||||
| Strange MSN 2.b Bot Behavior.Hacker? Strange MSN 2.b Bot Behavior |
2 | Lain_se | 2:59 pm Apr 13, 2009 | ||||
| Microsoft crawl, no bot UA |
13 | keyplyr | 10:34 am Apr 12, 2009 | ||||
| GoogleBot Declaring itself in my tracker stats? never before have I seen this |
3 | Kristos | 11:35 pm Apr 11, 2009 | ||||
| Comtorbot |
2 | keyplyr | 5:35 pm Apr 5, 2009 | ||||
| Complete List of Yahoo Crawlers IPs addresses to block them ! How to definitively block yahoo slurp bots? |
2 | oliondor | 2:57 pm Apr 4, 2009 |
