Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| renlifangbot MSN China |
9 | keyplyr | 12:33 am Jun 5, 2009 | ||||
| FollowSite dot com What is this thing? |
15 | tangor | 12:26 am Jun 5, 2009 | ||||
| AppEngine-Google: New Google UA to watch for |
14 | GaryK | 8:29 am Jun 4, 2009 | ||||
| Bing user agent? Does Bing use Msnbot or not? |
5 | ning | 7:40 am Jun 3, 2009 | ||||
| msnbot/2.0b New crawler from MS in the works?[2] ( 1 2 ) |
41 | GaryK | 6:20 am Jun 2, 2009 | ||||
| Hunting Google Safebrowsing Diagnostic Spidering Anyone seen anything? |
6 | incrediBILL | 2:56 am May 31, 2009 | ||||
| livebot-searchsense/0.1; +http://search.msn.com/msnbot.htm MSN strikes again |
12 | Pfui | 12:27 am May 31, 2009 | ||||
| Topsy.com / labs.topsy.com / butterfly.topsy.com Butterfly/1.0 *and* libwww-perl |
2 | Pfui | 7:37 pm May 27, 2009 | ||||
| How often does googlebot hit a large new website? How many reqs/min? |
12 | DiscoStu | 9:56 pm May 25, 2009 | ||||
| is there a data centre ip list |
7 | topr8 | 4:46 am May 25, 2009 | ||||
| Total Domain Data Corp Just got a total eclipse of the heart... |
7 | caribguy | 4:31 am May 25, 2009 | ||||
| searchers.co.uk crawling from 213.122.174.x |
6 | Hobbs | 4:56 am May 23, 2009 | ||||
| Google IP Information Request |
4 | hugh | 8:46 pm May 21, 2009 | ||||
| flatlandbot/allspark |
13 | wilderness | 8:23 am May 20, 2009 | ||||
| Profane user agents Do you get any/many? |
16 | GaryK | 9:50 am May 17, 2009 | ||||
| New MSN Mobile Crawler UA Launched without fanfare |
12 | Samizdata | 8:54 pm May 12, 2009 | ||||
| ms research robot MSN strikes again. Again. |
3 | Pfui | 8:28 pm May 10, 2009 | ||||
| Why is Tumblr crawling from ThePlanet? |
5 | GaryK | 7:45 pm May 6, 2009 | ||||
| Google Wireless Transcoder 74.125.74.nnn acting badly |
8 | Umbra | 4:31 pm May 6, 2009 | ||||
| Scraper thread on WebmasterWorld |
3 | incrediBILL | 6:30 pm Apr 30, 2009 | ||||
| MSN stealth 'bot? MSIE 7 User-agent |
5 | jdMorgan | 10:52 pm Apr 29, 2009 | ||||
| New junk from Yahoo |
7 | GaryK | 1:41 am Apr 26, 2009 | ||||
| Foxy in User-Agent |
10 | dstiles | 11:05 pm Apr 24, 2009 | ||||
| Question about an IE UA |
3 | GaryK | 8:15 pm Apr 24, 2009 | ||||
| bnf.fr bot Digital National Library of France |
9 | keyplyr | 9:34 am Apr 23, 2009 |