Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| MSN Feed Manager New feed syndicator? |
4 | GaryK | 1:15 am Jan 19, 2007 | ||||
| Please correct this .htaccess exclusion tnx! |
3 | morales2k | 12:11 am Jan 19, 2007 | ||||
| bot named bot bot identification |
2 | micadelic | 3:48 pm Jan 15, 2007 | ||||
| Google Web Accelerator Cache Warmer browser enhancement possibly using proxies, varied IP |
5 | privacyman | 10:20 pm Jan 14, 2007 | ||||
| DepSpid Despite claims, distributed crawler doesn't check robots.txt |
3 | Pfui | 9:39 pm Jan 13, 2007 | ||||
| SMBot Crawling from Amazon.com in South Africa |
24 | GaryK | 8:08 pm Jan 12, 2007 | ||||
| Ntt |
6 | wilderness | 12:51 am Jan 12, 2007 | ||||
| core-project/1.0 core-project/1.0 - Front page vulnerability |
4 | seanx | 10:11 pm Jan 11, 2007 | ||||
| Mirar Search Indexing Agent adware/spyware toolbar active now |
4 | Mokita | 9:57 pm Jan 11, 2007 | ||||
| BuzzRankingBot What's the buzz? |
9 | GaryK | 9:39 pm Jan 11, 2007 | ||||
| BOTW Spider |
2 | Mokita | 12:13 am Jan 10, 2007 | ||||
| WaybackMachine and Scrapers Do scrapers scrape archive.org? |
2 | Umbra | 7:32 pm Jan 9, 2007 | ||||
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;) Repeated hits by this UA, with thedomain.com as referer.[2] ( 1 2 ) |
33 | montclairguy | 2:03 pm Jan 4, 2007 | ||||
| Sogou spider? Good or Bad? |
2 | phi148 | 3:28 pm Jan 2, 2007 | ||||
| fake google bot? |
6 | pp46 | 11:19 am Jan 2, 2007 | ||||
| LinkWalker/2.0 Anyone knows what this bot does? |
3 | ibnuasad | 9:29 pm Jan 1, 2007 | ||||
| bot/1.0 (bot; http://; bot@bot.bot) |
7 | Mokita | 7:41 am Dec 29, 2006 | ||||
| Yodaobot 1.0 is coming Do you notice the Outfoxbot 0.x? Last weekend it released as Yodaobot 1.0 |
16 | chedong | 6:04 pm Dec 28, 2006 | ||||
| Exabot-Test/1.0 |
2 | GaryK | 6:35 am Dec 25, 2006 | ||||
| Yahoo invents browser that doesn't need an OS! Y! sitemap checker |
2 | jdMorgan | 5:05 pm Dec 22, 2006 | ||||
| IE7 Save Site UA? |
7 | keyplyr | 9:58 am Dec 21, 2006 | ||||
| User-Agent begins with "User Agent::" |
4 | Umbra | 12:51 am Dec 19, 2006 | ||||
| WorQmada/1.0 |
6 | bobothecat | 6:02 am Dec 18, 2006 | ||||
| Bigsearch.ca new Canadian search engine currently Beta |
3 | Mokita | 2:58 am Dec 16, 2006 | ||||
| MSNBot from APNIC? Is this a spoofed user-agent? |
5 | Kadence | 7:53 pm Dec 15, 2006 |