Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Is this some sort of IE downloader? |
5 | SEOPTI | 11:01 pm Jan 27, 2007 | ||||
| AppleSyndication/54 Is this from Apple? |
2 | bouncybunny | 2:17 am Jan 27, 2007 | ||||
| IP ranges for Good Bots? Avoiding ban good bots, ip ranges |
6 | silverbytes | 3:15 pm Jan 26, 2007 | ||||
| Taiga web spider |
17 | wilderness | 8:51 pm Jan 25, 2007 | ||||
| Bot coming from Microsoft IPs |
5 | Mokita | 8:34 pm Jan 25, 2007 | ||||
| Podtech Network crawler |
6 | Mokita | 7:39 pm Jan 24, 2007 | ||||
| Dealing with the rogue bots and botnets |
10 | incrediBILL | 2:09 am Jan 24, 2007 | ||||
| Morfeus #*$!ing Scanner Spam scanner or referral bomber? |
5 | Wemic | 1:07 am Jan 23, 2007 | ||||
| HTTP::Lite/2.1.6 |
2 | wilderness | 7:58 am Jan 22, 2007 | ||||
| Is there a way to combat this junk? Mozilla/4.0 (compatible ; MSIE 6.0; Windows NT 5.1) |
2 | SEOPTI | 2:00 am Jan 22, 2007 | ||||
| Garbage user agent srings and changing IP addreses Impossible to block? |
12 | bouncybunny | 10:59 pm Jan 21, 2007 | ||||
| MSN Feed Manager New feed syndicator? |
4 | GaryK | 1:15 am Jan 19, 2007 | ||||
| Please correct this .htaccess exclusion tnx! |
3 | morales2k | 12:11 am Jan 19, 2007 | ||||
| bot named bot bot identification |
2 | micadelic | 3:48 pm Jan 15, 2007 | ||||
| Google Web Accelerator Cache Warmer browser enhancement possibly using proxies, varied IP |
5 | privacyman | 10:20 pm Jan 14, 2007 | ||||
| DepSpid Despite claims, distributed crawler doesn't check robots.txt |
3 | Pfui | 9:39 pm Jan 13, 2007 | ||||
| SMBot Crawling from Amazon.com in South Africa |
24 | GaryK | 8:08 pm Jan 12, 2007 | ||||
| Ntt |
6 | wilderness | 12:51 am Jan 12, 2007 | ||||
| core-project/1.0 core-project/1.0 - Front page vulnerability |
4 | seanx | 10:11 pm Jan 11, 2007 | ||||
| Mirar Search Indexing Agent adware/spyware toolbar active now |
4 | Mokita | 9:57 pm Jan 11, 2007 | ||||
| BuzzRankingBot What's the buzz? |
9 | GaryK | 9:39 pm Jan 11, 2007 | ||||
| BOTW Spider |
2 | Mokita | 12:13 am Jan 10, 2007 | ||||
| WaybackMachine and Scrapers Do scrapers scrape archive.org? |
2 | Umbra | 7:32 pm Jan 9, 2007 | ||||
| Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;) Repeated hits by this UA, with thedomain.com as referer.[2] ( 1 2 ) |
33 | montclairguy | 2:03 pm Jan 4, 2007 | ||||
| Sogou spider? Good or Bad? |
2 | phi148 | 3:28 pm Jan 2, 2007 |
