Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Steve Souders offers help in user agent decoding for UA Profiler |
8 | Clark | 10:59 pm Jan 28, 2009 | ||||
| same IP switching UAs many bad queries |
5 | smallcompany | 7:00 pm Jan 28, 2009 | ||||
| Do search engines bots crawl proxy servers data? |
3 | Ezer | 1:40 pm Jan 25, 2009 | ||||
| Distributed scraping looks like a zombie network |
9 | caribguy | 10:22 pm Jan 23, 2009 | ||||
| Iana |
4 | phred | 10:54 am Jan 23, 2009 | ||||
| Digg's UA |
5 | incrediBILL | 8:20 pm Jan 22, 2009 | ||||
| Guestling Spiders interest in East Sussex |
2 | RHaworth | 11:34 am Jan 22, 2009 | ||||
| AJAX Can Turn Browsers into Scrapers and Spammers |
4 | incrediBILL | 2:56 pm Jan 21, 2009 | ||||
| Curious sequence of events |
12 | phred | 6:35 am Jan 21, 2009 | ||||
| \wbot[\/\-] in AWStats Robots/Spiders visitors But does not appear on raw log |
8 | bouncybunny | 3:30 pm Jan 20, 2009 | ||||
| Northwest Open Access Network |
8 | wilderness | 8:33 pm Jan 19, 2009 | ||||
| Tse/1.0 |
2 | keyplyr | 3:20 am Jan 18, 2009 | ||||
| AliBaba and the Forty . Hits Just Heads up |
4 | blend27 | 3:39 am Jan 12, 2009 | ||||
| AVG-8 User-Agents revisited Scamming using an AVG signature |
6 | dstiles | 5:20 am Jan 11, 2009 | ||||
| Weird google hostname yx-​out-​f136.​google.​com |
2 | johnnie | 4:56 am Jan 8, 2009 | ||||
| ForumSeek |
6 | dstiles | 10:45 pm Jan 4, 2009 | ||||
| Spinn3r RSS Aggregator Spins out of control from Serverbeach range |
2 | caribguy | 7:19 am Jan 4, 2009 | ||||
| DBLBot dontbuylists.com |
6 | dstiles | 7:14 am Dec 27, 2008 | ||||
| Do Bot-Blocking Techniques Alter Bot Behavior? Various methods discussed, 403 forbidden vs 200 OK[2] ( 1 2 ) |
57 | dstiles | 3:30 am Dec 24, 2008 | ||||
| YoudaoBot |
6 | mcneely | 12:26 pm Dec 23, 2008 | ||||
| P3P Policy |
4 | wilderness | 9:48 pm Dec 22, 2008 | ||||
| Feedfetcher-Google-iGoogleGadgets Something new from Google? |
4 | GaryK | 4:23 pm Dec 21, 2008 | ||||
| Google manual review which user agent do they use? |
5 | SEOPTI | 9:10 am Dec 19, 2008 | ||||
| 127.0.0.1 visits from 127.0.0.1 |
4 | dolcevita | 8:10 am Dec 19, 2008 | ||||
| Yahoo-Test ignoring robots.txt |
3 | dstiles | 6:27 am Dec 18, 2008 |