Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| "gmail.com" bots? A bunch are showing up |
7 | tangor | 6:16 am Dec 18, 2008 | ||||
| cscinfo.com aipbot bot.com consult dynamics? |
7 | Megaclinium | 5:26 pm Dec 15, 2008 | ||||
| NetcraftSurveyAgent from new Amazon EC2 range didn't read robots.txt |
8 | thetrasher | 4:54 am Dec 15, 2008 | ||||
| Form spammer |
7 | wilderness | 2:20 am Dec 14, 2008 | ||||
| Webcollage Barrage bizarre consecutive distributed hits |
24 | incrediBILL | 3:58 pm Dec 13, 2008 | ||||
| Technorati Bot |
2 | engine | 2:08 pm Dec 10, 2008 | ||||
| Broken MSIE User-Agent nested MSIE strings |
8 | dstiles | 2:00 pm Dec 9, 2008 | ||||
| What is ATT.net/s/s.dll and why is it spamming my website? Is it possibly related to a security overhaul I performed? |
2 | JS_Harris | 6:13 pm Dec 8, 2008 | ||||
| WHttpTest and WinInetSimpleRequest Unwanted and abusive |
6 | GaryK | 4:23 pm Dec 8, 2008 | ||||
| WordTracker Attempts Crawling My Site! Trying to be low key at 180 pages in 14 days |
8 | incrediBILL | 7:03 pm Dec 4, 2008 | ||||
| YesupBot - No soup for you! Bot or real visitor? |
12 | caribguy | 9:10 pm Dec 3, 2008 | ||||
| Fake Googlebot from StandardShell Spoofed UA, no additional headers |
14 | jdMorgan | 11:58 pm Dec 2, 2008 | ||||
| rdfbot drops "Nutch" UA Change |
3 | caribguy | 9:46 am Nov 30, 2008 | ||||
| Identifying bots via ip and host Identifying bots via ip and host |
30 | enigma1 | 5:20 pm Nov 29, 2008 | ||||
| CFNetwork again. More junk from Yahoo? |
7 | GaryK | 7:31 pm Nov 28, 2008 | ||||
| User Agent = Feedfetcher-Google |
17 | smallcompany | 5:02 pm Nov 28, 2008 | ||||
| GurujiBot/1.0 Obeyed Robots.txt |
3 | mcneely | 2:31 am Nov 28, 2008 | ||||
| DomainCrawler What purpose does it serve? |
11 | GaryK | 9:30 pm Nov 26, 2008 | ||||
| UA's starting with "User-Agent:" Redundant UA declaration from MSIE look-alikes |
4 | caribguy | 11:17 am Nov 25, 2008 | ||||
| SkyGrid |
2 | keyplyr | 2:12 am Nov 25, 2008 | ||||
| Voracious |
2 | keyplyr | 11:13 am Nov 24, 2008 | ||||
| BTWebClient |
5 | koan | 5:00 pm Nov 23, 2008 | ||||
| Personifi heads up HEAD request with Wget/1.10.2 and then GET with Mozilla/5.0 X11 |
8 | caribguy | 8:52 pm Nov 21, 2008 | ||||
| More netsweeper. I feel the urge to 86 216.171.96.nnn - 216.171.111.nnn |
3 | caribguy | 8:25 pm Nov 19, 2008 | ||||
| Intersting UA |
13 | wilderness | 5:13 pm Nov 18, 2008 |