Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Come on Yahoo. Really? |
7 | bobothecat2 | 8:38 pm Dec 27, 2013 | ||||
| archive.org bot bad bot |
7 | keyplyr | 7:20 am Dec 20, 2013 | ||||
| Google Test-Bot: Google-Test2 |
11 | doc_z | 8:28 pm Dec 19, 2013 | ||||
| Google? Is that you? |
13 | lucy24 | 10:33 pm Dec 18, 2013 | ||||
| Is there a Browser. . . |
10 | wilderness | 12:53 am Dec 16, 2013 | ||||
| TECH UPDATE: Javascript Makes Dumb Bots Obsolete Javascript Must Be Crawled For Scrapers To Survive |
3 | incrediBILL | 10:12 pm Dec 15, 2013 | ||||
| the plainclothes bingbot |
8 | lucy24 | 8:03 pm Dec 14, 2013 | ||||
| Yahoo:LinkExpander:Slingstone |
7 | keyplyr | 2:06 pm Dec 13, 2013 | ||||
| Google Translate |
7 | wilderness | 2:43 am Dec 11, 2013 | ||||
| YaBrowser |
15 | lucy24 | 10:04 am Dec 10, 2013 | ||||
| New stupid referer spammer Referer spam via agent string (mistyped) |
9 | yaimapitu | 2:58 pm Dec 8, 2013 | ||||
| rarely used |
12 | lucy24 | 9:22 am Dec 1, 2013 | ||||
| New Scraping Tool |
9 | keyplyr | 9:07 am Nov 28, 2013 | ||||
| Yahoo phone home |
19 | lucy24 | 8:07 pm Nov 23, 2013 | ||||
| Amazon aws, dedicatedcpanel, serverloft Issues These domains are dedicatedly screwing my site - Need Help. |
10 | Future | 7:59 pm Nov 23, 2013 | ||||
| TECH UPDATE: Bots as Browsers Using JavaScript The Bot Detection Game Has Changed[2] ( 1 2 ) |
48 | incrediBILL | 9:18 pm Nov 11, 2013 | ||||
| google-xrawler Unknown User agent "google-xrawler" |
15 | DutchTheWiz | 7:51 pm Nov 7, 2013 | ||||
| Zylon |
4 | lucy24 | 8:32 pm Nov 6, 2013 | ||||
| Is CompSpyBot really an NSASpyBot? |
17 | evsiz | 7:55 pm Oct 30, 2013 | ||||
| 50.16.178.xx Amazon of course linkCheck |
2 | Bewenched | 8:21 pm Oct 24, 2013 | ||||
| Who/what is ns4006554.ip-198-27-80.net? |
7 | iomfan | 2:01 am Oct 22, 2013 | ||||
| another one for the profilers |
6 | lucy24 | 6:52 am Oct 19, 2013 | ||||
| What is GABot/1.0 |
6 | woodpecker | 11:33 pm Oct 12, 2013 | ||||
| "DAPPER-HOST-IP" why Baidu ? "DAPPER-HOST-IP" usrer agent baidu |
18 | aboshakeeb | 9:20 am Oct 10, 2013 | ||||
| CrawlDaddy abot Site Harvester |
3 | WarmGlow | 11:37 pm Sep 28, 2013 |