Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| MS impersonating FB |
7 | keyplyr | 8:39 am Jan 6, 2017 | ||||
| brandwatch magpiecrawler |
5 | lucy24 | 5:32 am Jan 6, 2017 | ||||
| now what's Yandex up to? |
4 | lucy24 | 12:52 pm Jan 3, 2017 | ||||
| HubSpot Marketing Grader |
5 | keyplyr | 9:13 pm Jan 1, 2017 | ||||
| Baiduspider Crawl Ranges verified IP ranges |
16 | keyplyr | 2:13 am Dec 31, 2016 | ||||
| UniversalFeedParser |
3 | keyplyr | 5:19 am Dec 23, 2016 | ||||
| Screaming Frog SEO Spider |
8 | keyplyr | 9:50 pm Dec 22, 2016 | ||||
| spbot |
3 | keyplyr | 5:03 am Dec 22, 2016 | ||||
| CSSCheck |
3 | keyplyr | 7:34 am Dec 20, 2016 | ||||
| Magic Browser |
4 | jonasjacek | 1:28 am Dec 17, 2016 | ||||
| netarkivet.dk/webcrawler heritrix variant |
3 | keyplyr | 11:03 pm Dec 13, 2016 | ||||
| vkShare |
2 | keyplyr | 11:39 pm Dec 12, 2016 | ||||
| Android 4.4.4 Googlebot |
3 | lucy24 | 4:13 am Dec 12, 2016 | ||||
| bl.uk lddc bot |
3 | keyplyr | 10:16 pm Dec 11, 2016 | ||||
| HubSpot Marketing Grader |
3 | keyplyr | 7:16 am Dec 11, 2016 | ||||
| SemrushBot |
5 | keyplyr | 7:13 am Dec 8, 2016 | ||||
| ActiveWorlds |
3 | keyplyr | 3:23 am Dec 8, 2016 | ||||
| Google Web Preview Analytics |
2 | jonasjacek | 8:43 am Dec 6, 2016 | ||||
| Datasift data analysis |
2 | keyplyr | 5:29 pm Dec 5, 2016 | ||||
| Tianyu-ktouch |
2 | keyplyr | 8:03 am Dec 2, 2016 | ||||
| AlienBlue |
3 | keyplyr | 8:08 pm Nov 30, 2016 | ||||
| redditbot |
4 | keyplyr | 7:51 pm Nov 29, 2016 | ||||
| Symfony2 BrowserKit |
2 | keyplyr | 9:55 pm Nov 28, 2016 | ||||
| More semalt variations [2] ( 1 2 ) |
41 | blend27 | 11:28 pm Nov 20, 2016 | ||||
| RDFa Parser |
2 | jonasjacek | 1:27 pm Nov 20, 2016 |