Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Mozilla/5.0
Google Proxy
2 keyplyr 9:31 pm Oct 11, 2018
nsoftware.com
2 TorontoBoy 1:27 pm Oct 8, 2018
Google-Security-Zonelenny
8 ClosedForLunch 4:54 pm Oct 7, 2018
MeWeBot
4 keyplyr 2:01 am Oct 6, 2018
DuckDuckBot-Https
new UA, new range
8 keyplyr 8:52 pm Oct 4, 2018
Nexgate Ruby Client
3 keyplyr 7:39 pm Oct 3, 2018
Gowikibot
4 lucy24 6:36 pm Oct 3, 2018
Nimbostratusbot
5 lucy24 2:17 am Sep 28, 2018
Twisted client
3 keyplyr 3:51 am Sep 26, 2018
YaK
new range
3 keyplyr 4:02 am Sep 25, 2018
PhantomJS
3 keyplyr 11:49 pm Sep 23, 2018
VelenPublicWebCrawler
3 TorontoBoy 9:30 pm Sep 20, 2018
Cloudflare-AMP User Agents
2 keyplyr 10:01 am Sep 18, 2018
og-scraper
2 keyplyr 12:09 pm Sep 17, 2018
Jaunt
3 keyplyr 10:38 am Sep 17, 2018
SitemapParser-VIPnytt
Choopa & Vultr IP ranges
7 keyplyr 5:49 am Sep 11, 2018
drupalgeddon2
4 TorontoBoy 10:00 pm Sep 5, 2018
Emeigh Investments
3 dstiles 10:13 am Sep 5, 2018
GoogleImageProxy
9 keyplyr 7:29 pm Sep 1, 2018
Qwantify/Mermoz
3 keyplyr 9:27 pm Aug 29, 2018
Grammarly
6 keyplyr 5:32 pm Aug 29, 2018
Yeti
new UA
7 keyplyr 6:25 pm Aug 28, 2018
So blocking semrush. block bot or IP?
10 born2run 6:04 pm Aug 28, 2018
plainclothes facebook?
4 lucy24 1:50 am Aug 27, 2018
Faraday
New UA, new IP range
8 jonasjacek 12:09 am Aug 27, 2018