Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Why Do Bots Generally Only Visit My Navigation? |
7 | RedBar | 3:35 pm Jun 25, 2022 | ||||
| Bytedance / bytespider IP addresses here are a few of them |
3 | SumGuy | 2:07 am Jun 20, 2022 | ||||
| A curious set of php file requests today From an IP that should not have made it... |
9 | SumGuy | 12:13 pm Jun 18, 2022 | ||||
| OVH Hosted Bots & Scrapers |
7 | JamieW | 4:53 pm Jun 11, 2022 | ||||
| IP blocking lists - Ipxo Fresh source of ip;s to block |
2 | SumGuy | 10:33 am Jun 8, 2022 | ||||
| Ludicrously stupid form spam |
10 | dstiles | 7:52 am Jun 8, 2022 | ||||
| Honor or reject HTTP OPTIONS query? (From a suspicious British Telecom /24...) |
3 | SumGuy | 8:31 am May 31, 2022 | ||||
| Blocking Google's Proxies |
20 | dstiles | 8:05 am May 22, 2022 | ||||
| novice IP blacklist question query of the nature of banning IP's vs. IP groups |
5 | Dancing_Otter | 12:52 am May 8, 2022 | ||||
| Blocking web site access using Sec-Fetch |
16 | dstiles | 3:08 pm May 3, 2022 | ||||
| Hetzner Hosted Bots & Scrapers |
3 | JamieW | 8:34 am Apr 30, 2022 | ||||
| User Agent Change For Bing Crawler Bingbot |
3 | engine | 10:15 pm Apr 29, 2022 | ||||
| Brave Search User Agent |
5 | brotherhood_of_LAN | 1:08 am Mar 26, 2022 | ||||
| User-Agent Google AppsViewer (drive.google.com) What's with this? |
2 | SumGuy | 6:41 pm Feb 23, 2022 | ||||
| navigator.userAgent, navigator.appVersion, and navigator.platform. Starting in Chrome 101.... more fun! |
3 | blend27 | 2:26 am Feb 10, 2022 | ||||
| iPhone, safari and bots |
6 | dstiles | 9:48 am Feb 3, 2022 | ||||
| thum.io Website Screenshot Generator |
3 | SumGuy | 6:18 pm Jan 31, 2022 | ||||
| Wikki bots |
3 | dstiles | 9:52 am Jan 25, 2022 | ||||
| Sidetrade indexer bot ? |
2 | SumGuy | 6:56 am Jan 25, 2022 | ||||
| Is it Okay to Block AS8075 MICROSOFT I get scraped from non-MSN bots from that range |
5 | martinibuster | 4:26 am Jan 24, 2022 | ||||
| Server Farms July 2021 Continuing discussion of hosting and data center IP ranges |
13 | not2easy | 2:56 am Jan 3, 2022 | ||||
| Adsbot redux |
2 | lucy24 | 4:26 am Dec 31, 2021 | ||||
| Is This a New Hack Method? |
12 | Angonasec | 2:33 pm Dec 19, 2021 | ||||
| Apple bot? Not according to DNS |
5 | dstiles | 9:51 pm Dec 16, 2021 | ||||
| crawl-66-249-66-66.googlebot.com How? |
8 | blend27 | 6:00 pm Nov 27, 2021 |