Hey, all. LTNS. FYI: For a few weeks now, I've seen variations of the following CIDR's IPs sniffing around once a day or so:
IP: 143.20.219.
UA: Mozilla/5.0 ... Now multiple (see below).
URI: robots.txt ... Now ignored.
All IPs route back to "The Procter And Gamble Company", P&G, which has a massive number of IPs: 143.1.0.0 - 143.40.255.255 (2,621,440). All hits have been well-behaved and heeded robots.txt. Until today.
Today they became a fellow traveler with what I'm calling "Stub" because apparently that's what it is, an APNIC-STUB that was: "Transferred To the Ripe Region On 2025-05-14T08:37:17Z". So much for transparency.
Point is, Stub's related IPs, hitting with P&G's, ask for and receive robots.txt, but totally ignore it. So beware of the above, and of:
IP: 222.167.251.
UA: Mozilla/5.0
PLUS:
UA: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36
UA: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.199 Safari/537.36
UA: Mozilla/5.0 (Macintosh; Intel Mac OS X 13_4_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.199 Safari/537.36
URI: robots.txt... Now ignored.
Both CIDRs are undeterred by UA-specific 403s. They just flip to a new (or old) ID. Time to block into the Chrome/140s now. (sigh)