Digital Ocean has new IP's - Crawler, Spider, and User Agent ID forum at WebmasterWorld - WebmasterWorld

Forum Moderators: open

Message Too Old, No Replies

Digital Ocean has new IP's

SumGuy

11:30 pm on Mar 31, 2024 (gmt 0)

Top Contributors Of The Month

I got these 2 requests today:

GET /.env
POST /

The POST isin't exactly a request, it's an attempt to write something to my site. The user-agent being:

Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; HM NOTE 1W Build/KOT49H) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 UCBrowser/11.0.5.850 U3/0.8.0 Mobile Safari/534.30

The IP this came from was 152.42.194.186. It's the garbage dump known as Digital Ocean. AS14061

For the last few months, maybe once a week or twice a month, I've seen a garbage hit like this from a new Digital Ocean CIDR. Usually a new /20 CIDR. I just add the CIDR to my general-purpose IP blocking list in my router. This time, it's a new /19 CIDR.

So I say enough of this garbage, I go and get the entire ASN and see what exactly is new since the last time I compiled a complete list. Wow. 29 new CIDR's. I boil them down to these 16 new CIDR's:

24.144.76.0/22
64.23.224.0/20
152.42.144.0/21
152.42.192.0/19
161.35.128.0/20
161.35.176.0/20
161.35.240.0/20
164.90.128.0/18
164.90.208.0/20
164.90.240.0/21
164.90.248.0/23
164.90.250.0/24
164.90.252.0/22
207.154.192.0/19
207.154.240.0/20
209.38.160.0/20

If you maintain your own IP blocking list and you make it a point of totally blocking Digital Ocean and you haven't updated your list in the past 6 months, those are going to be new to you.

They're now in my total IP blocking list - which is now up to 38,967 CIDR's. My router can handle that with ease.

not2easy

1:16 pm on Apr 1, 2024 (gmt 0)

WebmasterWorld Administrator

10+ Year Member

Top Contributors Of The Month

Maybe you don't need 38,967 CIDRs? You can cut that list unless these smaller CIDRs used to poke holes? For example, the three CIDRs for 161.35.x.x could be one, yours overlap. Same with 164.90.x.x.

24.144.64.0/18 blocks everything at 24.144.64.0 - 24.144.127.255
64.23.128.0/17 blocks everything at 64.23.128.0 - 64.23.255.255
152.42.128.0/17 blocks everything at 152.42.128.0 - 152.42.255.255
161.35.0.0/16 blocks everything at 161.35.0.0 - 161.35.255.255
164.90.128.0/17 blocks everything at 164.90.128.0 - 164.90.255.255
207.154.192.0/18 blocks everything at 207.154.192.0 - 207.154.255.255
209.38.0.0/16 blocks everything at 209.38.0.0 - 209.38.255.255

SumGuy

10:18 pm on Apr 1, 2024 (gmt 0)

Top Contributors Of The Month

The above 16 CIDR's that I posted do not over-lap. They were condensed from the 29 CIDR's that were listed in DO's ASN that were new to me.

Your suggestion to change 24.144.76.0/22 to 24.144.64.0/18 would be fine - assuming that that entire /18 is indeed assigned to DO.

My list of 38k CIDR's is non-overlapping, and many of them are /16's and I probably have a few /9's or /10's.

I do periodically condense the list, and by doing so different ASN's that I've blocked that have interleaved CIDR's get condensed if the math works out.

not2easy

4:11 pm on Apr 2, 2024 (gmt 0)

WebmasterWorld Administrator

10+ Year Member

Top Contributors Of The Month

The ranges I posted with the CIDRs are what WHOIS shows is registered to DO - they offer hosting space by Direct Allocation and those IPs are all DO. As far as I know, Digital Ocean does not do any scraping, it's just a popular server farm that hosts scrapers.

For example, your three 161.35.x.x. ranges leave a lot of unblocked DO because everything from 161.35.0.0 - 161.35.255.255 is DO.
This is what yours are blocking:
161.35.128.0/20
161.35.128.0 - 161.35.143.255

161.35.176.0/20
161.35.176.0 - 161.35.191.255

161.35.240.0/20
161.35.240.0 - 161.35.255.255

so everything from 161.35.191.255 up to 161.35.240.0 for example is DO, but not blocked.

SumGuy

2:58 pm on Apr 13, 2024 (gmt 0)

Top Contributors Of The Month

I go by the assigned prefixes for ASN's when I add CIDR's to my blocking strategy. So at some point not so long ago, those three /20's in the 161.35.0.0/16 subnet were not being announced by DO.

Upon processing the most recent prefix list for DO (when I first posted this) it is likely that 161.35 was fully enumerated and hence would have been condensed down to a single /16 entry. Now whether or not it was known that DO had control of that entire /16 for some time, but was not advertising any routing for some of it, I don't know.

And since the first post here, when I fully enumerated DO's prefix list, these three "new" DO cidr's have shown up

152.42.152.0/22
152.42.224.0/19
209.38.128.0/19

Upon some manual investigation, I see that DO is assigned 152.42.128.0/17 but there are parts of it that it does not (currently) announce. Maybe there is an easier way to see all the CIDR prefixes assigned to (but not necessarily announced by) a given entity?

not2easy

2:55 pm on Apr 14, 2024 (gmt 0)

WebmasterWorld Administrator

10+ Year Member

Top Contributors Of The Month

I use whois, there are a LOT of sites that offer whois lookups. Whois lookups are done by database searches of ARIN, RIPE, APNIC, AFRINIC and LACNIC registries. They normally return all info on IP ranges included with whatever information the range includes. It includes registration and renewal dates as well as ASNs if that's what you use.