Forum Moderators: bakedjake

Message Too Old, No Replies

IPtables - multiple files possible?

Can iptables have include files, or is there a good alternative?

         

dstiles

10:24 am on Apr 5, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



My soon-to-be-replaced online server for mail and websites uses iptables to block unwanted accesses in two categories: IP ranges of /24 and more, most of them /16 plus; and individual IPs (/32) added dynamically according to bad activity on the mail server. These are all within a single file headed with three ACCEPT lines: INPUT, FORWARD and OUTPUT. The dynamic ones are periodically pruned by hand to reduce file size and increase performance, though I'm aware the improvements are minimal.

I am implementing two VPS servers to replace the combination one, whose OS reaches EOL this month. I have already implemented a minimal iptables along the lines shown at [webmasterworld.com...] I now wish to add in my original server's block ranges but preferably in a structured way:

Basic ACCEPT/REJECT (as per example above);
Fixed IP ranges (server farms etc) (added from a bash file using iptables-persistent save);
Dynamic IPs added automatically (added from a bash file using iptables-persistent save).

It would be helpful if I could do this using (eg) three included files but I can't find a way of doing this.

I understand from reading online that the order of the commands is important, that some should be in the file above others (eg to allow SSH access). Using iptables-persistent places the entries at the top of a single file, just beow any entries prefixed with a colon.

In the absence of an INCLUDE it occurs to me to run three or four separate accept/reject files which can then be concatenated to form the real iptables file.

Is this a reasonable way to work (assuming I'm right about the INCLUDE option)?

lammert

10:44 am on Apr 5, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The magic word here is "tables". Iptables is named that way for a reason. The best way to handle this is to define your own custom tables. In my configuration I have a table Enemies which is defined in the main iptables configuration file but not filled with any entries at startup.

In this main iptables file it look something like
:INPUT ACCEPT [0:0]
(...)
:Enemies - [0:0]
(...)
-A INPUT -j Enemies
Then in another iptables config file, I can add entries to the Enemies table with lines like:
-A Enemies -s 99.99.99.99 -j DROP
The nice thing is that I can clear and rebuild the Enemies table without the fear that I delete important entries like the SSH lines.
The other nice thing is that I can dynamically add and delete entries in the Enemies table with a script, without worrying about the sequence the lines are processed. The location of the -A INPUT -j Enemies line defines where the enemies are rejected, not the sequence in which enemies are added or deleted to the firewall.

dstiles

2:14 pm on Apr 5, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Thanks, lammert. That feature never occurred to me. I've just begun (again!) to RTFM - I'll pay closer attention this time.

In the manual there seems no mention of this syntax. I even searched on ":" with no relevant match.

So the Enemies rules do not require an INPUT/OUTPUT in the Enemies file, just the -s and the -j DROP? I assume the Enemies etc files also live in /etc/iptables. Does the table name *Enemies need to be included in that file?

I also assume that a "systemctl restart iptables" command needs to be issued following a change to one of the rules files. Does this in any way affect the current SSH connection?

One of the things I notice is: in my original rules, cribbed mostly from an online source, there are lines with the action REJECT. I assume it would be better to use DROP, is that correct?

> the sequence in which enemies are added or deleted

I guess the correct location in mine would be:
(...)
# my rules
:Ranges - [0:0]
:Dynamics - [0:0]
# my actions
-A INPUT -j Ranges
-A INPUT -j Dynamics
# existing rules
-A INPUT -j REJECT
-A FORWARD -j REJECT
COMMIT

What are the [0,0] bits for? At the head of my old iptables it has:
:INPUT ACCEPT [4927:1439976]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [4896:2452260]

lammert

2:42 pm on Apr 5, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Depending on the system configuration, the system may save the current state of the firewall to the configuration file. The values between [:] are the number of packets and the number of bytes processed before shutdown. In your case, the INPUT table processed 4927 packets and 1439976 bytes since the contents of the firewall tables was last saved to disk.

dstiles

2:54 pm on Apr 5, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Ok. Thanks.

dstiles

10:30 am on Apr 6, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Not sure this is working. I have three tables (one currently empty) defined as above. The systemctl command fails (no file found - I'm having trouble with systemctl anyway) so I used iptables restore. Then iptables -L shows (eg):
Ranges all -- anywhere anywhere
(...)
Chain Ranges (1 references)
target prot opt source destination

but iptables -t Ranges -n -L returns "table Ranges does not exist".

I am unsure if iptables-restore is supposed to load the external tables (I can find no reference in the manual to external tables at all) but apparently the external tables cannot be loaded. Any advice, please?

lammert

11:02 am on Apr 6, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Try iptables -n -L Ranges instead. The -t switch lets you switch between the main tables, i.e. nat, mangle, raw, security and filter. You want to see the chain Ranges within the filter table.

dstiles

1:31 pm on Apr 6, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



That gives just a pair of Chain and target rules. Same with iptables -L Ranges.

dstiles

2:48 pm on Apr 7, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



lammert, I'm having no luck with this. The three table-files I've added are not being loaded. I've used iptables-restore and even rebooted the VPS, but the only thing that works in the sparse rules-v4 in /etc/iptables. One of my postings above mentioned putting the new files into /etc/iptables - you made no comment so I assumed that was correct. I've searched the OS for other locations of iptables but with no relevant result. Some references I've viewed online mention additional tables but only within the primary rules file.

Could you help, please?

lammert

11:26 pm on Apr 7, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



iptables-restore reads from stdin. You can feed it with any file with the command
cat somefile | iptables-restore

dstiles

8:31 am on Apr 8, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Yes, I know that, I've been using it for years in the form:
sudo iptables-restore < /etc/iptables/rules.v4
but it does not show the content of the external tables, only the references to the table itself.

What I can't get is an input from the three external files. Are they in the correct folder (/etc/iptables - same as rules.v4)? Do they need any special addressing from (eg) a config file? I have added the table name at the top of each file (eg: *Ranges) but no difference.

My current rules.v4 file includes, at the end:
(...)
# my rules
:Ranges - [0:0]
:Ranges_80_443 - [0:0]
:Dynamics - [0:0]
# my actions
-A INPUT -j Ranges
-A INPUT -j Ranges_80_443
-A INPUT -j Dynamics
# existing rules
-A INPUT -j REJECT
-A FORWARD -j REJECT
COMMIT

The Ranges file contains a number of entries along the lines of:
-A Ranges -s 20.192.0.0/10 -p tcp -m multiport --dports 443 -j DROP

Ranges_80_443 is similar but with no ports or protocol. The Dynamics one is still empty.

lammert

9:13 am on Apr 8, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Iptables does not support file includes on the kernel level. All loading of data has to be done with the mediocre command line tools which don't have an include functionality either. The following sequence should load all the rules and then output all the rules:
sudo cat /etc/iptables/rules.v4 /etc/iptables/Ranges.v4 /etc/iptables/Ranges_80_443.v4 | iptables-restore
sudo iptables -L -n -v -x
You can also load the rules with three separate commands, but the --noflush parameter has to be used to prevent iptables-restore to delete the contents of the previous loads.
sudo iptables-restore < /etc/iptables/rules.v4
sudo iptables-restore --noflush < /etc/iptables/Ranges.v4
sudo iptables-restore --noflush < /etc/iptables/Ranges_80_443.v4
sudo iptables -L -n -v -x

dstiles

12:39 pm on Apr 8, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Ah. Sorry, I misunderstood. I thought, from your original posting, that the extra config files would be automatically loaded from the iptables directory assuming they were named the same as the tables in the primary rules file.

So, the solution is to hold the primary rules and the secondary tables somewhere convenient and then use them to restore to the in-memory iptables, I assume with a bash file for convenience. Now I understand. :)

Many thanks for your help and for the time.

lammert

1:24 pm on Apr 8, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



No need for a sorry :) IPtables is an incredible powerful concept in the kernel, but the userland tools to access it from the command line seem to have been written by a toddler. They lack the versatility and scalability which are normal for many other command line utilities in Linux. The only practical use for these tools is to embed them in higher level scripts like the firewalld daemon and fail2ban.

dstiles

3:50 pm on Apr 8, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Yes, there are a few little quirks like that in linux. But as long as they are known they can usually be worked around. The problem is that people who are unfamiliar with the quirks (me, in this case) have trouble finding out about them.

I found a minor hiccup in the cat sequence. There was a COMMIT at the end of the first file which it complained about. I tried it at the end of the final file and it was fine (at least, in test mode). I did try without a COMMIT but got another complaint.

What I intend doing is to put the cat string into a bash file, concatenating rules.v4 (minus the COMMIT), Ranges, Ranges_80_443 and Dynamics, and finishing it off with a short COMMIT file for convenience.

I assume there is no need to place the sections #myactions and and #existing from my previous posting into the COMMIT file?

Thanks again for the help. :)

dstiles

3:16 pm on Apr 11, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I thought I should report back on this.

My initial test suggested that sudo cat /etc/iptables/rules.v4 etc worked but the source files were not then properly populated. Once they had proper content I could not get that solution to work, nor the restore noflush suggestion.

Solutions I tried included careful checking of all files and moving or removing the COMMIT line between files but nothing seemed to work. It's taken me several hours to come to a compromise.

Concatenate all the different files into a single file, removing all comments (which caused a failure if between files) and putting the COMMIT at the end of the final concatenated file. Then iptables-restore < /etc/iptables/concatfile. Everything stays with it own table and leaves room to add new entries into the appropriate table. Once a working iptables is live and has some extra IPs added the table can be permanently saved using iptables-save > /etc/iptables/concatfile.

dstiles

4:19 pm on Apr 17, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



To continue the saga:

Whatever I do I cannot get iptables to block baddies. In desperation I've reduced the file to a single table and still it does not work. The basic table is as shown. I've also tried the port 80/443 pair immediately before the COMMIT and in both cases commented them out to see what happens - if they are not in place where they are the web site access gets blocked.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -d 127.0.0.0/8 ! -i lo -j REJECT --reject-with icmp-port-unreachable
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport (my ssh port) -j ACCEPT
-A INPUT -s (my IP)/32 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -j ACCEPT
-A INPUT -s 111.0.0.0/10 -p tcp --dport 443 -j DROP
-A INPUT -s 166.70.0.0/16 -p tcp --dport 443 -j DROP
-A INPUT -s 58.48.0.0/12 -j DROP
(etc)
-A INPUT -s 13.124.0.0/14 -p tcp --dport 443 -j DROP
-A INPUT -s 81.168.89.142/32 -p tcp --dport 443 -j DROP
COMMIT

This is running on debian buster with iptables and iptables-persistent installed. I know the basic start section works because I accidentally locked myself out at the beginning. I'm wondering if I should cut my losses and begin over with netfilter, but am worried that a) it will make no difference and b) I will make a hash of it.

lammert

6:04 pm on Apr 17, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



IPtables rules are executed from top to bottom. Line 8 and 9 accept all traffic to ports 80 and 443. Therefore the rules later in the list to block bad traffic are never reached. The ACCEPT lines have to be after all DROP lines for bad neighborhoods.

dstiles

9:28 pm on Apr 17, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



That's why I tried moving them, but the whole server is blocked then. I'll try them there again and see what happens this time... Same. Completely blocked except for my own access from the IP ACCEPT at the top. I only moved the port 80/443 pair as the others are for unrelated features - aren't they?

lammert

10:28 pm on Apr 17, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Your server blocks on the following line, it rejects all packets to all ports and sends a port unreachable notification back.
-A INPUT -j REJECT --reject-with icmp-port-unreachable
You should be safe with the following. It first accepts all localhost traffic and all established connections. This is for performance reasons. There is no need to check if a connection is from a bad source if the TCP handshake sequence was already successful. Then it jumps to the Enemies chain and processes all the DROP rules in the chain. If no enemy was found, processing is continued with scanning for new traffic on port 80, 443 and your SSH port.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:Enemies - [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -d 127.0.0.0/8 ! -i lo -j REJECT --reject-with icmp-port-unreachable
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -j Enemies
-A INPUT -p tcp -m state --state NEW --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport 443 -j ACCEPT
-A INPUT -p tcp -m state --state NEW --dport (my ssh port) -j ACCEPT
-A INPUT -s (my IP)/32 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -j ACCEPT
-A Enemies -s 1.1.0.0/10 -p tcp --dport 443 -j DROP
-A Enemies -s 2.2.0.0/16 -p tcp --dport 443 -j DROP
-A Enemies -s 3.3.0.0/12 -j DROP
(etc)
-A Enemies -s 4.4.0.0/14 -p tcp --dport 443 -j DROP
-A Enemies -s 5.5.5.5/32 -p tcp --dport 443 -j DROP
COMMIT
Added bonus with this setup with the Enemies chain is that you can manually add extra IP blocks while the firewall is running with a simple command line call:
iptables -A Enemies -s 11.11.11.0/24 -j DROP
This will add an extra rule to the end of the Enemies chain. If you then run iptables-save, that new rule is included in the saved configuration.

dstiles

11:44 am on Apr 18, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've spent the past couple of hours putting this all together with three separate tables and testing it.

It works! Thank you very much!

My previous experience of iptables was limited to the top four lines followed by loads of DROPs. On this new server the iptables basics was put in place by the hosting company and I just added my previous block list to it, with mods as to tables as discussed earler in this thread. I've gained a lot of insight into iptables now. Again, many thanks! :)