iptables and combining additional rule sources
Hi Everyone, We are having trouble with our MediaWiki installation on a low-end VM. The VM is servicing a lot of spam traffic, and it is driving cpu usage up to about 80%. The 404's appear to be more expensive then the 200's. GoDaddy wrote to us and told us they were going to suspend our service if we don't get cpu usage down. I experimented with several Apache and MediaWiki plugins and I have a design I like. The plugin scans the URL, detects the problematic URLs, and sends the ip address to a privileged out-of-proc proxy to update iptables. The proxy is privileged and can update iptables rules. It also maintains a database to remove the host after 45 days. The problem I am having is, adding the new information to the existing iptables rules in /etc/sysconfig/iptables. I want to write my rules to a separate file and then tell /etc/sysconfig/iptables to include it at the correct position. I read the iptables(8), iptables-save(8) and iptables-restore(8) man pages, but I don't see how to combine the different sources. How do I tell iptables to include a second external source at a specific location? # iptables --version iptables v1.4.21 Thanks in advance. ========================= Here is an example of /etc/sysconfig/iptables with the position I want to insert the MediaWiki ban rules. # cat /etc/sysconfig/iptables *nat :PREROUTING ACCEPT [4276:232374] :POSTROUTING ACCEPT [270:136514] :OUTPUT ACCEPT [270:136514] COMMIT *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [269:205262] -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT ### I want to insert rules here ### *include my-mediawiki-rules ### Back to normal rules ### -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT ...
On Fri, 24 Apr 2020 18:28:21 -0400, Jeffrey Walton said:
The problem I am having is, adding the new information to the existing iptables rules in /etc/sysconfig/iptables. I want to write my rules to a separate file and then tell /etc/sysconfig/iptables to include it at the correct position.
I read the iptables(8), iptables-save(8) and iptables-restore(8) man pages, but I don't see how to combine the different sources.
How do I tell iptables to include a second external source at a specific location?
Turn the problem on its side.... #!/bin/bash cat /etc/iptables.header /etc/iptables.newstuff /etc/iptables.trailer > /etc/sysconfig/iptables iptables-restore < /etc/sysconfig/iptables (basically the solution I did for an NFS server, where 'newstuff' and /etc/exports were both machine-generated by a perl script that read a config file of authorized clients.
On Fri, Apr 24, 2020 at 11:07 PM Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:
On Fri, 24 Apr 2020 18:28:21 -0400, Jeffrey Walton said:
The problem I am having is, adding the new information to the existing iptables rules in /etc/sysconfig/iptables. I want to write my rules to a separate file and then tell /etc/sysconfig/iptables to include it at the correct position.
I read the iptables(8), iptables-save(8) and iptables-restore(8) man pages, but I don't see how to combine the different sources.
How do I tell iptables to include a second external source at a specific location?
Turn the problem on its side....
#!/bin/bash cat /etc/iptables.header /etc/iptables.newstuff /etc/iptables.trailer > /etc/sysconfig/iptables iptables-restore < /etc/sysconfig/iptables
(basically the solution I did for an NFS server, where 'newstuff' and /etc/exports were both machine-generated by a perl script that read a config file of authorized clients.
That's a good idea. I think that may work better for some data sets. One last question... Should I create my own target - say mediawiki - and append my rules to it? That may simplify things: * Header, newstuff and trailer is fixed * newstuff just jumps to mediawiki target * if mediawiki does not ban, then control returns to trailer Then, my out-of-proc service just keeps adding to mediawiki target. I don't need to write files in this case. I'll just keep adding to the running config. Jeff
On Sat, 25 Apr 2020 02:55:08 -0400, Jeffrey Walton said:
One last question... Should I create my own target - say mediawiki - and append my rules to it? That may simplify things:
* Header, newstuff and trailer is fixed * newstuff just jumps to mediawiki target * if mediawiki does not ban, then control returns to trailer
Then, my out-of-proc service just keeps adding to mediawiki target. I don't need to write files in this case. I'll just keep adding to the running config.
That totally fails if your machine reboots, because there's no memory of what the rules were before the reboot.
On Sat, Apr 25, 2020 at 12:53 PM Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:
On Sat, 25 Apr 2020 02:55:08 -0400, Jeffrey Walton said:
One last question... Should I create my own target - say mediawiki - and append my rules to it? That may simplify things:
* Header, newstuff and trailer is fixed * newstuff just jumps to mediawiki target * if mediawiki does not ban, then control returns to trailer
Then, my out-of-proc service just keeps adding to mediawiki target. I don't need to write files in this case. I'll just keep adding to the running config.
That totally fails if your machine reboots, because there's no memory of what the rules were before the reboot.
Yeah, I'm OK with that. Dropping the database means the code simplifies _a lot_. I can remove the administrivia, connection information and all the code for inserts and deletes. Looking at the logs, these folks are aggressive. It looks like (to me) the wiki installation will see most spammers in the first 5 or 10 minutes of starting up. After initial startup the system reaches stability rather quickly. I think it is a good tradeoff. Jeff
Assuming these IP address are treated the same way in your iptables rule, ipset may help to make it simpler. On Fri, Apr 24, 2020 at 3:30 PM Jeffrey Walton <noloader@gmail.com> wrote:
Hi Everyone,
We are having trouble with our MediaWiki installation on a low-end VM. The VM is servicing a lot of spam traffic, and it is driving cpu usage up to about 80%. The 404's appear to be more expensive then the 200's. GoDaddy wrote to us and told us they were going to suspend our service if we don't get cpu usage down.
I experimented with several Apache and MediaWiki plugins and I have a design I like. The plugin scans the URL, detects the problematic URLs, and sends the ip address to a privileged out-of-proc proxy to update iptables. The proxy is privileged and can update iptables rules. It also maintains a database to remove the host after 45 days.
The problem I am having is, adding the new information to the existing iptables rules in /etc/sysconfig/iptables. I want to write my rules to a separate file and then tell /etc/sysconfig/iptables to include it at the correct position.
I read the iptables(8), iptables-save(8) and iptables-restore(8) man pages, but I don't see how to combine the different sources.
How do I tell iptables to include a second external source at a specific location?
# iptables --version iptables v1.4.21
Thanks in advance.
=========================
Here is an example of /etc/sysconfig/iptables with the position I want to insert the MediaWiki ban rules.
# cat /etc/sysconfig/iptables *nat :PREROUTING ACCEPT [4276:232374] :POSTROUTING ACCEPT [270:136514] :OUTPUT ACCEPT [270:136514] COMMIT
*filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [269:205262]
-A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT
### I want to insert rules here ### *include my-mediawiki-rules
### Back to normal rules ### -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT -A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT ...
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
<snip>
The problem I am having is, adding the new information to the existing iptables rules in /etc/sysconfig/iptables. I want to write my rules to a separate file and then tell /etc/sysconfig/iptables to include it at the correct position.
I read the iptables(8), iptables-save(8) and iptables-restore(8) man pages, but I don't see how to combine the different sources.
How do I tell iptables to include a second external source at a specific location? <snip>
1 - Get the iptables rules list with the line numbers enabled $ iptables -nL –line-numbers 2 - Look up the line number you want to use and insert your rule. ( I am inserting a rule at line number 10, the existing rule will shift down) $ sudo /sbin/iptables -I INPUT 10 -s 202.54.1.1 -j DROP -m comment --comment "DROP spam IP address - " 3 - Save the rules to a file in etc so you can reload them at the next reboot $ sudo /sbin/iptables-save > /etc/iptables.local 4 - To make the rules persistent, add the following rule to your /etc/rc.local file) sudo /sbin/iptables-restore < /etc/iptables.local If your system reboots you will not loose the rules now. Aruna
On 2020-04-25 00:28, Jeffrey Walton wrote:
Hi Everyone,
We are having trouble with our MediaWiki installation on a low-end VM. The VM is servicing a lot of spam traffic, and it is driving cpu usage up to about 80%. The 404's appear to be more expensive then the 200's. GoDaddy wrote to us and told us they were going to suspend our service if we don't get cpu usage down.
I experimented with several Apache and MediaWiki plugins and I have a design I like. The plugin scans the URL, detects the problematic URLs, and sends the ip address to a privileged out-of-proc proxy to update iptables. The proxy is privileged and can update iptables rules. It also maintains a database to remove the host after 45 days.
Hi Jeffrey, have you looked into Fail2Ban? It seems to do what you need, but real-time. Kind regards, Thorondir
participants (5)
-
Aruna Hewapathirane -
Jeffrey Walton -
Keh-Ming Luoh -
Thorondir -
Valdis Klētnieks