SOLVED TrueNAS-13.0-U4 - ipfw issue loading custom rules

[Stane1983]

Hi guys,

here is my conf...
TrueNAS CORE version: TrueNAS-13.0-U4.
Service enabled: OpenVPN Server
Jails: Deluge

Please note that everything is working as expected if I manually execute ipfw script (I can access Jails and internal LAN) from shell:

Code:

#!/bin/sh

# Flush out the list before we begin.
ipfw -q -f flush

# Set rules command prefix
cmd="ipfw -q add"
# interface name of NIC attached to Internet
net="em0"

#################################################################

ipfw -q nat 462 config if $net

# OpenVPN
$cmd 0050 nat 1 all from 10.20.0.1/24 to any out via $net
$cmd 0100 nat 1 all from any to any in via $net

# Jails
$cmd 0050 nat 462 ipv4 from 172.16.0.0/24 to any out via $net
$cmd 0100 nat 462 ip4 from any to any in via em0

ipfw -q nat 1 config if $net

Script won't run if I set it as firewall_script in Tunables or as Init script executed as post init.

After boot my ipfw show looks like this:

Code:

00100   246   34488 nat 462 ip4 from 172.16.0.0/24 to any out via em0
00101   961  173901 nat 462 ip4 from any to any in via em0
65535 18031 1422093 allow ip from any to any

And after manually running script it looks like this:

Code:

00050     0       0 nat 1 ip from 10.20.0.0/24 to any out via em0
00050     0       0 nat 462 ip4 from 172.16.0.0/24 to any out via em0
00100    10     660 nat 1 ip from any to any in via em0
00100     0       0 nat 462 ip4 from any to any in via em0
65535 18292 1472380 allow ip from any to any

I'm fairly new to TrueNAS but I do have general idea how it should work as I'm long time linux user.

Any ideas?

Thanks,
Stan

 

[Samuel Tai]

Welcome to the TrueNAS forums!

iX disables ipfw by default, mostly because it's too easy to screw up and completely firewall off your TrueNAS Core installation. Since it sounds like you know what you're doing, and understand enabling ipfw will result in an unsupported configuration, set a rc.conf tunable firewall_enable="YES" alongside your firewall_script tunable to allow ipfw to start on boot.

 

[Stane1983]

Welcome to the TrueNAS forums!

iX disables ipfw by default, mostly because it's too easy to screw up and completely firewall off your TrueNAS Core installation. Since it sounds like you know what you're doing, and understand enabling ipfw will result in an unsupported configuration, set a rc.conf tunable firewall_enable="YES" alongside your firewall_script tunable to allow ipfw to start on boot.

I forgot to mention, all tunables are set, including firewall_enable. Here are tunables:

Code:

firewall_enable="yes" (RC)
firewall_script="/mnt/Data/apps/scripts/firewall.sh" (RC)
firewall_type="open" (RC)
gateway_enable="yes" (RC)
natd_enable="yes" (RC)
natd_flags="dynamic -m" (RC)
natd_interface="em0" (RC)
net.inet.ip.forwarding=1 (SYSCTRL)

 

[Samuel Tai]

OK, then try changing your rc.conf type tunables to loader type tunables. It could be things aren't working because things need to be set before networking comes up.

 

[Stane1983]

OK, then try changing your rc.conf type tunables to loader type tunables. It could be things aren't working because things need to be set before networking comes up.

Same thing...

 

[Samuel Tai]

I forgot to mention loader tunables only take effect on the next boot. Did you reboot after changing the type?

 

[Stane1983]

I forgot to mention loader tunables only take effect on the next boot. Did you reboot after changing the type?

Yes, no luck...

Just won't load script. Or maybe my script gets executed but something overwrite my rules? I'm not at home atm but will add to script line that creates temp file and will update here.

 

[Stane1983]

Update: Script gets loaded with rc.conf tunable firewall_script (I added to script that it creates temp file) but it looks like something else overwrites my rules to default ones?

 

[wyatt.mcgehee]

@Stane1983, did you ever get that working?
I seem to have no problem getting things to stick on boot.
Here's my config, if it may help you at all.

My ipfwrules.sh looks like this:

Code:

#!/bin/bash
ipfw -q -f flush
ipfw -q nat 1 config if igb0
# OpenVPN requires NAT
ipfw -q add nat 1 all from 10.9.8.0/24 to any out via igb0
ipfw -q add nat 1 all from any to any in via igb0
# Jails that require NAT
ipfw -q add nat 2 all from 172.16.0.0/24 to any out via igb0
ipfw -q add nat 2 all from any to any in via igb0

'System > Tunables' look like this:

I don't have a net.inet.ip.forwarding=1 SYSCTL tunable,
but sysctl net.inet.ip.forwarding returns '1' for me.

'Tasks > Init/Shutdown Scripts' looks like this:

Best of luck!

 

[Stane1983]

Ok, I found solution for my issue but it is actually really dumb one :)

I left firewall_script tunable (type RC) to call script with content:

Code:

#!/bin/bash

/mnt/Data/apps/scripts/real_firewall.sh &

and real_firewall.sh looks like this:

Code:

#!/bin/sh

# Wait 1 minute before executing
sleep 60

# Flush out the list before we begin.
ipfw -q -f flush

# Set rules command prefix
cmd="ipfw -q add"
# interface name of NIC attached to Internet
net="em0"

#################################################################

ipfw -q nat 462 config if $net

# OpenVPN
$cmd 0050 nat 1 all from 10.20.0.1/24 to any out via $net
$cmd 0100 nat 1 all from any to any in via $net

# Jails
$cmd 0050 nat 462 ipv4 from 172.16.0.0/24 to any out via $net
$cmd 0100 nat 462 ip4 from any to any in via em0

ipfw -q nat 1 config if $net

it works for me but obviously its some kind of bug. I had to use 2 scripts because sleep command freezes boot process so I had to execute it in background.

 

[kole5o]

Hi, this solution work perfect, until start some plugins in jail.
when nat and nat port forwarding set in jail this python script run on jail start
/usr/local/lib/python3.9/site-packages/iocage_lib/ioc_start.py
row 1742 run command "ipfw -q flush'" and bye bye custom ipfw rules
somebody know solution to combine custom ipfw rules and embeded true nas nat port forwardig functional?

 

[kole5o]

the solution is truenas scale + init script for iptables placed on /mnt/zpool/sys-persistent/iptables.rules
chmod a+x iptables.rules
something like that
#!/bin/sh
/usr/sbin/iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
/usr/sbin/iptables -A INPUT -p icmp -j ACCEPT
/usr/sbin/iptables -A INPUT -i lo -j ACCEPT
/usr/sbin/iptables -A INPUT -s "somenetwork"/24 -p tcp -m tcp --dport 22 -j ACCEPT
/usr/sbin/iptables -A INPUT -s "somenetwork"/24 -p tcp -m tcp --dport 443 -j ACCEPT
/usr/sbin/iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited

with this config starting some apps like minio dont delete iptables rules and insert their own kube rules in Chain INPUT upper custom.
So minio ports 9000 9002 accessible from any, but other like ssh and web accessible only from "somenetwork"