Debian (Devuan) Linux jail network access problems

Chris123

Dabbler
Joined
Feb 21, 2016
Messages
34
I got a Debian (Devuan) Linux jail up and running on Freenas by following these instructions: https://forums.freebsd.org/threads/setting-up-a-debian-linux-jail-on-freebsd.68434/ (I had to make a few changes in order to make it work on FreeNAS).

So far it seems to work well EXCEPT for one issue: Network access. This issue is not specific to FreeNAS, I get the same problem on a pure FreeBSD-install as well. Other users have reported the same problem, but to my knowledge there hasn't been any solution to this yet. It seems to be an ifconfig / inet / route issue, but everything I've tried have failed.

I'm hoping someone might have some ideas or be able to point me in the right direction on how to configure ifconfig / inet routing.

devuan.png


Trying ifconfig eth0 inet 192.168.1.113 netmask 255.255.255.0 gives the following error:

devuan2.png


Added the following to /etc/network/interfaces without any success
Code:
iface eth0 inet static
    address 192.168.1.113/24
    gateway 192.168.1.1



Iocage config.json for 'debjail':
Code:
{
    "CONFIG_VERSION": "14.1",
    "allow_chflags": "0",
    "allow_mlock": "0",
    "allow_mount": "1",
    "allow_mount_devfs": "1",
    "allow_mount_nullfs": "0",
    "allow_mount_procfs": "1",
    "allow_mount_tmpfs": "1",
    "allow_mount_zfs": "0",
    "allow_quotas": "0",
    "allow_raw_sockets": "1",
    "allow_set_hostname": "1",
    "allow_socket_af": "0",
    "allow_sysvipc": "0",
    "allow_tun": "0",
    "available": "readonly",
    "basejail": "no",
    "boot": "off",
    "bpf": "no",
    "children_max": "0",
    "cloned_release": "EMPTY",
    "comment": "none",
    "compression": "lz4",
    "compressratio": "readonly",
    "coredumpsize": "off",
    "count": "1",
    "cpuset": "off",
    "cputime": "off",
    "datasize": "off",
    "dedup": "off",
    "defaultrouter": "none",
    "defaultrouter6": "none",
    "depends": "none",
    "devfs_ruleset": "4",
    "dhcp": "off",
    "enforce_statfs": "2",
    "exec_clean": "1",
    "exec_fib": "0",
    "exec_jail_user": "root",
    "exec_poststart": "/usr/bin/true",
    "exec_poststop": "/usr/bin/true",
    "exec_prestart": "/usr/bin/true",
    "exec_prestop": "/usr/bin/true",
    "exec_start": "/etc/init.d/rc 3",
    "exec_stop": "/etc/init.d/rc 0",
    "exec_system_jail_user": "0",
    "exec_system_user": "root",
    "exec_timeout": "60",
    "host_domainname": "none",
    "host_hostname": "debjail",
    "host_hostuuid": "debjail",
    "host_time": "yes",
    "hostid": "744f9272-1cb5-11e8-919c-6c0b84084db5",
    "hostid_strict_check": "off",
    "interfaces": "vnet0:bridge0",
    "ip4": "new",
    "ip4_addr": "em0|192.168.1.113/24",
    "ip4_saddrsel": "1",
    "ip6": "new",
    "ip6_addr": "none",
    "ip6_saddrsel": "1",
    "jail_zfs": "off",
    "jail_zfs_dataset": "iocage/jails/debjail/data",
    "jail_zfs_mountpoint": "none",
    "last_started": "2019-12-17 06:27:44",
    "login_flags": "-f root",
    "mac_prefix": "02ff60",
    "maxproc": "off",
    "memorylocked": "off",
    "memoryuse": "off",
    "mount_devfs": "1",
    "mount_fdescfs": "0",
    "mount_linprocfs": "1",
    "mount_procfs": "1",
    "mountpoint": "readonly",
    "msgqqueued": "off",
    "msgqsize": "off",
    "nmsgq": "off",
    "notes": "none",
    "nsemop": "off",
    "nshm": "off",
    "nthr": "off",
    "openfiles": "off",
    "origin": "readonly",
    "owner": "root",
    "pcpu": "off",
    "priority": "99",
    "pseudoterminals": "off",
    "quota": "none",
    "release": "EMPTY",
    "reservation": "none",
    "resolver": "/etc/resolv.conf",
    "rlimits": "off",
    "securelevel": "2",
    "shmsize": "off",
    "stacksize": "off",
    "stop_timeout": "30",
    "swapuse": "off",
    "sync_state": "none",
    "sync_target": "none",
    "sync_tgt_zpool": "none",
    "sysvmsg": "new",
    "sysvsem": "new",
    "sysvshm": "new",
    "template": "no",
    "type": "jail",
    "used": "readonly",
    "vmemoryuse": "off",
    "vnet": "off",
    "vnet0_mac": "none",
    "vnet1_mac": "none",
    "vnet2_mac": "none",
    "vnet3_mac": "none",
    "vnet_default_interface": "auto",
    "vnet_interfaces": "none",
    "wallclock": "off"
}


And yes, I know Linux can easily run in a Bhyve VM.
 
Last edited:

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Jails are having issues with static IPs at the moment.. possibly fixed in 11.3.

Can you get a DHCP address? (would need to set bpf=yes)
 

Chris123

Dabbler
Joined
Feb 21, 2016
Messages
34
Jails are having issues with static IPs at the moment.. possibly fixed in 11.3.

Can you get a DHCP address? (would need to set bpf=yes)

Thanks for your reply!

Ok, so I set bpf=yes (and vnet=on, required when setting bpf=yes I guess). I was able to start and log in to the jail, but now ifconfig returned nothing. Could it be that Devuan doesn't recognize the vnet interface? Seems to be some mismatch between what iocage is providing to the jail OS (devuan) and what the jail OS (devuan) is able to interpret....

Here's my output:
devuan3.png
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
How about running ip address or dhclient in the jail?

Also setting dhcp=on (I guess I should have said that too... my bad).

The IP in caps is the interface, not my typing, use lower case.

Also, I think there's something to do to create the vnet interfaces in the jail... it's in a script somewhere for the native jails, but I guess not in devuan. It may be worth searching around for it as I think the concept here is dangerously close to the holy grail of having a jail where we could run a supported version of docker and mount storage directly into it from FreeNAS for use by the docker containers.... I know, some will say that sounds like the product that shall not be named, but whis would effectively be doing it with Linux, so maybe it can be different this time?
 
Last edited:

Chris123

Dabbler
Joined
Feb 21, 2016
Messages
34
How about running IP address or dhclient in the jail?

Also setting dhcp=on (I guess I should have said that too... my bad).

The IP in caps is the interface, not my typing, use lower case.

Also, I think there's something to do to create the vnet interfaces in the jail... it's in a script somewhere for the native jails, but I guess not in devuan. It may be worth searching around for it as I think the concept here is dangerously close to the holy grail of having a jail where we could run a supported version of docker and mount storage directly into it from FreeNAS for use by the docker containers.... I know, some will say that sounds like the product that shall not be named, but whis would effectively be doing it with Linux, so maybe it can be different this time?

When dhcp=on the jail doesn't start. Seems like it is looking for /etc/rc.conf, which will of course result in an error when the jail is not FreeBSD. I guess it want to add ifconfig_epair0b="DHCP". Is there a way to direct iocage to use another file?

1576808170173.png


I tried to add a rc.conf (with 'debjail' as hostname) just to see what would happen

1576820097581.png


Since I'm not able to enter the jail when using DHCP, i was only able to try IP address and dhclient in a static-ip-configuration:
1576821640481.png


Seems like the network interfaces is not configured properly in the jail...?

I agree, having a near full-fledged (non-gui) Linux jail would open some possibilities. All the software I'm planning use ran without any issues (besides network access).
 
Last edited:

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
maybe the next thing to try is ifconfig -a (to see if there are any network interfaces at all, even if they may be inactive).

Also, ip address is intended to be the command, don't subsitute the actual address (I would have used ip <address> in that case).

Maybe some of the stuff here will provide a way forward if there are no interfaces at all:
 
Last edited:

Chris123

Dabbler
Joined
Feb 21, 2016
Messages
34
ifconfig -a shows that eth0 and lo0 are "active", so Devuan must somehow see a "network adapter".
1576863784675.png


Also, there are some statistics of the network adapter in /proc/net/dev
1576865087660.png


ip address produced the same error as when using ip 192.168.x.x.:
1576863974050.png


I had a quick look at the link you sent, I see there are some things to try out. It also led me to look into how networking gets initialized on Devuan through init.d and run levels rc0.d, rc1.d, rc2.d,....,rc6.d. The devuan jail uses run level 2 (rc2.d) by default, which supposedly is without networking according to this, but after installing a pure Devuan VM in VIrtualbox it turns out that it also uses run level 2 (with a working network). But I will compare the two in more detail to see if anything obvious is missing.
 
Last edited:

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
You could try ifconfig eth0 192.168.x.x netmask 255.255.255.0 up

That might work, but I'm not sure if you will then need to somehow specify a default gateway.

Possibly also useful to do ifconfig lo0 127.0.0.1 up
 

Chris123

Dabbler
Joined
Feb 21, 2016
Messages
34
ifconfig eth0 192.168.1.113 netmask 255.255.255.0 up and ifconfig lo0 127.0.0.1 up gave error. I tried to google the errors but didn't find anything that made sense to me..

1577020741611.png


Yesterday I tried to install different versions of Devuan (oldstable, testing, and unstable) and also trying on FreeBSD 13 with linux compatibility layer kernel 3.2. Nothing worked. Same issue. (Not sure I understand how the Linux compatibility layer kernel integrates into the linux OS on FreeBSD, 2.6.32 and even 3.2 is a big difference form the kernel version 4.x that comes with Devuan/Debian stable ... can that cause compatibility issues?).
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
How about ifconfig eth0 up

I agree there's sufficient mysterious magic going on here to have me at a loss regarding how it all works.

Based on the last results, maybe the linux commands are the right ones rather than the FreeBSD ones.
 

vesper

Cadet
Joined
May 22, 2023
Messages
3
Sorry for necroposting, but to those who got same problem there is solution:
  • copy ifconfig and dhclient utitilites from TrueNAS host /rescue to jail: cp /rescue/ifconfig /mnt/Storage/iocage/jails/debjail/root/sbin/ where /mnt/Storage/ will be path to your pool with iocage
  • check how interface named in host OS (em0/re0/eth0 etc) in my case re0
  • add IP manualy iocage set ip4_addr="re0|192.168.88.200/24" debjail
  • restart jail iocage restart debjail where debjail is my jail name
It should work now.
Code:
* Starting debjail
  + Started OK
  + Using devfs_ruleset: 1004 (cloned from devfs_ruleset 0)
  + Using IP options: ip4.addr=re0|192.168.88.201/24 ip4.saddrsel=1 ip4=new ip6.saddrsel=1 ip6=new
  + Starting services OK
  + Executing poststart OK


Note: vnet, dhcp, bpf should be disabled
iocage set dhcp=0 debjail iocage set bpf=0 debjail iocage set vnet=0 debjail
 
Top