docker-freebsd in iocage

[NetSoerfer]

I'm trying to install docker-freebsd in an iocage jail. I realise that docker-freebsd is far from stable and currently doesn't even have a maintainer, but I'd still like to try.

I've created a VNET iocage jail:

iocage create --name docker -r LATEST
ip4_addr="vnet0|10.1.1.2/24"
vnet="on" defaultrouter="10.42.1.1"
boot="on" host_hostname="docker"
allow_raw_sockets="1"
mount_devfs="1"
allow_mount_devfs="1"
allow_mount_zfs="1"
allow_mount="1"
enforce_statfs="0"


I managed to fix a couple of errors:

root@docker:/ # docker -d
netstat: kvm not available: /dev/mem: No such file or directory

Fixed by editing /usr/local/etc/rc.d/docker with ${docker_nat_iface=epair0b} in the jail.

root@docker:/ # docker -d
WARN[0000] Kernel version detection is available only on linux
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
FATA[0000] Error starting daemon: error initializing graphdriver: Cannot find root filesystem tank/iocage/jails/docker/root: exit status 1: "/sbin/zfs zfs get -rHp -t filesystem all tank/iocage/jails/docker/root" => cannot open 'tank/iocage/jails/docker/root': dataset does not exist

Fixed with root@pegasus:~ # zfs jail 32 freenas-boot/ROOT/11.1-U5.

Question: How do I persist this, as I believe this is undone on reboot? What happens when I update to 11.1-U6 or 11.2?

root@docker:/ # docker -d
WARN[0000] Kernel version detection is available only on linux
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
WARN[0000] Running kldload pf failed with message: , error: exit status 1
WARN[0000] port allocator - using fallback port range 49153-65535 due to error: open /proc/sys/net/ipv4/ip_local_port_range: no such file or directory
INFO[0000] Bridge interface not found, trying to create it
FATA[0000] Error starting daemon: unknown exec driver native

Um, I'm lost here. Any suggestions on how to proceed?

 

[m0nkey_]

Docker on FreeNAS requires both Docker and Linux support. FreeNAS does not have the linux-c7 emulation, so it'll make it impossible to get working. In order to use Docker on FreeNAS, you must install a VM.

http://doc.freenas.org/11/vms.html#docker-rancher-vm

 

[sretalla]

FreeNAS does not have the linux-c7 emulation, so it'll make it impossible to get working

Well, that isn't really true as it was done in FreeNAS 10 and I have also done it myself in the past in a Warden Jail (but quickly saw the limitations and gave up with it).

I agree it isn't really a good idea to persist with having docker directly run on FreeNAS/FreeBSD despite the positive benefit of having docker being able to directly access pool locations.

 

[NetSoerfer]

Docker on FreeNAS requires both Docker and Linux support. FreeNAS does not have the linux-c7 emulation, so it'll make it impossible to get working. In order to use Docker on FreeNAS, you must install a VM.

http://doc.freenas.org/11/vms.html#docker-rancher-vm

There's a docker port called Docker-Container - does that only work on FreeBSD but not on FreeNAS? Has linux-c7 emulation been specifically removed from FreeBSD for FreeNAS?

Well, that isn't really true as it was done in FreeNAS 10 and I have also done it myself in the past in a Warden Jail (but quickly saw the limitations and gave up with it).

I agree it isn't really a good idea to persist with having docker directly run on FreeNAS/FreeBSD despite the positive benefit of having docker being able to directly access pool locations.

I'm curious, what are those limitations? I'm not looking for a lot, I'm not planning to set up a complex microservice infrastructure. All I need is a host for a home use DMS that isn't available for FreeBSD, but for Docker.

I kind of dislike the need for a VM - not so much for its resource overhead, which shouldn't amount to much more than some RAM, but for the need to access storage through the network. Native ZFS for Docker seems promising, and I haven't heard about any specific limitations of Docker on FreeBSD that would prevent me from using it.

Could you elaborate on those, and why you ultimately gave up on it?

 

[sretalla]

Could you elaborate on those, and why you ultimately gave up on it?

I saw that docker was no longer developed and can't recall the specific bug I had hit, but with no hope of having it fixed, I understood that abandoning it was the right option.

I'm not sure that I fully understood the zfs jail intent and detailed operation... as far as I could tell at the time, it wanted a ZFS dataset to be mounted in the jail as a dataset (using zfs jail), which then could not be accessed outside the jail (for me, defeating the purpose of the whole thing)... perhaps I had got that part wrong, but still doesn't help with the bugs in the docker daemon.

NFS between the FreeNAS host and a guest VM is 10Gbit virtIO, so there should be no concerns over bottlenecks there (it never really hits the real-world network).

 

[Ericloewe]

Well, that isn't really true as it was done in FreeNAS 10

It was a Linux VM.

There are several important things to keep in mind here:

• Docker on FreeBSD would effectively be a jail manager. Running it within a jail is weird, if it even works.
• Every Docker image or whatever out there is Linux software. FreeBSD could have 100% compatibility with the management layer, but that still wouldn't help anyone since the software you want to run requires Linux.
• If FreeBSD ends up supporting all the Linux stuff, like Smart OS does in Illumos-land, it would be possible to run all that stuff on FreeBSD, but you'd still be tied to some of the Linux-isms (besides the system call layer, which isn't too hard to do)
• Without such support, you need a Linux VM or something else that is very good at pretending to be Linux (Hello Windows and Smart OS!)

 

[sretalla]

It was a Linux VM.

Thanks for the trademark paddling to remind me about that @Ericloewe.

I was getting it confused due to the CLI (wrapper) allowing me to type docker start or whatever in a shell which could also interact directly with the FreeNAS system (I guess it was the intention to make people not see the difference).

I now remember that there was that DockerVM setup behind it (which was also quite a problem as it kept losing its connection to the main system).

 

[Ericloewe]

Yeah, there was a fancy management layer across the VM boundary. Unfortunately, it's tied to the Corral middleware.