Rancher/RancherOS docker engine HOWTO on FreeNAS 11.0-RC2

[aportelli]

First of all, hi all! This is my first message here.

Like many I have been rather attracted by the Docker possibilities of Corral and the cold shower has been hard. Moreover this is happening at the moment I have to setup home & professional servers and I wanted to use Docker for applications. So I insisted a bit and ended up with a setup based on Rancher & RancherOS which I now found very satisfying. Because it sounds like there is a lot of people around struggling with setting this thing up, I thought I should write something here to help. This is an example on how to setup a Plex server as a container in Rancher running under a RancherOS VM. I used this procedure successfully with two different machines on two very different networks.

The full guide can be found in this Resource's page, accessible via the tabs above and to the left of this post.

 

[Ericloewe]

I took the liberty of porting this to the Resources section, to improve visibility and functionality. Thanks for the documentation!

The Resource itself can be accessed using the tabs at the top of the thread.

 

[KrisBee]

@aportelli

Many thanks for this valuable resource. I wonder if you would consider re-ordering the section between

iohyve cpiso rancheros.iso

&

iohyve create ranch 4G

or at least clarify that your VM config actually uses three virtual disks, seen as devices /dev/sd{a,b,c} inside rancher. It's not clear to me why you need an additional drive on which to mount
/var/lib/docker, why not just make /dev/sda bigger than 4GB?

Personally, I would have thought it better to give rancher a static IP, perhaps you can create an addendum for that.

 

[aportelli]

Alright no worries I'll do that when I get a bit of fresh air from work.

 

[aportelli]

I took the liberty of porting this to the Resources section, to improve visibility and functionality. Thanks for the documentation!

The Resource itself can be accessed using the tabs at the top of the thread.

Thanks, I looked at doing that myself but cannot figure it out (probably I just can't).

 

[Ericloewe]

I think you need a few posts/likes before you're allowed to post there, but that's easily solved by me creating it and then reassigning it to you.

 

[aportelli]

I made some requested changes.

 

[Ericloewe]

For future reference, it's best to use the update functionality. There's a Post Resource Update button near the top, you can enter your changelogs there.

 

[aportelli]

Ah ok thanks I totally missed that.

 

[adrum]

In the resources guide, I think

Code:

iohyve set mac_tap0=<MAC>

should be

Code:

iohyve set ranch mac_tap0=<MAC>

 

[blacs30]

I had a problem with the rancher-nfs for a while.
It either created new folder with random suffixes and the volume name or it didn't want to start at all, throwing an 500 internal server error.

In the end the documentation from rancher helped me: http://rancher.com/docs/rancher/v1.6/en/rancher-services/storage-service/rancher-nfs/

DRIVER OPTIONS

• Host - (host): NFS host
• Export - (export): When the volume is configured with the host and export, no subfolder is created; the root export directory is mounted.
• Export Base - (exportBase): By default, the volume can be configured with the host and export base, which creates a uniquely named subfolder on the NFS server.
• Mount Options - (mntOptions): Comma delimited list of default mount options.

In particular the driver_opts, driver options.

I had to use something like this:

docker-compose.yml (excerpt of the volumes part)

Code:

version: '2'
volumes:
  movies:
	driver_opts:
	  host: 192.168.0.2
	  export: /mnt/data/media/movies
	driver: rancher-nfs
  plex-db:
	driver_opts:
	  host: 192.168.0.2
	  export: /mnt/data/media/plex-db
	driver: rancher-nfs

Looking later at the docker-composer.yml didn't show me this result though.

______

There is a known bug in Rancher which is really annoying https://github.com/rancher/rancher/issues/8433 and connected with the rancher-nfs container. That rancher-nfs container is not always started before the requiring containers start and then docker seems stuck. Seems not to work good with that rancher-nfs.
I first thought about switching to CoreOS but RancherOS is actually nice for easy home usage.

The workaround I use now is to mount my required NFS shared on the RancherOS with this ros config:

https://forums.rancher.com/t/mounting-multiple-disks/1039/2

cloud-config.yml

write_files:
- path: /opt/rancher/bin/start.sh
permissions: "0755"
owner: root
content: |
#!/bin/bash
mkdir -p /opt/seconddisk
mount -t xfs /dev/vdb1 /opt/seconddisk


This command will merge the config into the existing one:

Code:

ros config merge -i cloud-config.yml

This one will show the complete existing config:

Code:

ros config export

 

[tofagerl]

I'm a little confused about the NFS part myself. Right now I simply have all my data in a single ZFS volume, including all configuration for things like plex. What I would like to do is simply mount that ZFS volume in the Rancher host, and use those directories in each docker-compose file like I've been doing from my old Docker server. (Which was first a second server, and then part of FreeNAS Corral)
This rancher-nfs stuff confuses me to no end, and I have no idea what this guide actually does, since you don't explain anything - you just tell me what to type; but I don't intend on typing things I don't know what will do! Please explain...

 

[amiskell]

You can't simply mount the ZFS volume to the VM running docker.

You have to use some kind of a network filesystem to share the data with the docker host.

In Corral, you could use 9pfs to share via a socket type connection but this is no longer supported on FreeNAS 11. So you have to use NFS or SMB to share the data from FreeNAS's ZFS volumes to the docker host. NFS is the best choice because you are sharing from one Unix machine to another.

 

[tofagerl]

You can't simply mount the ZFS volume to the VM running docker.

You have to use some kind of a network filesystem to share the data with the docker host.

In Corral, you could use 9pfs to share via a socket type connection but this is no longer supported on FreeNAS 11. So you have to use NFS or SMB to share the data from FreeNAS's ZFS volumes to the docker host. NFS is the best choice because you are sharing from one Unix machine to another.

Sure. Granted. But I don't understand what the process described for configuring NFS is actually doing, and so I can't adapt it to work in my use case.

 

[Ericloewe]

Creating an NFS share and mounting it inside the VM.

 

[tofagerl]

Creating an NFS share and mounting it inside the VM.

Oh, for f...
I don't understand the SPECIFIC instructions! What does the rancher-nfs docker image do? How do I configure the volumes correctly?
I've tried, but I keep getting little triangles on the containers that use the volumes, with hard to understand messages like "unhealthy" and "re-evaluating state"...

 

[nitrobass24]

Even outside of freenas I have had nothing but issues with Rancher and their documentation. Things as simple as adding hosts don't work the way they are described, etc.

I have completely dumped it at this point and run an Ubuntu VM with Docker, Portainer and map and NFS share from FreeNAS into the VM. Works great!


Sent from my iPhone using Tapatalk

 

[tofagerl]

Even outside of freenas I have had nothing but issues with Rancher and their documentation. Things as simple as adding hosts don't work the way they are described, etc.

I have completely dumped it at this point and run an Ubuntu VM with Docker, Portainer and map and NFS share from FreeNAS into the VM. Works great!


Sent from my iPhone using Tapatalk

Ironically we're using it at work, and the devops guys rave about it... Is that ironic...?

 

[Hisma]

Even outside of freenas I have had nothing but issues with Rancher and their documentation. Things as simple as adding hosts don't work the way they are described, etc.

I have completely dumped it at this point and run an Ubuntu VM with Docker, Portainer and map and NFS share from FreeNAS into the VM. Works great!


Sent from my iPhone using Tapatalk

I did this too. spun up a ubuntu server VM, installed docker and portainer from the cli, and it's all working perfectly. It all auto-boots from the GUI too using UEFI bios (had to do that little hack to copy the uefi bios driver inside the VM, but there's a guide that explains that perfectly here in the forums).

It's sooo much easier dealing with widely supported *nix flavors since documentation is everywhere and of the highest quality, so even newbs like me can get everything working following all the available online guides. Nothing against the method described in this particular rancher guide, but it's not exactly easy to understand exactly what is happening in each step, and I find that no two systems are exactly alike, so once you get stuck following a guide like this you're s.o.l unless you're an expert.

Question - 11.1 is going to have freebsd native docker support, correct? Will that mean better performance and more seamless deployment, or will docker effectively be installed by some lightweight template VM like this rancherOS, just baked into the GUI? Just curious if using docker in VM is a viable long-term solution, or if once 11.1 comes along, we're better off migrating to the baked-in docker.

 

[amiskell]

I did this too. spun up a ubuntu server VM, installed docker and portainer from the cli, and it's all working perfectly. It all auto-boots from the GUI too using UEFI bios (had to do that little hack to copy the uefi bios driver inside the VM, but there's a guide that explains that perfectly here in the forums).

It's sooo much easier dealing with widely supported *nix flavors since documentation is everywhere and of the highest quality, so even newbs like me can get everything working following all the available online guides. Nothing against the method described in this particular rancher guide, but it's not exactly easy to understand exactly what is happening in each step, and I find that no two systems are exactly alike, so once you get stuck following a guide like this you're s.o.l unless you're an expert.

Question - 11.1 is going to have freebsd native docker support, correct? Will that mean better performance and more seamless deployment, or will docker effectively be installed by some lightweight template VM like this rancherOS, just baked into the GUI? Just curious if using docker in VM is a viable long-term solution, or if once 11.1 comes along, we're better off migrating to the baked-in docker.

I highly doubt it'll be a FreeBSD native docker. It'll be a Linux VM running docker with some abstraction to allow control via the FreeNAS UI.

If you did FreeBSD native docker, you'd still be locked out from using containers built for linux docker so you have to rebuild dockers for FreeBSD or try to rely on the FreeBSD Linux compatibility libraries which might not work 100%.