Resource icon

How-to: RancherOS on FreeNAS 11.0

Like many I have been rather attracted by the Docker possibilities of Corral and the cold shower has been hard. Moreover this is happening at the moment I have to setup home & professional servers and I wanted to use Docker for applications. So I insisted a bit and ended up with a setup based on Rancher & RancherOS which I now found very satisfying. Because it sounds like there is a lot of people around struggling with setting this thing up, I thought I should write something here to help. This is an example on how to setup a Plex server as a container in Rancher running under a RancherOS VM. I used this procedure successfully with two different machines on two very different networks.

1 - Setting up a RancherOS VM
RancherOS is a ultra-lightweight Linux OS specialised for Docker. RancherOS does not support UEFI boot so the VM can't be setup through the FreeNAS 11 GUI but it can be done using iohyve. Open a root shell on your FreeNAS server and start by setting up iohyve:

iohyve setup pool=vm kmod=1 net=igb0

where vm is the ZFS pool you want to use for iohyve and igb0 is the network interface to which the VMs will be connected. Now through FreeNAS GUI, add two tunables to rc.conf :

name: iohyve_enable -- value: YES
name: iohyve_flags -- value: kmod=1 net=<nic>

This will insure that iohyve starts correctly when you'll reboot your machine. Now download RancherOS image:

wget https://releases.rancher.com/os/latest/rancheros.iso

and copy it in iohyve internal folder:

iohyve cpiso rancheros.iso

Here I used RancherOS 1.0.1-LTS. Now we will create two virtual disks for the VM. Open the FreeNAS GUI and in storage create two zvols wherever you want, in my case that will be

tank/vdisks/ranch/docker: 20 G
tank/vdisks/ranch/plex-transcode: 20 G

The first disk will contain the /var/lib/docker of the system if something goes wrong and the second is for Plex transcoding. A bit of explanation here: the rest of the data from the FreeNAS box will be shared with containers through NFS which is more convenient because you can browse the files. But Plex will HATE doing some write-intensive transcoding operation through NFS (even locally) so for that a direct disk is better. Let's create the VM

iohyve create ranch 4G

4G is enough to install the system. Set the VM parameters:

iohyve set ranch cpu=8 ram=10G os=custom boot=1 loader=grub-bhyve \
pcidev:1=ahci-hd,/dev/zvol/tank/vdisks/ranch/docker \
pcidev:2=ahci-hd,/dev/zvol/tank/vdisks/ranch/plex-transcode


You might want to adapt the CPU & memory allocated depending on your hardware. Now go to the VM directory

cd /iohyve/ranch

Use whatever you want to create the three following files in this directory:

device.map:
Code:
(hd0) /dev/zvol/vm/iohyve/ranch/disk0
(cd0) /iohyve/ISO/rancheros.iso/rancheros.iso

grub.cd.cfg:
Code:
linux (cd0,msdos1)/boot/vmlinuz-4.9.24-rancher rancher.password=rancher
initrd (cd0,msdos1)/boot/initrd-v1.0.1
boot

grub.hd.cfg (sorry for the long line, I am not sure these files support \ line breaks):
Code:
linux (hd0,msdos1)/boot/vmlinuz-4.9.24-rancher printk.devkmsg=on rancher.state.dev=LABEL=RANCHER_STATE rancher.state.wait console=tty0
initrd (hd0,msdos1)/boot/initrd-v1.0.1
boot

The file device.map set the system disk and virtual CD, grub.cd.cfg is a Grub configuration to boot RancherOS from the CD and grub.hd.cfg to boot RancherOS from the hard drive with persistence.

Just a comment here, why three virtual disks (the RancherOS drive and the two zvols for docker & Plex)? Well you don't have to, you can just create the VM with a big ~50G drive. Bust experience told me that with this kind of story things can go real wrong real quick if you play too much. Then it is nice to have an easy way to recover. With this setup if you mess up the system you can still reinstall the system drive and then remount the docker drive and every container you had active before will be there. I just like to have the system drive isolated in case of. But again you don't have to, another strategy is to carefully snapshot clean configurations and roll back if something wrong happens.

2 - Installing RancherOS
Let's use the CD boot:

cp grub.cd.cfg grub.cfg

and start the VM with iohyve start ranch. Go into the VM serial console with iohyve console ranch and you should see the system booting. When the login screen appear, login with rancher both for the username and the password. Become root with sudo -i. Let's start by formatting the virtual drives. Do fdisk /dev/sdb, type n and Return to accept all the default options then w to create the partition an quit. Format the disk with mkfs.ext4 /dev/sdb1. Repeat the procedure for /dev/sdc.

Installing the system is a bit different if you have a DHCP or static network configuration so I will split the two cases.

2.1 - DHCP network configuration
If you have a DHCP then your VM should have picked up IP, DNS & gateway and you should be able to reach the net. Give Google a ping to check. If you want the MAC of the VM to be fixed for a static lease, do the following in FreeNAS:

iohyve set ranch mac_tap0=<MAC>

where <MAC> is whatever MAC address you want. Now create a cloud-config.yml containing:

Code:
#cloud-config
hostname: ranch.local
ssh_authorized_keys:
  - XXXXXXXXXXXXXXXX
mounts:
  - ["/dev/sdb1", "/var/lib/docker",	 "ext4", ""]
  - ["/dev/sdc1", "/mnt/plex-transcode", "ext4", ""]
rancher:
  network:
	interfaces:
	  eth0:
		dhcp: true
	dns:
	  nameservers:
	  - 8.8.8.8
	  - 8.8.4.4

Where the XXXXXX... should be replaced by the SSH public key of the machine you want to use to SSH into RancherOS. Here the DNS are the Google ones as an example, don't let Google know your traffic and change it if possible.

2.2 - Static IP network configuration
If you have no DHCP on the network then surely you can't reach the net, and you need to for the installation. Start by writing the following cloud-config.yml:

Code:
#cloud-config
hostname: cliath-ranch.ph.ed.ac.uk
ssh_authorized_keys:
  - XXXXXXXXXXXXXXXX
mounts:
  - ["/dev/sdb1", "/var/lib/docker",	 "ext4", ""]
  - ["/dev/sdc1", "/mnt/plex-transcode", "ext4", ""]
rancher:
  network:
	interfaces:
	  eth0:
		addresses:
		  - 192.168.0.5/24
		  - 192.168.0.6/24
		gateway: 192.168.0.1
		dhcp: false
	dns:
	  nameservers:
	  - 8.8.8.8
	  - 8.8.4.4

Where the XXXXXX... should be replaced by the SSH public key of the machine you want to use to SSH into RancherOS. The IPs are just an example, replace them with what you need. Similarly, the DNS are the Google ones as an example, don't let Google know your traffic and change it if possible. Now put the config in the system:

ros config merge -i cloud-config.yml

and restart the network service container:

system-docker restart network

Now you should be able to access Internet. Double check with ifconfig and ping some website.

2.3 - Installation
Now install the system with

ros install -d /dev/sda -c cloud-config.yml

When this is finished the installer will ask you if you want to reboot, say no. Go back to a FreeNAS shell and stop the VM with iohyve stop ranch. Now we will boot from our fresh install, change the Grub config with

cp grub.hd.cfg grub.cfg

and start the VM again iohyve start ranch. Wait a bit and figure out which IP the VM is connected on and SSH into RancherOS

ssh rancher@<IP>

If everything went well no password was asked to you are in.

3 - Setting up Rancher
Spawn a Rancher container in RancherOS with

sudo docker run -d --restart=unless-stopped --name='rancher-server' -p 8080:8080 rancher/server:stable

Wait a bit, you can do docker logs -f rancher-server to see Rancher starting. At some point you should be able to see the web interface at http://<RancherOS IP>:8080. Go to Infrastructure > Hosts and click Add Host. There will be a command on the page to copy & paste into the RancherOS shell, do it. After it started, you should see your machine in the Rancher hosts. Now secure the login to Rancher by going to Admin > Access Control and set things up the way you like.

We will now setup the NFS access. In FreeNAS create a dataset that will hosts the container volumes, for me it was tank/cloud-data, within this dataset create two other ones called plex-db and medias. Make some user called plex owner of these datasets. Start the NFS service with the 'Enable NFSv4' option enabled. Add two NFS shares for the two datasets created, map all the permissions to plex and only authorise the RancherOS IP to mount. Back to Rancher go to Catalog > Library and install Rancher NFS. Fill your FreeNAS IP and /mnt/tank/cloud-data as a mount point.

4 - Setting up Plex
In Rancher go to Stack > Add Stack, name the stack the way you like and provide the two following files:

docker-compose.yml
Code:
version: '2'
volumes:
  medias:
	external: true
	driver: rancher-nfs
  plex-db:
	external: true
	driver: rancher-nfs
services:
  plex-server:
	image: plexinc/pms-docker
	environment:
	  CHANGE_CONFIG_DIR_OWNERSHIP: 'false'
	stdin_open: true
	tty: true
	volumes_from:
	- plex-storage
	ports:
	- 32400:32400/tcp
	- 32413:32413/udp
	- 32414:32414/udp
	- 32469:32469/tcp
	- 3005:3005/tcp
	- 32410:32410/udp
	- 32412:32412/udp
	- 8324:8324/tcp
	- 1900:1900/udp
	labels:
	  io.rancher.container.pull_image: always
	  io.rancher.sidekicks: plex-storage
  plex-storage:
	image: busybox
	stdin_open: true
	volumes:
	- medias:/data
	- plex-db:/config
	- /mnt/plex-transcode/transcode:/transcode
	tty: true
	labels:
	  io.rancher.container.pull_image: always

rancher-compose.yml
Code:
version: '2'
services:
  plex-server:
	scale: 1
	start_on_create: true
  plex-storage:
	scale: 1
	start_on_create: true


And click Create. If everything goes well after a bit you should be able to reach Plex web interface at your RancherOS IP on port 32400. Rancher allows you through its web interface to directly see the logs of the containers which might help you know what's going on. With this setup the folder /mnt/tank/cloud-data/medias is supposed to be used to put movies, etc... for Plex. It will be seen inside of Plex as /data.

This is all! In principle if you reboot FreeNAS the VM and Rancher & Plex should restart automatically. I did not write the YML files for Plex, but I exported them from containers I created with Rancher awesome interface, I let you experiment and read the doc to figure that out.

EDIT 1: Added static IP configuration and commented on the virtual disks.
EDIT 2: Typos
  • Like
Reactions: blacs30
Author
aportelli
Views
3,603
First release
Last update
Rating
5.00 star(s) 2 ratings

Latest reviews

Really helpful. Needed a different setting for the rancher-nfs stuff which I commented in the discussion. But now I am happy and look forward doing more with it.
Great guide!
Top