The Layman's Guide to RancherOS or How I Learned to Stop Worrying and Switch to Docker

[n3IVI0]

First some caveats. This is not a definitive guide. I am well aware that there are multiple methods of running Docker on RancherOS. This is what worked for me. I have thoroughly tested my system, and I am confident I have worked out all the gotchas (with Linux/FreeBSD there are ALWAYS gotchas), and got a smooth running setup. This guide is written with the understanding that you're not a complete NOOB, but I have tried to make the language as noob-friendly and easy to follow as I can. My thanks to Keith Walker, who wrote the Rancher-NFS script that makes mounting the shares work, and produced these handy videos that helped me get started on this path. Those are in four parts, and can be found (here, here, here, and here).

So here goes.

How to Set Up the Ridiculously Complicated RancherOS and Docker on FreeNAS 11.2-U2:

1) Create the RancherOS VM:

Login to the slick new FreeNAS UI and scroll down to 'Virtual Machines'. Click on 'Actions', and 'Add Docker Container'. Give your container a name, all lowercase. Give your VM at least 2 CPUs and 4096 MiB of RAM. I run Emby on mine, so I gave mine more. Attach the virtual NIC to the real NIC on your router (mine is igb2). I also took the added step of using the virtual MAC address to setup a static IP on my router. You want your RancherOS to have the same IP every time. Give your RancherOS enough space. 20 GiB is probably adequate, but if you think you might need more, give it more. Choose where to keep the file this will create. If all looks good, click 'Submit' to create the VM.

2) Configure and load the VM:

Go to your new VM, scroll to the right, click on the three dots. Click 'Devices', click the three dots next to 'RAW', then change the password to whatever you can remember. 'Save' it. Click on the original three dots and click 'Start'. Then go back to the original three dots and click 'Serial'. This will bring up the CLI (Command Line Interface) for your RancherOS VM. Wait a bit while RancherOS boots up. Login with userid: rancher and password: whatever you set.

3) Networking:

Set the network configuration to a static IP using these commands, and the IP address you chose for your VM.

I set mine to 192.168.0.5. Yours needs to match your subnet (if your router address ends in .1.1, then all your addresses need to end in 1.some other number. Mine ends in 0.1, so all my addresses end in 0.some other number).

Change your nameservers to match whatever the DNS name servers are for your router. Only change the IP though - all the brackets and punctuation are necessary.

Code:

sudo ros config set rancher.network.interfaces.eth0.address 192.168.0.5/24

sudo ros config set rancher.network.interfaces.eth0.gateway 192.168.0.1

sudo ros config set rancher.network.interfaces.eth0.mtu 1500

sudo ros config set rancher.network.interfaces.eth0.dhcp false

sudo ros config set rancher.network.dns.nameservers "['103.86.96.100','103.86.99.100']"

type sudo reboot and wait for the VM to reboot.

4) Permissons:

As you know, the biggest beeyatch with Linux and getting the Docker containers to play well together (looking at you Sonarr and SABnzbd) is permissions. So, you will need to create a user/group that all shares and containers will use. Logout of the slick new FreeNAS UI and login with the Legacy Interface.

Go to Account > Groups. Click 'Add Group'. Choose a Group ID of 1020, and a Group Name of 'share'. Click OK. (you can set it to whatever you please. Just make sure both the GID and UID are the same, and use them for everything). Go to Account > Users. Click 'Add User'. Choose a User ID of 1020, and a User Name of 'share'. Uncheck 'Create a new primary group for the user.' Click the dropdown box and change Primary Group to 'share'. Click the 'Shell' dropdown and change it to 'bash'. Set Full Name to 'share user'. Set the email to share@freenas.local. Set a password you can remember. Click OK.

5) Storage:

You will need to add some storage where the Docker containers will keep their configuration files. This will be someplace outside RancherOS. This is a wise practice in case something terrible happens and you have to reinstall. In such an instance, your config files and databases will be safe, and you can rebuild reasonably quickly.

Again, using the FreeNAS Legacy Interface, click Storage along the top row.

Click the volume where you will be creating a dataset. Click the button at the bottom called 'Create Dataset'. Give the dataset a lowercase name. I chose 'docker'. Click OK (the defaults are fine). Then click the new dataset in the list of volumes on the left and click the button at the bottom called 'Edit Permissions'. Check the 'Apply Owner' buttons, and choose 'share' for the user and group. Check the box 'Set permission recursively' and click 'Change'.

Click 'Services' along the top. Go down to 'NFS' and click the wrench to change the settings. Check the 'Enable NFSv4' and 'NFSv3 ownership model for NFSv4' boxes. Click OK. Check the 'Start on boot box' next to NFS and then click 'Start Now'. If it was already running, Stop and restart to apply the changes.

Click 'Sharing' along the top. Click 'UNIX (NFS)'. Now you need to create some shares for your VM to use. Click 'Add Unix (NFS) Share'. Set the path to the Dataset you just made. Check 'All Directories', then set the 'Maproot User' to root and the 'Maproot Group' to wheel. This keeps the containers you create from freaking out. Permissions... You can create other shares in the same way to grant access to whatever data your containers need. If you want your NFS shares to be accessible to you in SMB, you can add shares to those directories by clicking 'Windows (SMB)', add the path to your new Dataset, UNCHECK 'Apply Default Permissions', then click OK.

6) Install Portainer and Create Mount Points for the Shares:

Logout and login with the slick new FreeNAS UI. Go back to your RancherOS Virtual Machine and your Serial CLI.

Type

adduser -u 1020 share

and hit Enter.

Enter the same password you created previously for user 'share'.

Type each line separate, and hit Enter to issue each command in turn:

Code:

su share

cd /mnt

mkdir nfs

If you have any other shares, make directories for them also. These will all be owned by share.

Code:

exit

sudo su -

Now we are going to install Rancher-NFS in order to mount those NFS shares.

Code:

cd /var/lib/rancher/conf/cloud-config.d

wget https://github.com/walkerk1980/docker-nfs-client/blob/master/rancheros-cloud-config.yml

When the file downloads, you will need to edit it.

vi rancheros-cloud-config.yml

Arrow key down to the environment section. Press 'i' to insert. Change the SERVER: to the IP address of your router. Change the SHARE: to the path for your NFS share you created. AND THIS IS SUPER IMPORTANT if you plan on using SABnzbd and Sonarr. Beneath SHARE:, add MOUNT_OPTIONS: rw,noatime,nolock,soft,_netdev,auto. Tap 'Esc', then type :wq and press Enter. This saves the changes and quits.

Here's what my modified rancheros-cloud-config.yml looks like:

#cloud-config

# /var/lib/rancher/conf/cloud-config.d/nfs.yml

# https://github.com/rancher/os/issues/641
write_files:
- path: /etc/rc.local
permissions: "0755"
content: |
#!/bin/bash
[ ! -e /usr/bin/docker ] && ln -s /usr/bin/docker.dist /usr/bin/docker

rancher:
services:
nfs:
image: walkerk1980/rancher-nfs-client
labels:
io.rancher.os.after: console, preload-user-images
io.rancher.os.scope: system
net: host
privileged: true
restart: always
volumes:
- /usr/bin/iptables:/sbin/iptables:ro
- /mnt/nfs:/mnt/nfs-1:shared
environment:
SERVER: 192.168.0.2
SHARE: /mnt/warehouse14/docker
MOUNT_OPTIONS: rw,noatime,nolock,soft,_netdev,auto

Now, type and hit Enter:

sudo reboot

Wait for RancherOS to reboot, then login again.

Lets make sure the mount was created:

Code:

cd /mnt

ls -l

If all is well, you should see the nfs directory you made, mounted with 1020:1020.

Open a terminal window (a separate program). I use a Mac, so mine is in Apps > Utilities > Terminal.

ssh root@192.168.0.2
type password for root and press Enter

Code:

cd /mnt/warehouse14/docker/nfs (use the path to the 'docker' share you created)

touch test.txt

Go back to the web browser and the RancherOS VM serial window.

Code:

cd /mnt/nfs

ls -l

Is the file test.txt in there?

If yes, then you have successfully mounted your first NFS share. Hooray!

rm test.txt

It should delete without error because you have r/w access.

Now to install Portainer.

docker run -d -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v /mnt/nfs/portainer:/data --restart always --name portainer portainer/portainer

Let it install.

sudo reboot

Once you are back up, open another tab in your browser, and browse to the address for Portainer, which is the <address of your VM>:9000

So for me:

http://192.168.0.5:9000

Add a password for the admin account, and login. On the far left hand side, click 'Local', then click 'Connect'. You will see your new local Portainer Endpoint. Click 'Endpoints' down under 'Settings'. Click 'local'. Set the Public IP to your VM IP.

Again for me:

192.168.0.5

Click 'Update endpoint'

This will fix it so that on the Containers screen, the one with the list of all your running Docker containers, you can click the port number for the Web GUI on each container, and it will automatically open a tab with the correct address.

You will likely want to create some more mount points for your shares. Now that Rancher-NFS is installed and running:

Code:

sudo su -

cd /mnt

adduser -u 1020 share

type the password again

su share

mkdir <whatever>, for each share you are adding

These are mine:

mkdir videos
mkdir music
mkdir books
mkdir sonarr
mkdir downloads

The 'sonarr' and 'downloads' directories are not shares, but they are directories within RancherOS used by various Docker containers.

You can make as many as you like. You have to first create a UNIX (NFS) share for each one as described previously back in Step 5. For each of these, you have to change the Volume path (the part after '-v') to match the directories you create. In each case, leave the part '/mnt/nfs-1:shared' untouched, or they won't mount.

'-e' sets Environment variables for the Docker container you are creating. Again, SERVER is the IP address of your ROUTER. SHARE is the path to the NFS share you want to mount within RancherOS. MOUNT_OPTIONS are necessary to grant r/w access to Docker.

Code:

docker run -itd --privileged=true --name videos --net=host --restart always -v /mnt/videos:/mnt/nfs-1:shared -e SERVER=192.168.0.2 -e SHARE=/mnt/warehouse14/shares/videos -e MOUNT_OPTIONS=rw,noatime,nolock,soft,_netdev,auto walkerk1980/rancher-nfs-client

docker run -itd --privileged=true --name music --net=host --restart always -v /mnt/music:/mnt/nfs-1:shared -e SERVER=192.168.0.2 -e SHARE=/mnt/warehouse14/shares/music -e MOUNT_OPTIONS=rw,noatime,nolock,soft,_netdev,auto walkerk1980/rancher-nfs-client

docker run -itd --privileged=true --name books --net=host --restart always -v /mnt/books:/mnt/nfs-1:shared -e SERVER=192.168.0.2 -e SHARE=/mnt/warehouse14/shares/books -e MOUNT_OPTIONS=rw,noatime,nolock,soft,_netdev,auto walkerk1980/rancher-nfs-client
sudo reboot

cd /mnt
ls -l

See if the shares are there.

cd into each share and make sure the files you are sharing are present.

'ls -l' to make sure they have the proper permissions (1020:1020)

7) Install Docker Containers

Now we are going to install a buttload of containers. You are by no means limited to these. These are the ones I use.

A Docker container is basically a recipe that instructs Docker what to install and where. You can find all the Docker containers you will ever want at https://hub.docker.com.

I will be installing each one using the RancherOS VM CLI. If you prefer, you can also login to Rancher directly using a terminal window and SSH. So for me, that would be:

ssh rancher@192.168.0.5
and my password

Whichever you use, always start with

sudo su -

Then you will be root and hopefully avoid any issues (permissions...).

Every Docker container has a name --> --name=whatever
Environment variables are given with --> -e whatever
Ports, needed to access each app separately on the same IP, are set with --> -p port:port
Volumes are linked between the shared storage and the container using --> -v path to the share:path within the container
You want your Docker container to restart automatically whenever you boot up, or restart RancherOS, so --> --restart always
And lastly, the address of the Docker image on Docker hub --> somegroup/appname

Obviously, change the paths on my recipes to match what you have setup on your rig.
Change your timezone, ports, volumes to match your setup.

I include my recipes, then, for reference.

Github:

Should you need git for anything, you can install it like so:

Code:

alias git="docker run -ti --rm -v $(pwd):/git bwits/docker-git-alpine"

git --help

Watchtower (https://hub.docker.com/r/v2tec/watchtower):

Watchtower monitors your containers and automatically updates them without you having to do anything. Definitely a must install.

docker run -d --name watchtower -v /var/run/docker.sock:/var/run/docker.sock v2tec/watchtower --cleanup
'--cleanup' deletes old images as they are replaced so orphaned images don't pile up on your server.

Sabnzbd (https://hub.docker.com/r/linuxserver/sabnzbd):

Code:

docker create \
  --name=sabnzbd \
  -e PUID=1020 \
  -e PGID=1020 \
  -e TZ=Africa/Kampala \
  -p 8082:8080 \
  -p 9090:9090 \
  -v /mnt/nfs/sabnzbd:/config \
  -v /mnt/nfs/downloads:/downloads \
  -v /mnt/nfs/downloads/incomplete:/incomplete-downloads \
  --restart always \
  linuxserver/sabnzbd

Normally, SAB uses port 8080 for its Web GUI. This conflicts with Unifi. So in my case, I have it mapped to port 8082. If you have no such conflict, then leave it on '-p 8080:8080'.

Sonarr (https://hub.docker.com/r/linuxserver/sonarr):

Code:

docker create \
  --name=sonarr \
  -e PUID=1020 \
  -e PGID=1020 \
  -e TZ=Africa/Kampala \
  -p 8989:8989 \
  -v /mnt/sonarr:/config \
  -v /mnt/videos:/tv \
  -v /mnt/nfs/downloads:/downloads \
  --restart always \
  linuxserver/sonarr

Sonarr cannot have its config files on an NFS share - it locks the database, which slows its function to a crawl. That's why I created a directory for it further back. You have to keep its config files within RancherOS or it slows to glacier speed.

Transmission (https://hub.docker.com/r/linuxserver/transmission):

Code:

docker create \
  --name=transmission \
  -e PUID=1020 \
  -e PGID=1020 \
  -e TZ=Africa/Kampala \
  -p 9091:9091 \
  -p 51413:51413 \
  -p 51413:51413/udp \
  -v /mnt/nfs/transmission:/config \
  -v /mnt/nfs/downloads:/downloads \
  -v /mnt/nfs/downloads/watch:/watch \
  --restart always \
  linuxserver/transmission

Subsonic (https://hub.docker.com/r/freenas/subsonic):

Code:

docker run -d --net=bridge
  -v /mnt/nfs/subsonic:/subsonic \
  -v /mnt/music:/music \
  -v /mnt/music:/podcasts \
  -p 4040:4040 \
  --restart always \
  -e TZ=Africa/Kampala \
  -e APP_USER=share \
  -e APP_UID=1020 \
  -e APP_GID=1020 \
  -e HTTP_PORT=4040 \
  --name=subsonic \
  freenas/subsonic

Emby (https://hub.docker.com/r/emby/embyserver):

docker run -d --name=emby -v /mnt/nfs/emby:/config -v /mnt/videos:/mnt/share1 -p 8096:8096 -p 8920:8920 -e UID=1020 -e GID=1020 emby/embyserver

Calibre Web (https://hub.docker.com/r/technosoft2000/calibre-web):

Code:

docker create --name=calibre-web --restart=always \
-v /mnt/books:/books \
-v /mnt/nfs/calibre:/calibre-web/config \
-e USE_CONFIG_DIR=true \
-e SET_CONTAINER_TIMEZONE=true \
-e CONTAINER_TIMEZONE=Africa/Kampala \
-e PGID=1020 -e PUID=1020 \
-p 8083:8083 \
technosoft2000/calibre-web

Unifi (https://hub.docker.com/r/goofball222/unifi):

Code:

docker run --name unifi -d --net=host \
  -v /mnt/nfs/unifi:/usr/lib/unifi/data \
  -v /mnt/nfs/unifi/cert:/usr/lib/unifi/cert \
  -v /mnt/nfs/unifi/logs:/usr/lib/unifi/logs \
  --restart always \
  goofball222/unifi

Unifi has to be run with the switch '--net=host'. Otherwise, it cannot find your APs to adopt and provision them.

JDownloader (https://hub.docker.com/r/jlesage/jdownloader-2):

Code:

docker run -d \
    --name=jdownloader-2 \
    -p 5800:5800 \
    -p 3129:3129 \
    -e USER_ID=1020 \
    -e GROUP_ID=1020 \
    -e TZ=Africa/Kampala \
    -e CLEAN_TMP_DIR=1 \
    -v /mnt/nfs/jdownloader:/config:rw \
    -v /mnt/nfs/downloads:/output:rw \
    --restart always \
    jlesage/jdownloader-2

This app runs best with the JDownloader extension for Chrome/Firefox. You will need to create an account at MyJDownloader.

Open the JDownloader UI.
Go to Settings->Advanced Settings.
Search for MyJDownloaderSettings.
Set Custom Device IPs to the IP address of the host running the container, between double quotes (e.g. "192.168.0.5").
Change the Direct Connection Mode to Allow lan/wan connections with manual port forwarding.
Restart JDownloader (File->Restart).

Now, you can right click on links to download them and it will send them directly to JDownloader on your server, and you can monitor the progress with the extension.

8) FOR THE LOVE OF GOD, HOW DO I GET SONARR AND SABNZBD TO WORK RIGHT!!?

Because that is always the question, isn't it? The reason I walked you through creating shares, creating users and groups, and assigning the shares to them, is because of permissions. It always comes back to permissions, with Linux/FreeBSD. I trust by now you are familiar with setting up Sonarr and SABnzbd. There are plenty of guides on how to do this. This guide presupposes you know about Usenet, have access to it, know about NZB indexers, have access to them, and know how to setup SABnzbd and Sonarr to use them.

However, I will review the key part that makes the two apps talk to each other.

Open the Web GUIs for each app by browsing to its address <YOUR VM IP>:<HTTP PORT YOU ASSIGNED>

So, for me, that would be http://192.168.0.5:8082 and http://192.168.0.5:8989.

Now, in SAB, in the upper right hand corner, click the cog labeled 'SABnzbd Config'. Along the top, click 'Folders'. Set 'Temporary Download Folder' to '/downloads/incomplete'. Set 'Completed Download Folder' to '/downloads/complete'. Click 'Save'.

Next, along the top click 'Categories'. In the open box below 'Default' type 'sonarr'. Set the 'Folder/Path' to '/downloads/complete/sonarr'. Click 'Add'. Then click 'Save'.

Click 'General' along the top. Scroll down to 'API Key' and copy this somewhere, like to a Notepad/TextEdit window. You'll need this for the next part.

You're done with SABnzbd.

Switch over to Sonarr. Along the top click 'Settings'. Click 'Download Client'. Under 'Download Clients' click the +. In the box that pops up, under 'Usenet' click 'SABnzbd'. Set the 'Name' to 'SABnzbd'. Set the 'Host' to the IP of your VM (in my case 192.168.0.5). Set the 'Port' to 8080, unless you needed to change it like I did. Paste the API Key you copied earlier into the box labeled 'API Key'. Set the 'Category' to 'sonarr'. Click 'Test'. If all goes well, you should get a little popup indicating the test succeeded. Then finally click 'Save'.

Make sure that all the boxes under 'Completed Download Handling' and 'Failed Download Handling' are set to Yes.

Along the top click 'Media Management'. MAKE SURE 'Rename Episodes' is set to 'Yes'. Otherwise, Sonarr will move the finished downloads, but not rename them. Obviously, you need to have chosen some kind of organizational naming scheme for your media library. You can specify those below.

If all goes well, when you initiate downloads with Sonarr, it will hand those off to SAB, then grab them up when finished, and move and rename them to their correct directories on the /videos share you created.

This is where the MOUNT_OPTIONS come in. Without those, you will get repeated messages in the Sonarr log about how the 'Import failed, path does not exist or is not accessible by Sonarr'. I hope this guide will save you the frustration I suffered getting these blasted things to work together properly.

It is exceedingly tricky to set this stuff up, especially if you don't have much prior experience with FreeBSD, vi, Docker, etc. The biggest difficulty is that on any given question, there are typically 50 different solutions, none of which work. I hope this guide will serve to streamline the process for you, and get you up and running with RancherOS and Docker with minimal aggravation.

Now go forth and conquer!

 

[benjaminjsanders]

A couple of notes and updates:

1) Create the RancherOS VM:
There has been a small UI change. In the latest version of FreeNAS you click the 'ADD' button, then select Docker Host. If I remember correctly you need to click the 'Advanced Options' to see all the things you need to update.

6) Install Portainer and Create Mount Points for the Shares:
When I executed the command

Code:

wget https://github.com/walkerk1980/docker-nfs-client/blob/master/rancheros-cloud-config.yml

it didn't pull the text file I needed, it pulled the html for the webpage on GitHub. I'm sure there is a way to get the text file itself, but I don't know it so I typed out the entire config file by hand. Be sure to look at the example master/rancheros-cloud-config.yml that n3IVI0 showed and make sure your file looks like that. If your file has lots of things that look like <DIV>, <SPAN>, <TABLE>, etc, you have the wrong file.

This is where I am stuck. n3IVI0 says to ssh into something, but doesn't specify if that is the FreeNAS instance or the Rancher instance. I went into the FreeNAS console and added a file to the folder I had set up the NFS share on, then went into the serial console for Rancher and checked the /mnt/nfs/ folder, but I didn't see the file there. I presume I have not setup my NFS share correctly and am working on a solution. If I find one, I'll post it.

 

[KrisBee]

@benjaminjsanders You just need to get the raw file: https://raw.githubusercontent.com/walkerk1980/docker-nfs-client/master/rancheros-cloud-config.yml

 

[Zwck]

Can some of the docker pros elucidate me what the difference is between employing a host nfs mount with a container and a direct volume mount with something for example

Code:

version: '3'
volumes:
  some_volume:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.123,nfsvers=4.1,soft,timeo=180,rsize=1048576,wsize=1048576"
      device: ":/mnt/something"
  jsome_other_volume:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.123,nas.zwck.lan,nfsvers=4.1,soft,timeo=180,rsize=1048576,wsize=1048576"
      device: ":/mnt/somnethingelse"

 

[Myriad]

I keep getting stuck on step 6. Every time I try to create the nfs share as 'share' I get a permission denied error. Why?

 

[lukyjay]

I keep getting stuck on step 6. Every time I try to create the nfs share as 'share' I get a permission denied error. Why?

Hi

Try:
sudo su
mkdir nfs
chown share:share nfs
exit

:)

 

[Ahmed Badr]

Thanks for the great guide ...

It seems that the mount location (ENV MOUNTPOINT) is hard coded in the DockerFile. I pulled the git repo into rancher but I can't locate it to edit it. Where is it located? or is there another way to change the mounting location and direcotry name?

Thank you.