[HowTo] FreeNAS 11.1 + RancherOS + Docker + RancherUI + PLEX

[KrisBee]

for some reason I need to redo sudo ros config merge -i mount.yaml
every reboot of VM.
And also I also need to restart PLEX container manually from Rancher web. Any Idea why?

What does sudo ros config export show?
Sounds like you have not set a restart policy for your plex container.

 

[IZON]

Code:

sudo ros config export
EXTRA_CMDLINE: /init
mounts:
- - //192.168.0.20/Media
  - /media/media
  - cifs
  - username=USERNAME,password=PASSWORD,iocharset=utf8,_netdev
rancher:
  environment:
	EXTRA_CMDLINE: /init
  network:
	interfaces:
	  eth0:
		address: 192.168.0.80
		dhcp: true
		gateway: 192.168.0.1
		mtu: 1500
  password: PASSWORD
  resize_device: /dev/sda
  services_include:
	bhyve-vm-tools: true
  state:
	autoformat:
	- /dev/sda
	dev: LABEL=RANCHER_STATE
	wait: true
ssh_authorized_keys: []

REMOVED USERNAME and PASSWORDS.

 

[KrisBee]

Set dhcp to false in your cloud-config.

 

[Ascotg]

I installed (or tried to) Couchpotato and Sickrage and came across the following two problems:

- I can't reach the web ui of either.
I've used host as a network configuration and enabled Rancher DNS service discovery. I managed to connect to it once, but it failed when I refreshed.

- The container can't acces the (sub)folders.
From the SSH (-l cu /dev/nmbm2B) on RancherOS I can browse through all my media folders and subfolders without problem, but my containers can't see anything.
On the Volume tab I typed: "/media/videos/TVShows:/media/videos"
The way I thought it was is that the first part is the path on the FreeNas server and the right the relative path in the container, right?

PUID and PGID are added as Environment Variables and set the same as my Rancher account on FreeNas (owner of the videos folder)

Any suggestions?

 

[Ascotg]

I installed (or tried to) Couchpotato and Sickrage and came across the following two problems:

- I can't reach the web ui of either.
I've used host as a network configuration and enabled Rancher DNS service discovery. I managed to connect to it once, but it failed when I refreshed.

- The container can't acces the (sub)folders.
From the SSH (-l cu /dev/nmbm2B) on RancherOS I can browse through all my media folders and subfolders without problem, but my containers can't see anything.
On the Volume tab I typed: "/media/videos/TVShows:/media/videos"
The way I thought it was is that the first part is the path on the FreeNas server and the right the relative path in the container, right?

PUID and PGID are added as Environment Variables and set the same as my Rancher account on FreeNas (owner of the videos folder)

Any suggestions?

Ok so I got the web ui stable when I configured it as 'managed' under network settings when creating the container.

For the second problem I found out that instead of /path/on/host being linked to /media in the container, the Volume receives the name /path/on/host. Any ideas?

 

[Gunsmithy]

Great guide! I'm having some problems though that seem to be related to Samba/SMB/CIFS.

Sometimes the shares don't mount to my RancherOS VM when started, which can cause the Plex container to then start up without the config and then creates a new one.
Also, Plex Media Server only runs for a day at most before crashing due to the database being locked.

I couldn't see from your pictures/mount examples how you mounted /config and /transcode to Rancher and Plex. My mounts.yaml looked like this:
mounts:
- - //172.16.0.20/Movies
- /media/movies
- cifs
- username=plex,password=password,iocharset=utf8,_netdev
- - //172.16.0.20/TV Shows
- /media/tv
- cifs
- username=plex,password=password,iocharset=utf8,_netdev
- - //172.16.0.20/PlexContainer
- /mnt/PlexContainer
- cifs
- username=plex,password=password,iocharset=utf8,_netdev

PlexContainer being a Share owned by the plex user with the /config and /transcode directories inside.
Here is the bottom of the latest log file from the Plex container:
1/3/2018 11:12:58 PMSqlite3: Sleeping for 200ms to retry busy DB.
1/3/2018 11:12:59 PMSqlite3: Sleeping for 200ms to retry busy DB.
1/3/2018 11:23:31 PMGot nothing for: Series
1/3/2018 11:28:24 PM__code__:703: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
1/3/2018 11:43:24 PMterminate called after throwing an instance of 'soci::soci_error'
1/3/2018 11:43:24 PM what(): Cannot begin transaction. database is locked
1/3/2018 11:43:24 PM****** PLEX MEDIA SERVER CRASHED, CRASH REPORT WRITTEN: /config/Library/Application Support/Plex Media Server/Crash Reports/1.11.0.4666-fc63598ba/PLEX MEDIA SERVER/2eaa558d-5ae1-3280-346c3ca7-5ca63c65.dmp
1/3/2018 11:43:24 PMAborted
1/3/2018 11:43:24 PMStarting Plex Media Server.
1/3/2018 11:43:42 PMCritical: libusb_init failed
1/3/2018 11:50:40 PMConnection to 96.126.109.143 closed by remote host.
1/4/2018 12:04:00 AMConnection to 45.79.129.106 closed by remote host.

I'm using Host networking for the record. By the way, is it possible to do bridged with a static IP on my network much like you could do with Corral? It's much nicer for monitoring network traffic and dictating QoS.

 

[vikozo6]

@Gunsmithy - I had the same wish to add a IP Address like in Corral, this worked well!

 

[Yusuf Limalia]

Great guide! I'm having some problems though that seem to be related to Samba/SMB/CIFS.

Sometimes the shares don't mount to my RancherOS VM when started, which can cause the Plex container to then start up without the config and then creates a new one.
Also, Plex Media Server only runs for a day at most before crashing due to the database being locked.

I couldn't see from your pictures/mount examples how you mounted /config and /transcode to Rancher and Plex. My mounts.yaml looked like this:
mounts:
- - //172.16.0.20/Movies
- /media/movies
- cifs
- username=plex,password=password,iocharset=utf8,_netdev
- - //172.16.0.20/TV Shows
- /media/tv
- cifs
- username=plex,password=password,iocharset=utf8,_netdev
- - //172.16.0.20/PlexContainer
- /mnt/PlexContainer
- cifs
- username=plex,password=password,iocharset=utf8,_netdev

PlexContainer being a Share owned by the plex user with the /config and /transcode directories inside.
Here is the bottom of the latest log file from the Plex container:
1/3/2018 11:12:58 PMSqlite3: Sleeping for 200ms to retry busy DB.
1/3/2018 11:12:59 PMSqlite3: Sleeping for 200ms to retry busy DB.
1/3/2018 11:23:31 PMGot nothing for: Series
1/3/2018 11:28:24 PM__code__:703: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
1/3/2018 11:43:24 PMterminate called after throwing an instance of 'soci::soci_error'
1/3/2018 11:43:24 PM what(): Cannot begin transaction. database is locked
1/3/2018 11:43:24 PM****** PLEX MEDIA SERVER CRASHED, CRASH REPORT WRITTEN: /config/Library/Application Support/Plex Media Server/Crash Reports/1.11.0.4666-fc63598ba/PLEX MEDIA SERVER/2eaa558d-5ae1-3280-346c3ca7-5ca63c65.dmp
1/3/2018 11:43:24 PMAborted
1/3/2018 11:43:24 PMStarting Plex Media Server.
1/3/2018 11:43:42 PMCritical: libusb_init failed
1/3/2018 11:50:40 PMConnection to 96.126.109.143 closed by remote host.
1/4/2018 12:04:00 AMConnection to 45.79.129.106 closed by remote host.

I'm using Host networking for the record. By the way, is it possible to do bridged with a static IP on my network much like you could do with Corral? It's much nicer for monitoring network traffic and dictating QoS.

Have a look at this thread

https://stackoverflow.com/questions/35742807/docker-1-10-containers-ip-in-lan

 

[Yusuf Limalia]

Great guide! I'm having some problems though that seem to be related to Samba/SMB/CIFS.

Sometimes the shares don't mount to my RancherOS VM when started, which can cause the Plex container to then start up without the config and then creates a new one.
Also, Plex Media Server only runs for a day at most before crashing due to the database being locked.

I couldn't see from your pictures/mount examples how you mounted /config and /transcode to Rancher and Plex. My mounts.yaml looked like this:
mounts:
- - //172.16.0.20/Movies
- /media/movies
- cifs
- username=plex,password=password,iocharset=utf8,_netdev
- - //172.16.0.20/TV Shows
- /media/tv
- cifs
- username=plex,password=password,iocharset=utf8,_netdev
- - //172.16.0.20/PlexContainer
- /mnt/PlexContainer
- cifs
- username=plex,password=password,iocharset=utf8,_netdev

PlexContainer being a Share owned by the plex user with the /config and /transcode directories inside.
Here is the bottom of the latest log file from the Plex container:
1/3/2018 11:12:58 PMSqlite3: Sleeping for 200ms to retry busy DB.
1/3/2018 11:12:59 PMSqlite3: Sleeping for 200ms to retry busy DB.
1/3/2018 11:23:31 PMGot nothing for: Series
1/3/2018 11:28:24 PM__code__:703: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
1/3/2018 11:43:24 PMterminate called after throwing an instance of 'soci::soci_error'
1/3/2018 11:43:24 PM what(): Cannot begin transaction. database is locked
1/3/2018 11:43:24 PM****** PLEX MEDIA SERVER CRASHED, CRASH REPORT WRITTEN: /config/Library/Application Support/Plex Media Server/Crash Reports/1.11.0.4666-fc63598ba/PLEX MEDIA SERVER/2eaa558d-5ae1-3280-346c3ca7-5ca63c65.dmp
1/3/2018 11:43:24 PMAborted
1/3/2018 11:43:24 PMStarting Plex Media Server.
1/3/2018 11:43:42 PMCritical: libusb_init failed
1/3/2018 11:50:40 PMConnection to 96.126.109.143 closed by remote host.
1/4/2018 12:04:00 AMConnection to 45.79.129.106 closed by remote host.

I'm using Host networking for the record. By the way, is it possible to do bridged with a static IP on my network much like you could do with Corral? It's much nicer for monitoring network traffic and dictating QoS.

Not sure why your shares are failing =/ but it's definitely causing a problem with the config.

Is the DB locked due to the share failing?

 

[Yusuf Limalia]

Ok so I got the web ui stable when I configured it as 'managed' under network settings when creating the container.

For the second problem I found out that instead of /path/on/host being linked to /media in the container, the Volume receives the name /path/on/host. Any ideas?

Could this be a permission issue? how are you connecting your shares to RancherOS? is it CIFS?

 

[Ascotg]

Ok so I got the web ui stable when I configured it as 'managed' under network settings when creating the container.

For the second problem I found out that instead of /path/on/host being linked to /media in the container, the Volume receives the name /path/on/host. Any ideas?

Ok so I seemed to have solved most of my problems. First of all, don't bother with 'clone' it literally has never worked for me. Delete and start over.

Secondly, adding folders works quite well now, just avoid special characters in the path.
/media/videos:/data eventually did the trick for me.

 

[Yusuf Limalia]

Ok so I seemed to have solved most of my problems. First of all, don't bother with 'clone' it literally has never worked for me. Delete and start over.

Secondly, adding folders works quite well now, just avoid special characters in the path.
/media/videos:/data eventually did the trick for me.

+1 on clone I’ve never got it working. Considering dropping Rancher/ server altogether and just keeping it as light as possible. I think docker-compose will be better to work with.

I might also try portainer which seems more fit for my requirements and looks more lightweight

 

[Gunsmithy]

Not sure why your shares are failing =/ but it's definitely causing a problem with the config.

Is the DB locked due to the share failing?

As an update to my issues, yes I think this may be the case. I got my daily security output update last night and it had like 30 entries for the SMB Daemon crashing like this:
> pid 53397 (smbd), uid 0: exited on signal 6 (core dumped)

Clone not working is kind of a downer for me too. Far from the Docker experience I need. Haven't used compose before either, but if curious what you end up going with! I can't imagine SMB mounts being particularly performant for say hosting game servers vs mounting datasets to the VM host.

 

[Yusuf Limalia]

As an update to my issues, yes I think this may be the case. I got my daily security output update last night and it had like 30 entries for the SMB Daemon crashing like this:
> pid 53397 (smbd), uid 0: exited on signal 6 (core dumped)

Clone not working is kind of a downer for me too. Far from the Docker experience I need. Haven't used compose before either, but if curious what you end up going with! I can't imagine SMB mounts being particularly performant for say hosting game servers vs mounting datasets to the VM host.

compose is awesome. You can bring up a stack of containers with just one compose file. It doesn't have a GUI that I know of, but the YAML is very easy to understand. An example compose file looks like this:

Code:

version: '3'
services:
  web:
   build: .
   ports:
	- "5000:5000"
   volumes:
	- .:/code
  redis:
   image: "redis:alpine"

You can see it's almost exactly the information we enter in rancher. getting compose installed on RancherOS is a little bit of a ball-ache, but doable

 

[-fun-]

Fine, but does "docker volume ls" at the rancher CL show all your volumes as local? Because I'm talking about using the rancher-nfs driver, eg.:

Code:

root@rancher:~# docker volume ls
DRIVER			  VOLUME NAME
[...]
rancher-nfs		 LMS-Music
rancher-nfs		 LMS-State
[...]

KrisBee, can you explain how you managed to get this rancher-nfs to run?

 

[KrisBee]

KrisBee, can you explain how you managed to get this rancher-nfs to run?

First challenge for me is to remember the work flow order to get this going, it's four weeks since I did this and have not made much use of RancherUI since then. I got as far as creating data volumes using the rancher nfs driver for use with a couple of containers, and the driver can make use of pre-existing zpool datasets or can create sub-folders under existing datasets shared via NFS depending how you use the RancherUI to add storage volumes to the working rancher environment.

I started here http://rancher.com/docs/rancher/v1.6/en/rancher-services/storage-service/rancher-nfs/.

IIRC the basic sequence was:

1. In FreeNAS set up NFS service with correct export params.
2. In FreeNAS setup required NFS shares taking care with dataset ownership.
3. Via RancherUI, launch Rancher NFS from the catalog (I used the default environment and global scope)
4. In RancherUI under Infrastructure > Storage add individual volumes required by the container(s) you wish to run.
5. Setup containers linking to the data volumes created in step.4 as necessary.

That's the outline as I remember it, but of course the devil's in the detail. The first thing is to appreciated that the Rancher docs say if you were using a linux NFS server the export must be with "no_root_squash" and so how is this to be interpreted for a NFS server running in FreeNAS? Also, the rancher-nfs driver defaults to a nfs4 client not nfs3. So how will an owner:group of "root:root" in Linux map to "root:wheel" in FreeNAS, etc. ?

To avoid id mapping problems between FreeNAS and Rancher I checked both the "Enable NFSv4" and "NFSv3 ownership model for NFSv4" for the NFS service in FreeNAS.

In the NFS first test share I created, I was going to let the rancher-nfs driver create sub-folders in the dataset. The dataset owner & group were set to root & wheel, I selected "all directories" under the NFS share to be mounted and set "Maproot User" to "root" and "Maproot Group" to "wheel".

In another case, where I wanted only to link a pre-exisitng zpool dataset to a docker container, the dataset was owned by normal non-root FreeNAS account. But the NFS share had the same settings as I previously used. So for example my NFS exports were:

Code:

root@freenasT:/ # cat /etc/exports
V4: / -sec=sys
/mnt/TestPool/media/music  -alldirs -maproot="root":"wheel" -network 192.168.0.0/24
/mnt/TestPool/VM/docker  -alldirs -maproot="root":"wheel" -network 192.168.0.0/24
root@freenasT:/ #

In this example, the "music" dataset pre-existed and was owned by a normal user, the "docker" dataset was owned by root and rancher was to create sub-folders in this share.

I'll have to see what notes, if any, I made about the actual settings I used when setting up the rancher-nfs driver and the individual data volumes. More to follow ....

 

[-fun-]

That's very useful and helped me get a first container to run using NFS shares. I also let the container create subfolders as described in your post.

I defined a user and group mapping as environment parameters. This is possible in the linuxserver/calibre-web container I used (https://github.com/linuxserver/docker-calibre-web). Still the calibre server failed with access problems to configuration files. However a solution in FreeNAS / Rancher independent from such a mapping in the container would be preferable. Is a Maproot or Mapall setting in the NFS share sufficient for this?

I'm also still working on the following question:

As long as I use one environment in Rancher I can define the NFS Stack with exactly one location (MOUNT_DIR). This forces me to use the common denominator for all possible NFS shares. In your example this would be /mnt/TestPool/.

Apparently it must possible to use and share datasets below this common level. In your example: /mnt/TestPool/media/music/. How exactly must this be defined in terms of a Storage object in Rancher? Probably with a Volume Name in the storage object like media/music. I failed to define and use a Volume Name with a slash so far however.

Distinct datasets are very important to be able to have data from different containers separated.

 

[KrisBee]

The Rancher UI does not re-display the settings as entered on the original screens once rancher-nfs and various data volumes have been set up. You have to infer these from looking at the objects in the API (view in API), or by using the relevant docker inspect command at the CLI in rancheros itself.

In my example, rancher-nfs was set up with these options:

Code:

"environment": {

"MOUNT_DIR": "/mnt/TestPool/VM/docker",
"MOUNT_OPTS": "proto=tcp,nfsvers=4",
"NFS_SERVER": "192.168.0.99",
"ON_REMOVE": "retain",
"RANCHER_DEBUG": "false"

I was using this docker as a test:


docker run -d --init \
-p 9000:9000 \
-p 9090:9090 \
-p 3483:3483 \
-p 3483:3483/udp \
-v /etc/localtime:/etc/localtime:ro \
-v <local-state-dir>:/srv/squeezebox \
-v <audio-dir>:/srv/music \
larsks/logitech-media-server


So needed to create just two data volumes under Infrastructure > Storage. Giving each a name and then adding the appropriate driver options as "key = value" pairs.

I wanted to keep the container state data under /"mnt//TestPool/VM/docker", but use the existing "/mnt/TestPool/media/music" dataset as the container's <audio-dir>.

So the data volume options used were:

Code:

"driverOpts": {

"exportBase": "/mnt/TestPool/VM/docker",
"host": "192.168.0.99",
"name": "LMS-State",
"onRemove": "retain",
"rancher": "true"
},

and

Code:

"driverOpts": {

"exportBase": "/mnt/TestPool/media/",
"host": "192.168.0.99",
"name": "music",
"onRemove": "retain",
"rancher": "true"
},

In the rancherUI you would enter enter three driver option pairs for each of these data volumes:

host = <FreeNAS host ip >
exportBase = <path taken from FreeNAS NFS export>
OnRemove = retain

When the docker container is created via the RancherUI , you just pick the data volumes you want to use. Once the container is running, you can check all the NFS mounts made by Rancher within the logs of the rancher/storage-nfs container etc.

If you're only going to use a few containers, I think this is all a sledge hammer to crack a nut, heavy on resources and may not perform well. Yusuf who started this thread has given up on using RancherUI, finding docker-compose and configuring share mounts within in the rancheros cloud-config to be more direct.

To answer your question about linuxserver.io containers designed to run with a given PUID and GUID. You cannot add users to the base rancheros, there are only two users root and rancher with UID/GUID 1100:1100. So you might need to create a rancher user on FreeNAS rather than use mapall settings.

 

[-fun-]

So I can define an exportBase path per storage item via driverOpts? This did not work for me. I can define this but when I start a container using a storage item defined like this I get:

Error (Error response from daemon: error while mounting volume '/var/lib/rancher/volumes/rancher-nfs/books': VolumeDriver.Mount: Failed mount -o ,nfsvers=4 172.27.2.16:/mnt/flospool/jaildata//books /var/lib/rancher/volumes/rancher-nfs/books)

In the nfs stack I defined:

MOUNT_DIR: /mnt/flospool/jaildata/

And my storage item ´´books`` has exportBase = /mnt/flospool/jaildata/calibre defined. (The complete path is /mnt/flospool/jaildata/calibre/books.)

Apparently when trying to mount the name of the storage item is simply appended to the MOUNT_DIR instead of to the exportBase from the storage item.

An Rancher indeed sucks. Unless I'm missing something important I cannot correct / change any configuration but have to delete and redefine everything.

 

[-fun-]

Rancher does use an export driverOpt if and only if a host is also specified: So a storage item ´´cb2``can be defined with driverOpts export=/mnt/flospool/jaildata/calibre/books and host=172.27.2.16. This can be mounted successfully!

But the container I'm trying to setup cannot write to this share. According to logs it successfully identifies the assigned access rights in terms of UID and GID and the read / write / execute bits. But I cannot find any working combination including a rancher user and group in FreeNAS with and without mapall definition in the share as well as with and without GID and UID mapping in the container. :-(