Docker on 11.0

Status
Not open for further replies.
X

xCatalystx

Contributor
Joined
Dec 3, 2014
Messages
117
Zwck said:
i installed the nfs package, basically with a nfs.yml before everything
here it is
Click to expand...

What I don't get is your using a container to mount /mnt/zpool/docker, but you also have separate mounts in cloud-config? I'm just confused why you have them in two separate places. I need to mount multiple NFS exports so I'm trying to find the best method. I was planning to use host mounts but I keep seeing people using these containers.
 
Zwck

Zwck

Patron
Joined
Oct 27, 2016
Messages
371
xCatalystx said:
What I don't get is your using a container to mount /mnt/zpool/docker, but you also have separate mounts in cloud-config? I'm just confused why you have them in two separate places. I need to mount multiple NFS exports so I'm trying to find the best method. I was planning to use host mounts but I keep seeing people using these containers.
Click to expand...

What happens is in the first part is you create physical folders in /mnt namely docker and dbs get the nfs-container to start which gives you the option to use sudo mount 192.168.0.2:/mnt/zpool/whatever /mnt/whatever the first part also mounts the first share i.e. from host at 192.168.0.2:/mnt/zpool/docker to rancher at /mnt/docker (you can only mount one share and you have to (otherwise the yml is not complete to be executed)

Code:
rancher:
services:
	nfs:
	 image: d3fk/nfs-client
	 labels:
		io.rancher.os.after: console, preload-user-images
		io.rancher.os.scope: system
	 net: host
	 privileged: true
	 restart: always
	 volumes:
		- /usr/bin/iptables:/sbin/iptables:ro
		- /mnt/docker:/mnt/docker:shared
		- /mnt/dbs:/mnt/dbs:shared
	 environment:
		SERVER: 192.168.0.2
		SHARE: /mnt/zpool/docker
		MOUNTPOINT: /mnt/docker



the second part just uses a short hand notation to attach more nfs mounts to this one container but you can feel free to add as many mounts as you want to this one container with more lines following the last mount line (but make sure you also create the folders).

Code:
#cloud-config
mounts:
- ["192.168.0.2:/mnt/zpool/dbs", "/mnt/dbs", "nfs", ""]


I use a yml file so it loads the nfs driver before anything else, which helps with the whole setup as all my persistent data is on my nfs shares.

How do you use host mounts?
sorry for the terrible comment, just woke up
 
X

xCatalystx

Contributor
Joined
Dec 3, 2014
Messages
117
Zwck said:
How do you use host mounts?
sorry for the terrible comment, just woke up
Click to expand...

All Good. The plan was to run docker on 2 virtual machines. NFS is located on FreeNAS and a virtual machine will also be hosted (via bhyve), a separate for other purposes vm on a esxi.

I was hoping to move most of my jails to docker containers on the VM located on FreeNAS but ofc i'll need access to data in separate pools/datasets (thus multiple mounts). hoping to move everything but Plex into containers. I actually build containers sometimes at work so I've got a good understanding of the ecosystem but mounting remote storage still eludes me.

I did get NFS to work on my esxi docker-host using host mounts via fstab, but when I tried coreos/rancheros I could never get the containers to work.

Zwck said:
get the nfs-container to start which gives you the option to use
Click to expand...
ahh, that makes sense. so the container is to provide access to nfs-client when not locally available on the host?

so if the host has access to nfs-client natively, wouldn't it just make more sense to use cloud-init / cloud-config files?
 
Zwck

Zwck

Patron
Joined
Oct 27, 2016
Messages
371
xCatalystx said:
so the container is to provide access to nfs-client when not locally available on the host?
Click to expand...

RancherOS does not have a nfs-client available when you first set it up. Since RancherOS has more or less everything containerized, the nfs client need to be loaded as a container so the OS can make use of it, i do this in the yml file (its like installing a package for the os)


PS: i am not an expert on RancherOS, this is just a first approximation for me after quickly skimming the extensive wiki


To my setup:
Terminology:
rancher -> GUI for setting up containers
RancherOS -> OS
rancher:agent -> container to give rancher administration rights on docker-machine.

RancherOS+rancher+rancher:agent -> bhyuveVM -> 192.168.0.16
UbuntuVM+rancher:agent ->192.168.0.17

The installed rancher:agent allows rancher to administer the given host. So i can administer as many docker machines as i want. So if you want to have certain machines that run at certain times you can do this from the rancher GUI, no problem.

What i am hoping for is that at somepoint (freenas 11.1 or 11.2) will allow setting up RancherOS through the freenas GUI so i can setup a couple of rancherOS-VMs via GUI and have one rancherOS just to administer the other VMs, basically a HA setup.


xCatalystx said:
hoping to move everything but Plex into containers
Click to expand...
This is exactly what i have done (except replace plex with emby)

upload_2017-8-29_10-44-38.png
upload_2017-8-29_10-45-29.png


These are just the containers for my RancherOS system, i have them scheduled to restart once a week, or depending on the container once a day. Some of them i start by hand some of them i have set to inactive because i wanted to try out alternatives, simple export of docker-compose and rancher-compose files makes it super easy to move from one host to another.

unrelated:
I recently moved away from HA-proxy on my freenas to use HA-proxy on my pfsense firewall the 200$ extra for the pfsense which has a 100% update time was worth every penny :D and seperates a littlebit the load.
 
X

xCatalystx

Contributor
Joined
Dec 3, 2014
Messages
117
Zwck said:
RancherOS does not have a nfs-client available when you first set it up.
Click to expand...
Yep, I think that was the biggest issue I had getting my head around. But that makes total sense now. I have been using CoreOS+Portainer (tectonic kubernetes at work) has nfs-client installed. But that would be an interesting way to mount cifs shares aswell.

Zwck said:
This is exactly what i have done
Click to expand...
do you mind providing you download stack rancher-compose file? wouldnt mind giving it a read. I dunno about attachment rules here, so PM would be fine.

Zwck said:
I recently moved away from HA-proxy on my freenas to use HA-proxy on my pfsense firewall
Click to expand...
I use Sophos UTM so I have access to their WAF (which does most things ha does + security) so I most likely won't use the inbuilt ha-proxy other than maybe making clean urls.

anyways thanks, Zwck., that answered my questions.
 
Zwck

Zwck

Patron
Joined
Oct 27, 2016
Messages
371
xCatalystx said:
Yep, I think that was the biggest issue I had getting my head around. But that makes total sense now. I have been using CoreOS+Portainer (tectonic kubernetes at work) has nfs-client installed. But that would be an interesting way to mount cifs shares aswell.


do you mind providing you download stack rancher-compose file? wouldnt mind giving it a read. I dunno about attachment rules here, so PM would be fine.


I use Sophos UTM so I have access to their WAF (which does most things ha does + security) so I most likely won't use the inbuilt ha-proxy other than maybe making clean urls.

anyways thanks, Zwck., that answered my questions.
Click to expand...

Here you go, there should not be any passwords or so included so its quite easy to share

Code:
version: '2'
services:
  radarr:
	image: linuxserver/radarr
	hostname: radarr
	environment:
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	volumes:
	- /mnt/mm/unsorted/complete:/downloads
	- /mnt/mm/movies:/movies
	- /mnt/docker/rancher/radarr:/config
	- /mnt/mm2/movies:/movies2
	tty: true
	ports:
	- 7878:7878/tcp
	labels:
	  io.rancher.container.pull_image: always
  sonarr-anime:
	image: linuxserver/sonarr
	hostname: sonarr-anime
	environment:
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	volumes:
	- /mnt/mm/unsorted/complete:/downloads
	- /mnt/mm/anime:/tv
	- /mnt/docker/rancher/sonarr-anime:/config
	tty: true
	ports:
	- 8787:8989/tcp
	labels:
	  io.rancher.container.pull_image: always
  jdownloader:
	image: dbarton/jdownloader
	hostname: jdownloader
	environment:
	  JDOWNLOADER_GID: '1000'
	  JDOWNLOADER_UID: '1000'
	stdin_open: true
	volumes:
	- /mnt/docker/rancher/jdownloader:/jdownloader/cfg
	- /mnt/mm/unsorted/complete:/downloads
	- /mnt/mm/unsorted/complete:/media
	tty: true
	labels:
	  io.rancher.container.pull_image: always
  sonarr:
	image: linuxserver/sonarr
	hostname: sonarr
	environment:
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	volumes:
	- /mnt/mm/unsorted/complete:/downloads
	- /mnt/mm/tvshows:/tv
	- /mnt/docker/rancher/sonarr:/config
	tty: true
	ports:
	- 8989:8989/tcp
	labels:
	  io.rancher.container.pull_image: always
  headphones:
	image: linuxserver/headphones
	hostname: headphones
	environment:
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	volumes:
	- /mnt/mm/unsorted/complete/music:/downloads
	- /mnt/mm/music/Eng:/music
	- /mnt/docker/rancher/headphones:/config
	tty: true
	ports:
	- 8386:8181/tcp
	labels:
	  io.rancher.container.pull_image: always
  organizr:
	image: lsiocommunity/organizr
	stdin_open: true
	volumes:
	- /mnt/docker/rancher/organizr:/config
	tty: true
	ports:
	- 1180:80/tcp
	labels:
	  io.rancher.container.pull_image: always
  jdownloader-2:
	image: jlesage/jdownloader-2
	environment:
	  USER_ID: '1000'
	  GROUP_ID: '1000'
	  TZ: Europe/Berlin
	  KEEP_APP_RUNNING: '1'
	  VNC_PASSWORD: 
	stdin_open: true
	volumes:
	- /mnt/docker/rancher/jdownloader-2:/config:rw
	- /mnt/mm/unsorted/complete:/media
	- /mnt/mm/unsorted/complete:/output
	tty: true
	ports:
	- 5800:5800/tcp
	- 5900:5900/tcp
	labels:
	  io.rancher.container.pull_image: always
  couchpotato:
	image: linuxserver/couchpotato
	hostname: couchpotato
	environment:
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	volumes:
	- /mnt/docker/rancher/couchpotato:/config
	- /mnt/mm/unsorted/complete/movies:/downloads
	- /mnt/mm/movies:/movies
	- /mnt/mm2/movies:/movies2
	tty: true
	ports:
	- 6080:5050/tcp
	labels:
	  io.rancher.container.pull_image: always
  sabnzbd:
	image: linuxserver/sabnzbd
	hostname: sabnzbd
	environment:
	  PGID: '1000'
	  PUID: '1000'
	stdin_open: true
	volumes:
	- /mnt/mm/unsorted/complete:/downloads
	- /mnt/docker/rancher/sabnzbd:/config
	- /mnt/mm/unsorted/incomplete:/incomplete-downloads
	tty: true
	ports:
	- 6040:8080/tcp
	- 6041:9090/tcp
	labels:
	  io.rancher.container.pull_image: always
  deluge:
	image: linuxserver/deluge
	hostname: deluge
	environment:
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	network_mode: host
	volumes:
	- /mnt/docker/rancher/deluge:/config
	- /mnt/mm/unsorted/complete:/downloads
	tty: true
	ports:
	- 8112:8112/tcp
	labels:
	  io.rancher.container.pull_image: always
  hydra:
	image: linuxserver/hydra
	hostname: hydra
	environment:
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	volumes:
	- /mnt/mm/unsorted/complete:/downloads
	- /mnt/docker/rancher/hydra:/config
	tty: true
	ports:
	- 6050:5075/tcp
	labels:
	  io.rancher.container.pull_image: always
  ombi:
	image: rogueosb/ombi
	hostname: ombi
	environment:
	  API: bott0r:secretkey
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	volumes:
	- /mnt/docker/rancher/ombi:/config
	tty: true
	ports:
	- 3579:3579/tcp
	labels:
	  io.rancher.container.pull_image: always
  sickrage:
	image: zwck/docker-sickrage
	hostname: sickrage
	environment:
	  PGID: '1000'
	  PUID: '1000'
	  TZ: Europe/Berlin
	stdin_open: true
	volumes:
	- /mnt/mm/unsorted/complete/tvshows:/downloads
	- /mnt/mm/unsorted/complete/anime:/dllanime
	- /mnt/mm/tvshows:/tv
	- /mnt/mm/anime:/anime
	- /mnt/docker/rancher/sickrage:/config
	tty: true
	ports:
	- 6060:8081/tcp
	- 6061:8082/tcp
  sftpupload:
	image: atmoz/sftp
	stdin_open: true
	volumes:
	- /mnt/mm/tvshows:/home/Upload/share
	tty: true
	ports:
	- 2223:22/tcp
	command:
	- user:pw:1000:1000
	labels:
	  io.rancher.container.pull_image: always
  youtubedll:
	image: kmb32123/youtube-dl-server
	stdin_open: true
	volumes:
	- /mnt/mm/music/Videos:/youtube-dl
	tty: true
	ports:
	- 1234:8080/tcp
	labels:
	  io.rancher.container.pull_image: always

and
rancher.config.yml
Code:
version: '2'
services:
  radarr:
	scale: 1
	start_on_create: true
  sonarr-anime:
	scale: 1
	start_on_create: true
  jdownloader:
	scale: 1
	start_on_create: true
  sonarr:
	scale: 1
	start_on_create: true
  headphones:
	scale: 1
	start_on_create: true
  organizr:
	scale: 1
	start_on_create: true
  jdownloader-2:
	scale: 1
	start_on_create: true
  couchpotato:
	scale: 1
	start_on_create: true
  sabnzbd:
	scale: 1
	start_on_create: true
  deluge:
	scale: 1
	start_on_create: true
  hydra:
	scale: 1
	start_on_create: true
  ombi:
	scale: 1
	start_on_create: true
  sickrage:
	scale: 1
	start_on_create: true
  sftpupload:
	scale: 1
	start_on_create: true
  youtubedll:
	scale: 1
	start_on_create: true
 
X

xCatalystx

Contributor
Joined
Dec 3, 2014
Messages
117
Zwck said:
Here you go, there should not be any passwords or so included so its quite easy to share
Click to expand...
Coolies. Glad to see I'm not the only sane person to use a separate sonarr for anime XD
 
Osiris

Osiris

Contributor
Joined
Aug 15, 2013
Messages
148
About docker / rancher in an ubuntu VM...
I just came back from it completely. Done.
After unwinding from iohyve and forcing myself to use the VM section in the FN gui, I'm stepping back from it completely, for now.
Using Rancher/docker on that ubuntu (17) vm and containering a downloadstack (sab, transmission, sonarr & couch) rendered the whole setup completely unstable.
Networking, even when using just VIO nics, is a complete bitch.
I'll be investigating rancheros via iohyve (as I should have done months ago), but ... not using the actual FN functionality, then.
I really wanted to use FN as a full hypervisor, as I was about to do if the Corral stream had continued.

I'm extremely disappointed in the the FreeNAS crew and got some questions for them.
Whatever happened to embedding docker in FN11?
When it comes to VMs, there's so much more to bhyve than what you're offering here. Is this it?
Where are we on this? I can't see to find any feedback here. The topic on Freenas-weekly has died. Why not make it a FN monthly and just post something. Anything.


PS: Now that I'm ranting: this forum & searchability barely fit into a single sentence.
Somebody should make a complete reorganization on the topics.
 
Last edited by a moderator:
Zwck

Zwck

Patron
Joined
Oct 27, 2016
Messages
371
Osiris said:
About docker / rancher in an ubuntu VM...
I just came back from it completely. Done.
After unwinding from iohyve and forcing myself to use the VM section in the FN gui, I'm stepping back from it completely, for now.
Using Rancher/docker on that ubuntu (17) vm and containering a downloadstack (sab, transmission, sonarr & couch) rendered the whole setup completely unstable.
Networking, even when using just VIO nics, is a complete bunny.
I'll be investigating rancheros via iohyve (as I should have done months ago), but ... not using the actual FN functionality, then.
I really wanted to use FN as a full hypervisor, as I was about to do if the Corral stream had continued.

I'm extremely disappointed in the the FreeNas crew and got some questions for them.
Whatever happened to embedding docker in FN11?
When it comes to VMs, there's so much more to bhyve than what you're offering here. Is this it?
Where are we on this? I can't see to find any feedback here. The topic on Freenas-weekly has died. Why not make it a FN monthly and just post something. Anything.


PS: Now that I'm ranting: this forum & searchability barely fit into a single sentence.
Somebody should make a complete reorganization on the topics.
Click to expand...

for a simple download stack have you tried the plugins, they work really well, and are easy to use?
 
A

amiskell

Patron
Joined
Jun 25, 2015
Messages
266
Zwck said:
for a simple download stack have you tried the plugins, they work really well, and are easy to use?
Click to expand...

Or just do it in a jail, that's what I did.

Plex, PlexPy, Emby, Sonarr, Radarr, Jackett all in a single jail and then I have transmission in a separate jail with a VPN connection/killswitch.
 
Zwck

Zwck

Patron
Joined
Oct 27, 2016
Messages
371
amiskell said:
It's been pushed back to FN11.2
Click to expand...
amiskell said:
Or just do it in a jail, that's what I did.

Plex, PlexPy, Emby, Sonarr, Radarr, Jackett all in a single jail and then I have transmission in a separate jail with a VPN connection/killswitch.
Click to expand...
I thought plugins were just predefined jails. No?
 
Ericloewe

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
Osiris

Osiris

Contributor
Joined
Aug 15, 2013
Messages
148
Zwck said:
for a simple download stack have you tried the plugins, they work really well, and are easy to use?
Click to expand...
Mate, I have been using the pluginjails for years, succesfully. Somewhat. Upgrading is often screwed up.
Also, I'm doing things out of the ordinary with multiple nasses. Mapping & sharing drives isn't all that 'native'. It's scripting & scripting.

I just wanted to check out docker, kubernetes, rancher & bhyve. The dynamic docker volumes approach - I thought - was the way to go. Turns out it's not mature at all. If you ask me, I'd not use it in a production environment for years to come.
 
Jammin

Jammin

Dabbler
Joined
Dec 28, 2016
Messages
42
Osiris said:
About docker / rancher in an ubuntu VM...
I just came back from it completely. Done.
Click to expand...

Yes I gave up too. I came to the conclusion that since docker support on FreeNAS would always require an additional virtualization layer, this would always create problems for mapping drives etc. Corral made great steps but because docker doesn't run on BSD, the containers still ran in a VM (with the overhead that entailed). The 9pfs file system was problematic and there was no talk of continuing down that path in 11.x so it would all have to be run through NFS.

So I jumped to unRaid. It's not free and arguably the file system is not as resilient as ZFS. But it's plenty resilient enough for my needs and a lot more flexible. And the kicker for me was the native docker support. It's been super easy to get it all working and because the drive mounts are native, apps even get file system notifications when a file changes. The library of available and constantly updated docker containers is huge. I also have a legacy need to run a Windows VM and this was never reliable under FreeNAS but has been rock solid under unRaid.

It suits *my* needs better. I'm not posting here to convert people, but simply to suggest that if docker is a large requirement of yours, unRaid is worth a look.
 
Last edited by a moderator:
J

Josh D

Dabbler
Joined
Sep 17, 2013
Messages
10
My issue with unRaid is bit rot. The developers there simply don't seem to care about it. Granted this is more a zfs filesystem issue but zfs doesn't work well in unRaid.

Looks like we're stuck waiting for 11.2
 
Ericloewe

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
Josh D said:
The developers there simply don't seem to care about it.
Click to expand...
They only care about playing fast and loose with customer data, while charging for that privilege.

Their layer is pure crap and it's built upon traditional, unreliable filesystems. It's not just garbage, it's garbage squared.
 
K

KrisBee

Wizard
Joined
Mar 20, 2017
Messages
1,288
Josh D said:
My issue with unRaid is bit rot. The developers there simply don't seem to care about it. Granted this is more a zfs filesystem issue but zfs doesn't work well in unRaid.

Looks like we're stuck waiting for 11.2
Click to expand...

zfs + docker = linux ? But the zfs storage driver for Docker needs an experienced hand:

https://docs.docker.com/engine/userguide/storagedriver/zfs-driver/#zfs-and-docker-performance

Do FreeBSD/NAS and docker really make much of a marriage? I wonder how workable it will be if and when it arrives in FreeNAS 11.x
 
Last edited by a moderator:
Zwck

Zwck

Patron
Joined
Oct 27, 2016
Messages
371
K

KrisBee

Wizard
Joined
Mar 20, 2017
Messages
1,288
Zwck said:
well i used the nfs drivers and it works perfectly fine, for my home applications
haha
Click to expand...

The comment about zfs storage drivers is if you intended to run docker with zfs on Linux. I assume you are talking about NFS within a docker VM running on FreeNAS.
 
Status
Not open for further replies.

Similar threads

victorhooi
  • Locked
How to update Plex plugin to latest version under FreeNAS 11.0?
Replies
3
Views
11K
SweetAndLow
SweetAndLow
K
  • Locked
Where is FreeNAS going with Docker in 2018?
2 3 4 5
Replies
99
Views
53K
brando56894
B
Hisma
  • Locked
Another 11.0 Docker Setup Tutorial - This one using Ubuntu Server VM
Replies
14
Views
9K
Zwck
Zwck
gravely
  • Locked
SOLVED 11.1 "Container Provider" Rancher VM vs 11.0 iohyve Rancher VM
Replies
5
Views
2K
KrisBee
K
R
How many Docker Host VMs do you create?
Replies
1
Views
3K
sretalla
sretalla
Top