Where is FreeNAS going with Docker in 2018?

Status
Not open for further replies.

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
@diskdiddler

Scroll to the bottom of the side bar to access the latest FreeNAS guide. I’ve seen that unhelpful error message, but I did manage to create a “Docker VM” using the new UI in FreeNAS-11.2-MASTER-201805250605

Thanks, I'll try this, in the next couple of weeks. It definitely does not feel right to me in the currently nightly builds.

@diskdiddler

I’m not sure what makes you says this. As far as I can tell, behind the New UI nothing has fundamentally changed for “Virtual Machines”. Creating a “Docker VM” via the new UI still downloads an out of date 1.1.3 rancheros base which the user cannot upgrade. Hard coding the rancheros version seems to break one of the most basic rules of software development, as I understand it.
(See: /usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py )

So look I suppose I'm kicking a dead horse or stirring a bees nest here, but is there a reason why it's still pulling down an old ISO file?
How hard is it to fix this? Are there any requests / bugs logged about it?

(I think I best re-read this whole thread, properly)


EDIT: I've re-read the thread properly, people keep making a point of saying it's broken and won't be / can't or isn't fixed but not entirely clear why?

If bhyve is slowly improving and rancher has released newer and presumably better versions of the OS, why is it not pulling down that newer version?

If people here were content with docker support on FN10, which was also virtualised, why can't we attain at least that quality here?
 
Last edited:

silverback

Contributor
Joined
Jun 26, 2016
Messages
134
As has been stated earlier in the thread, unless Docker has a seamless storage mount in the UI, it is not really fully functional.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Did it have such functionality under FN10?
 

silverback

Contributor
Joined
Jun 26, 2016
Messages
134

KrisBee

Wizard
Joined
Mar 20, 2017
Messages
1,288
@diskdiddler

FN10 docker was based on a pre-built "boot2docker" image and FreeNAS maintained templates for adding docker containers. Access to pool data was via 9p filesystem as stated. The latter was buggy and experimental.

Look at the vm.py code I referenced that's used in FN11 and you'll see the version of the rancheros base is hard coded, as are the necessary config files. So FreeNAS have given themselves an additional maintenance job and are hardly going to issue a new FreeNAS version every time the rancheros base changes. (They can't even issue a new FreeNAS version to add the fix needed for bhyve to boot recent linux kernels as in Ubuntu 18.04). This is one obvious weakness of FreeNAS, the user can't simply upgrade the FreeNAS base as you would if using FreeBSD, for example.

It's a very poor implementation in this respect. They are hamstrung to some extent over the limited boot option that work with bhyve , i.e two UEFI variants and a grub boot loader which, AFAIK, does not recognise "syslinux" based ISOs.

About the only real bhyve improvement I've seen is ironing out the problems with connections using the web VNC device. For example, no one seems interested in making the UEFI-CSM option fully functional.

Cynics might say the current implementation was/is a stop gap in FN11 to satisfy any perceived demand for docker apps and keep FN10 users onboard.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,458
Yes, I clearly remember that it did via the 9p business.
Yes, and 9pfs was (and is) buggy af. So if that's your metric, you should probably look elsewhere for your Docker hosting needs.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
So docker under FN10 wasn't as good as people claim basically?
 

fracai

Guru
Joined
Aug 22, 2012
Messages
1,212
So docker under FN10 wasn't as good as people claim basically?
My understanding (caveat that I never used The Release That Must Not Be Named) is that while it was functional and some number of users never experienced any issues, the code was a mess and determined to be unmaintainable. Scraping it, and the year of expended development effort, was less expensive than committing to update and maintain what was there.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
So docker under FN10 wasn't as good as people claim basically?
It was not the magic bullet some claimed it to be and the filesystem corrupted data (so said iX, I didn't pay too much attention and didn't hear any complaints about that). So, it has the kinda-neat management interface, but it was tied to FreeNAS OS X edition Corral middleware, so that went down the drain, too. Not even the front-end could be salvaged because the framework was dead months before Corral was even released!
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Ok so we've established here, that maybe FN10 wasn't as good as people claim.


So here's the next question.

Has anyone here manually loaded a good and recent docker VM in to 11.1? Either a newer version of rancher or something else?

If so, was it perfectly fine once you did all the fiddling and work to get it going?

I'd like to establish if it's genuinely viable using a VM and if current bhyve is up to it.
 

NetSoerfer

Explorer
Joined
May 8, 2016
Messages
57
Ok so we've established here, that maybe FN10 wasn't as good as people claim.


So here's the next question.

Has anyone here manually loaded a good and recent docker VM in to 11.1? Either a newer version of rancher or something else?

If so, was it perfectly fine once you did all the fiddling and work to get it going?

I'd like to establish if it's genuinely viable using a VM and if current bhyve is up to it.
Bummer, I was hoping to find an answer to this question in here :)

Very informative and insightful thread though. Got my vote to be stickied.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Yeah, I'm trying to actually get to the bottom of all the questions, my point being, is the core OS actually capable of hosting docker reliably, if so, that means the devs need to simply do, whatever that end user did, or something similar.
 

dublea

Dabbler
Joined
May 27, 2015
Messages
33
Has anyone here manually loaded a good and recent docker VM in to 11.1? Either a newer version of rancher or something else?

If so, was it perfectly fine once you did all the fiddling and work to get it going?

I'd like to establish if it's genuinely viable using a VM and if current bhyve is up to it.

I've worked with the built in RancherOS, an up to date VM of RancherOS, and installing RancherOS on a Ubuntu VM. I've had a better experience with the Ubuntu VM but that may be due to how they do things with RancherOS and my inexperience with it. It's solid and I've not had any crashes running it in bhyve.

... is the core OS actually capable of hosting docker reliably ...

Unfortunately FreeBSD doesn't fully support Docker at this time and it's experimental.
 

NetSoerfer

Explorer
Joined
May 8, 2016
Messages
57
Has anyone here manually loaded a good and recent docker VM in to 11.1? Either a newer version of rancher or something else?

If so, was it perfectly fine once you did all the fiddling and work to get it going?

I'd like to establish if it's genuinely viable using a VM and if current bhyve is up to it.
I'm currently fiddling with a new docker-only setup (no rancher, kubernetes, etc.) on a minimal debian in a bhyve VM, using NFS shares for persisting data. I'm writing up a step-by-step as I go, which I will post here once I'm done.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
That is appreciated.

I'd like to hope, bhyve can run a VM well enough for docker to function well.
 

KrisBee

Wizard
Joined
Mar 20, 2017
Messages
1,288
That is appreciated.

I'd like to hope, bhyve can run a VM well enough for docker to function well.

Hasn't the last post from @dublea already indicated how? Debian + docker + docker-composer and/or portainer is another route and is probably what @NetSoerfer is referring to. But really there is noting new here and you are still left with managing multiple layers of software and using file sharing to give docker apps access to pool data.
 

NetSoerfer

Explorer
Joined
May 8, 2016
Messages
57
Hasn't the last post from @dublea already indicated how? Debian + docker + docker-composer and/or portainer is another route and is probably what @NetSoerfer is referring to. But really there is noting new here and you are still left with managing multiple layers of software and using file sharing to give docker apps access to pool data.
You're right, there's numerous examples of people running Docker successfully on FreeNAS.

But I think it's stating the obvious to say there's a lot of confusion about this topic and a lot of contradictory or outdated information. The discussion in this thread is a good indicator of that, and a very helpful resource at the same time because it answers a lot of questions. I've wasted quite some time myself chasing down outdated concepts like 9fs simply because I didn't realise they don't apply to FN11.

A lot of this confusion results from the many layers of software you mention, each of which requires its own decisions. UEFI or BIOS/CSM? Ubuntu, RancherOS, or some other Linux distro? NFS or SMB? Docker only, or Rancher, or Docker-Compose, or Portainer? I'm not saying it's bad to have all these options, on the contrary, otherwise I'd still be using my Synology. But they do make it hard to get started with any level of confidence.

And the problem is aggravated by the fact that a lot of these software layers still have teething issues. These will be resolved eventually, and many of these issues are not caused by FreeNAS, but at the moment they all add to the confusion: VMs getting stuck in the UEFI interactive shell, bhyve not booting Linux kernels 4.14+, RancherOS not getting any updates,...

All this is to say that FreeNAS Docker support looks frustratingly complicated at the moment. FreeNAS RancherOS VM may be a great choice for anyone that needs a full-scale swarm management solution, but I think it's overkill for many. It seems like a lot of users around here are looking for a way to run services that just aren't available on FreeBSD. Those users have two options, both of which are unpleasant: Either face the complexity and overhead of Rancher Server, which adds its own number of layers into the mix - so if something doesn't work, you're looking at another complex infrastructure you have to understand first before you can debug it. All on a RancherOS VM with an unknown OS update policy. Or go down the fully custom path, but that throws all those other decisions and problems in your face.

I think what's missing for those users is a simpler, leaner way of running Docker from the FreeNAS UI.
 

KrisBee

Wizard
Joined
Mar 20, 2017
Messages
1,288
@NetSoerfer Can't disagree with anything you've said except this:

.. and many of these issues are not caused by FreeNAS, but at the moment they all add to the confusion: VMs getting stuck in the UEFI interactive shell, bhyve not booting Linux kernels 4.14+, RancherOS not getting any updates,....

Actually I'd say all those issues are down to FreeNAS. The inescapable fact is docker is linux tech which means it must run in a VM which will access pool data via a file sharing protocol.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Hasn't the last post from @dublea already indicated how? Debian + docker + docker-composer and/or portainer is another route and is probably what @NetSoerfer is referring to. But really there is noting new here and you are still left with managing multiple layers of software and using file sharing to give docker apps access to pool data.


My point is, I want to identify that it's possible, how it's done and then relay this to the devs to put the framework in the base OS, clearly it's currently not ideal as it's been done.
 

KrisBee

Wizard
Joined
Mar 20, 2017
Messages
1,288
My point is, I want to identify that it's possible, how it's done and then relay this to the devs to put the framework in the base OS, clearly it's currently not ideal as it's been done.

Well of course it's possible, people have been doing this since iohyve appeared in FN9 and searching the forum will show you the different approaches. Good luck with the rest ...
 
Status
Not open for further replies.
Top