Mounting Ceph file share into TrueNAS

[RobbGG]

Hi all,

So first and foremost, I know this isn't how stuff is designed to work, I know I'm probably making anyone reading this shake their head.

My setup currently consists of 2xHP servers running Proxmox with a Ceph cluster to manage storage between the VMs and various containers. I want a nice and shiny "plug it in and work" NAS application to servce some of the Ceph cluster's capacity to my netowrk in such a way it's easy for my Macbook to backup to it with timemachine, windows computers can access the files on the shares, all the basic things that TrueNAS can do pretty well (I assume) but for whatever reason have been hard to sort out using more DIY solutions.

I have TrueNAS installed in a VM and have a pool setup on a single virtual drive hosted within my ceph cluster. It seems to be working ok, performance isn't great and it keeps complaining abouot lack of redundancy but that's no big deal right now. (Ceph handles redundancy for me and performance isn't immensely important). I don't have any spare hardware so I can't pass through a drive controller and virtualise TrueNAS that way (which I feel would be the best option if I had the hardware but I don't) so I would like to look at another solution:

Now, I know FreeBSD has a functional Ceph package which I can tie in with my cluster. I know TrueNAS has the main OS very locked down to prevent installation of extra packages ("It's an appliance!" I hear someone shouting from the back) so I'm in the process of setting up a jai into which I will install Ceph and tie it in with my cluster and mount the a CephFS share.

My question is, will there be a way I can take this directory with the CephFS mount from the jail and mount it as a pool in the main TrueNAS install? Or better yet, install the Ceph client directly into TrueNAS and cut out the jail completely.

If anyone has a suggestion of a better way to get some of my Ceph storage presented through TrueNAS (or something else that offers similar functionality) then please go ahead.

Thanks in advance, I will brace myself for the backlash from my question.

// RobbGG

 

[sretalla]

My question is, will there be a way I can take this directory with the CephFS mount from the jail and mount it as a pool in the main TrueNAS install? Or better yet, install the Ceph client directly into TrueNAS and cut out the jail completely.

With the right devfs ruleset for the jail, you can probably make something like what you're talking about happen.

Of course it's not recommended and may not produce the result you really want, but that's the path you should investigate.

 

[RobbGG]

With the right devfs ruleset for the jail, you can probably make something like what you're talking about happen.

Of course it's not recommended and may not produce the result you really want, but that's the path you should investigate.

Thanks, I'll investigate that route.

 

[RogerMurdock]

Thanks, I'll investigate that route.

Did you find a solution here? I‘m in the exact scenario you’ve described. Would like to add my ceph cluster to a TrueNAS storage pool.

 

[RobbGG]

Did you find a solution here? I‘m in the exact scenario you’ve described. Would like to add my ceph cluster to a TrueNAS storage pool.

No luck so far I'm afraid, I managed to get my current NAS solution working ok for what I need to gav up on getting TrueNAS to work for now.

As another possible way forward with this you might be able to expose some of your Ceph storage via iSCSI or NFS or something like that and mount that share into TrueNAS but that feels really inefficient.