Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

TrueNAS SCALE Project Start

abcslayer

Member
Joined
Dec 9, 2014
Messages
42
The reason for going gluster seems obvious: IX wants to focus on ZFS as underlaying filesystem, something which isn't possible to do (production ready) with CEPH.

But it is interesting, I'm very interested myself in a performance comparison between ceph and Gluster on ZFS, with the new Special Allocation Classes.
I don't know when you grow up to racks of storage servers why you need another storage management layer below (i.e ZFS), even if it is an incredible one. So even Gluster over something seems quite weird. Reducing the middle man is unavoidable.
 

ornias

Neophyte Sage
Joined
Mar 6, 2020
Messages
1,016
why you need another storage management layer below (i.e ZFS), even if it is an incredible one.
Lets take CEPH as an example:
ZFS caching and (special allocation classes) storage tiering is superior (in fact tiering is even deprecated) to the ones in CEPH.

So even Gluster over something seems quite weird. Reducing the middle man is unavoidable.
Gluster by its very design goes on top of another filesystem ;)
So it always has a middleman.
 

Kris Moore

VP of Engineering
Administrator
Moderator
iXsystems
Joined
Nov 12, 2015
Messages
616
Lets take CEPH as an example:
ZFS caching and (special allocation classes) storage tiering is superior (in fact tiering is even deprecated) to the ones in CEPH.


Gluster by its very design goes on top of another filesystem ;)
So it always has a middleman.

That is correct WRT Gluster. It is not a "bare-metal" filesystem, you're always running it on top of something else. We think ZFS is a far more suitable FS than say EXT or LVM in this case.
 

ornias

Neophyte Sage
Joined
Mar 6, 2020
Messages
1,016
We think ZFS is a far more suitable FS than say EXT or LVM in this case.
I agree:
Things like caching, tiering, checksumming, healing, compression are (in general) more fleshed out in ZFS...
Not as simple,sure, but more flushed out none the less.

We as users must not forget one of the core issues with scale-out solutions in enterprise: They require it to be tweakable to the insane degree. Adding the (also insane) tweakability of ZFS into this mix, seems like a good fit for enterprise customers. Maybe not home users or SMB, but thats also not the core audience of scale-out solutions.
 

Lennie

Newbie
Joined
Jun 9, 2020
Messages
2
I agree:
Things like caching, tiering, checksumming, healing, compression are (in general) more fleshed out in ZFS...
Not as simple,sure, but more flushed out none the less.

We as users must not forget one of the core issues with scale-out solutions in enterprise: They require it to be tweakable to the insane degree. Adding the (also insane) tweakability of ZFS into this mix, seems like a good fit for enterprise customers. Maybe not home users or SMB, but thats also not the core audience of scale-out solutions.
The 'hobby space' is also changing. I'm now seeing people run Kubernetes on Raspberry Pi 4s (and they have a 8GB version now), but those don't have a lot of storage options.
But I do run NanoPi M4V2 with a real PCIe-connected SATA HAT with ZFS and it works. So who knows what the future may bring. :)
 

brando56894

Dedicated Sage
Joined
Feb 15, 2014
Messages
1,506
I'm definitely anxiously awaiting nightly builds of this since I've been following FreeNAS for years, and ran it at times, but since I used it for everything FreeBSD never fit my needs, so I always went back to Linux, but stuck with ZFS.

I'm currently running ZFS on Arch Linux, and I'd love to be able to run the GUI on Arch, but I'm not holding my breath. I can always run Arch in VMs.
 

ornias

Neophyte Sage
Joined
Mar 6, 2020
Messages
1,016
I'm definitely anxiously awaiting nightly builds of this since I've been following FreeNAS for years, and ran it at times, but since I used it for everything FreeBSD never fit my needs, so I always went back to Linux, but stuck with ZFS.
There basically already are nightly builds and it's possible to update to the most recent nightly build. They just locked the nightly build iso files, but the nightly build update files are actually available once you grab the iso :)
 

volothamp

Member
Joined
Jul 28, 2019
Messages
46
Does this also mean we could get a build targeting ARM? It could be really interesting to have an ARM powered NAS
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,943
The OS was never really the limitation when it comes to ARM. Things like the lack of useful hardware and the crazy wild west of ARM platforms were.
 

Bostjan

Member
Joined
Mar 24, 2014
Messages
111
People will (want to) do this.
Is possible to run in one box TrueNAS scale and TrueNAS core/enterprise. Which one can be put into another?
 

ornias

Neophyte Sage
Joined
Mar 6, 2020
Messages
1,016
Does this also mean we could get a build targeting ARM? It could be really interesting to have an ARM powered NAS
I expect the bigest reason to be support, Free NAS-users basically get a freeride with the paying Enterprise NAS users. Those simply don't use ARM yet, because IX doesn't sell it and why would they, it isn't a clear improvement.

Simply put: For a bussiness it is not a smart move to support 20 platforms.

Just in case you are going to mention "but they don't have to give support":
Sorry to burst your bubble, but whatever you release even if you add "NOT SUPPORTED AT ALL" in caps, people are going to file bug and support requests either way.


People will (want to) do this.
Is possible to run in one box TrueNAS scale and TrueNAS core/enterprise. Which one can be put into another?
Why would you? TrueNAS SCALE would already support most of the features TrueNAS CORE/Enterprise does... In theory you could ofc. Nothing stopping you to make completely retarded virtualisation stacks....
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
477
@Kris Moore and @morganL
It would be very nice if the https://download.freenas.org/scale page would magically happen to get enabled/unlocked without any mention or support for it ;)
Yes, these things will happen, but when we are ready to support the community. At this stage, we have just indicated where we are heading.....not that we've reached any specific destination. Its not our nature to keep things secret. Thanks for all your insightful posts... if we had a mind reader award, you'd get one.
 

volothamp

Member
Joined
Jul 28, 2019
Messages
46
I expect the bigest reason to be support, Free NAS-users basically get a freeride with the paying Enterprise NAS users. Those simply don't use ARM yet, because IX doesn't sell it and why would they, it isn't a clear improvement.
What was the quote? "People who are serious software should make their own hardware"

Joking aside, thank you for the reply. I'd love to see an official HW ARM based from IXSystems, but I understand it's a small company compared to Apple and others and probably this is not a smart business model for them.

Cheers!
 

ornias

Neophyte Sage
Joined
Mar 6, 2020
Messages
1,016
Thanks for all your insightful posts... if we had a mind reader award, you'd get one.
Thanks for the feedback, much appreciated.
Don't say mindreader too hard, before you know it I start reading stuff you guys don't want published yet :p

I think you guys are pretty open and also getting more and more used to also using the forum yourselves to fetch feedback, which is a good thing!
Its fun seeing how many things are always already published and documented months before on Jira and Gitub, yet no-one notices. It shows secrecy isn't even needed :)


"People who are serious software should make their own hardware
Joking aside, thank you for the reply. I'd love to see an official HW ARM based from IXSystems, but I understand it's a small company compared to Apple and others and probably this is not a smart business model for them.
Yeah, thats one of the things I prefer about IX, if they ****up they don't only screw me (a mere free leech), but also screw their enterprise customers. I see this as an insurance policy of sorts. Not that I expect to need it, but ey, isn't that what insurance is for :)

Anyhow, lets talk options (outside of ARM), it looks like intel is going to release some interesting stuff coming months that might be of interest. But the ARM platform isn't bad either....
Actually, without disks a Fujitsu motherboard with intel 9100 idles at just a few watts, that isn't much either! :)
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,219
This vid https://youtu.be/EjzCElUhjvo was posted a couple of days ago and I was curious enough to build a TrueNAS SCALE iso myself. Thanks to the ixsystems devs you don’t need advanced skills to do this. At this early stage I see this as a statement of intent, it might all be rather different if and when the product reaches release.

Clearly, this represent a significant development effort. The middleware sits atop a systemd linux base with all that entails. Let alone all the additional functionality yet to be added to support gluster, k8s, ganesha nfs, etc. As linux does not yet support NSFV4ACL, the SMB stuff has to be adjusted to POSIXACL (no ACL editor and a new set of presets for share definitions, etc)

I was interested to see the ixsystems solution to zfs on root and boot environments. Here they’ve chosen to fork John Ramsden’s zectl. I was surprised to see mdadm in use before I realised the on the disk swap was combined into a mdadm device. Updating via the WebUI is kind of working now. The system is updated, a new boot environment is created, and the system reboots but the webui fails to start without disabling the kubelet.service.

scale_be.jpg

You can install qemu-guest-agent on a running system, so it’s boot/shutdown is well behaved for those interested in creating a TrueNAS SCALE VM with KVM.

Home users, like myself, my have picked up on the use of native docker, but this an Enterprise product that may not be a good fit for a typical home media server using a limited docker app stack in a VM and maybe some jails. But it does give a tantalising glimpse of might be if ixsystems ever choose to move TrueNAS CORE to a linux base.

Thanks for the opportunity of an early preview of TrueNAS SCALE.
 

Octopuss

Member
Joined
Jan 4, 2019
Messages
241
TrueNAS SCALE is new name for TrueNAS or something?
I don't even try pretend I understand any of this.
 

sretalla

Wizened Sage
Joined
Jan 1, 2016
Messages
3,900
TrueNAS SCALE is new name for TrueNAS or something?
No, it's a new option... you need to read the announcement... basically it's a version of TrueNAS that will run on top of Debian Linux, allowing for clusters which will support a shared filesystem and deliver the possibility to run Linux containers (including Docker) with access to that filesystem.
 

dashtesla

Member
Joined
Mar 8, 2019
Messages
44
About ARM I would like to say that Apple's decision to switch to ARM is a sign of what's to come in the future, I wouldn't be surprised if Intel and AMD in 10 years is also making ARM cpus along with their 'legacy' x86 line. Wendel from Level1Techs already reviewed some ARM based servers as well https://www.youtube.com/watch?v=vCodnn0HOyI this is from 2018 but still as relevant in 2020.

Now just imagine everything switching to ARM more and more, Microsoft Apple Linux and then why would FreeBSD stay behind and insist on not supporting ARM because of how the technology is today? Unlikely.

Obviously ixsystems doesn't have the R&D for all that on their own but I would at least start looking into it because within 2-5 years you guys may have something that is able to be released and after 5 years that might just be the build most people will look into including enterprise customers so all that time spent researching ARM on the side will pay off on top of that I suspect there will be more support from Linux ZFS and FreeBSD towards ARM in the years to come further making the ARM development of FreeNAS/TrueNAS easier.

I know i'm not your biggest enterprise customer here nor someone with a lot of influence at all but all i'm asking is, just take a look at it when you can, you might be thankful in the future :)
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
477
ARM helps Apple and Microsoft build their own proprietary processors and avoid paying "license fees" to Intel. The same logic doesn't yet apply to industry standard servers, but may in the future. There is nothing to prevent TRUENAS SCALE from adapting to use ARM. There is lots of work that has been done for Debian Linux and KVM to support ARM. It's just different from x86. Let us know when you have your ARM server for testing with.
 
Top