Provide ZFS profiles and advanced setup BEFORE install

[amtriorix]

Hello,

I run ZFS since day one on FreeBSD and several boxes using ZFSguru.
Despite the fact ZFSguru is not updated for about a year, it still runs rock solid.

After the years and using in fact only zfs to protect against bitrot, I found it very easy to NOT use SMB or NFS,
but simply a ZFS VOL in a raidz2 using iscsi and only ONE server that uses the iscsi target. Sharing was done on the server, including backup
So the NAS was pretty simple, the only things I did need was tuning, zfs, iscsi, smart, zfs scrub and selfhealing and of course reports.

However I would like to have my backup also on the nas and seen the plugins of zfsgure were pretty outdated, I had a problem.
I also did want to have a deep file search and index system, so I was stuck.

Of course I could run them on the server, but I am stubborn.

Seen I use a heave backup plan, using duplicity (s3 cloud using asymmetric encryption), borgbackup
and some sync methods with rclone and s3ql (using big cache), I decided to give a try to freenas, oeps truenas.

It is a complete disaster at the point of setup. I really do expect better guys.

FreeBSD runs ZFS at root since years. I want to see on my disks
1) a zfs construction using GPTid or in case of linux UUID, seen the later for can swap labels if a disk is down, the first, performs it automagically but still I prefer ID
better safe then sorry, especially with ZFS
2) ZFS using whole disks
3) in case of partitions, proper alignment (4K, ashift, etc)
4) before install of any nas software, I want to see ZFS predefined profiles, from the equivalent of stripe, mirror, raidz1, raidz2, raidz3 and the possibility to map
L2ARC cache
5) i want to see before install an advanced section of zfs to set compression, etc etc
6) a section to tune freebsd sysctl

So I did install treuenas, selected 6 discs and without any further question, the software did start to create a MIRROR zfs pool in my 6 discs, where I did
wanted actually a raidz2...

So in terms of truenas, it should be really better.

Everything related to nfs, samba and all those services are 3th priority services.

The first priority is:

1) high configurable and tunable setup, like:
a) FreeBSD system
b) ZFS filesystem, also with profiles to create the right zfs system and ZVOL, also config options such as compression type, prefetch or not, etc.
c) network
d) logging (including external newsyslog)
e) signaling with SNMP
f) reporting with mailreports
g) temperature control
h) WoL and powermanagement
i) UPS and handling in case of power outage
j) smart harddisk
k) heartbeat
l) NTP client to have the proper time
m) scrub and repair options at the cli menu/repair mode

The second priority:
nice status screens and GUI
create the proper authentication

Third priority
select the used 'sharing services' with plugins such as NFS, SAMBA and ISCSI
(remember there is no need to have NFS and SAMBA if ISCSI is used only but protection should be in place with firewall and ONE connection only to the target)

Fourth priority
backup plugins

Fifth priority
search and indexing

Six priority
everything else in truenas

Sixth priority
mobile app.

I suggest to try to find old articles on the internet about ZFS and FreeBSD, especially related to alignment and ZFS on root and for sure GPTid/UUID/etc
but also to setup a test base, then power off harddisks in a raidz...config, powercut and see if everything boots up properly or could be repaired.

Sop this is a highly recommended layout of course open for debate. But anyway, if truenas has no configurable ZFS profiles before install
of the truenas, it is basically for me, useless until this is modified properly.

Seen FreeBSD is porting OpenZFS, maybe it is time to move all together to the linux base, seen many ports like s3ql are broken on BSD.
And I say this with pain in my heart.

 

[Samuel Tai]

TrueNAS already does all this, except for the mobile app, but it sounds like you expected it to work like ZFSguru or vanilla FreeBSD. If you didn't read the documentation before jumping in, you're a victim of your own preconceptions.

Yes, it is derived from FreeBSD, but FreeNAS/TrueNAS absolutely requires careful planning BEFORE installation. The GUI allows creation of any pool topology equivalent to CLI commands using zpool, but if you didn't pull down to explore all the possible options, that's not the GUI's fault, that's your own.

As for sharing, out of the box all sharing methods are disabled, and it's up to you to enable what suits your requirements.

 

[amtriorix]

TrueNAS already does all this, except for the mobile app, but it sounds like you expected it to work like ZFSguru or vanilla FreeBSD. If you didn't read the documentation before jumping in, you're a victim of your own preconceptions.

Yes, it is derived from FreeBSD, but FreeNAS/TrueNAS absolutely requires careful planning BEFORE installation. The GUI allows creation of any pool topology equivalent to CLI commands using zpool, but if you didn't pull down to explore all the possible options, that's not the GUI's fault, that's your own.

As for sharing, out of the box all sharing methods are disabled, and it's up to you to enable what suits your requirements.

I am a BSD expert since the early versions of FreeBSD, You can not expect for the regular Joe to setup properly ZFS filesystems.
If ZFSguru can do it with php code and some scripts, You can do it as well

 

[Kris Moore]

There is no "one right way" to setup ZFS for regular Joe's. Its an enterprise class file-system, with many valid options and setups to each unique storage situation. TrueNAS takes the approach of trying to provide all the knobs available to setup for a variety of scenarios, its incumbent on the end-user to review that the pool layout is as they desire before clicking "create". If we could do a 'one-size fits all' ZFS, we would, but the reality is that would end up being not suitable for a large % of the user base.

 

[amtriorix]

I totally disagree, people who do know ZFSguru do know it is a piece of cake to install pretty advanced ZFS structures, from mirror, stripe to raidz1, raidz2, raidz3 and knowing freenas, rebranded to truenas, I can guarantee you in my about 30 years IT experience I did rescue many systems who went gogogogo with freenas/truenas. And the reason is exactly what You do post, the lack of proper ZFS setup for the regulkar joe.

Nothing is worse as to create and propose a NAS solution, ask it to trust your data on it and finally see, the setup procedure is that immature,
really You are better of to use not at all truenas.

You did rebrand freenas to a good name, I think, it is time to dig into the git repo of zfsguru and take over the installation process so You have
a decent ZFS setup at installation, also for those who are not expert in the field of ZFS.

Because I see even You guys make the terrible mistake to relay on slices and their labels and not GPTid/UUID. I do agree, freebsd does do this far better as linux OpenZfs to match up your disks, but I have seen many issues when one disk did crash and ZFS was using labels and not the harddisk ID

 

[Samuel Tai]

I am a BSD expert since the early versions of FreeBSD

Those with extensive Linux or BSD experience find their experience both enabling and hindering their ability to work on TrueNAS. They have a head start on the concepts and some of the CLI tools, but are often surprised the GUI manages much of their configuration via a database, and administration via GUI processes, API calls, or TrueCommand single-pane actions will overwrite direct edits to config files. This is just how the product was designed.

If you expected to be able to bypass the GUI and do everything via CLI and vi, again, that's an error of preconception on your part.

Because I see even You guys make the terrible mistake to relay on slices and their labels and not GPTid/UUID. I do agree, freebsd does do this far better as linux OpenZfs to match up your disks, but I have seen many issues when one disk did crash and ZFS was using labels and not the harddisk ID

TrueNAS already uses GPT UUIDs. However, the GUI displays disk partitions to be more user-friendly. Under the hood, it's still using GPT UUIDs.

 

[amtriorix]

I know that FreeBSD is using UUID unther the good, that is why I did state it is soing it automagically.
However you can still create your ZFS raid directly with GPT UUID which is still the most safe way to go and also a good practice btw to write down
the uuid on a label and put it on the box. Everything is always nice until a harddisk goes gogo.

I do expect indeed that basic setup, even a mirror, raid and raidz1/raidz2/raidz3 including L2ARC cache can be done through gui.

Look at ZFSguru, they did it, the code is there to read. It is php and it simply threws out the proper zfs commands.

 

[Yorick]

if truenas has no configurable ZFS profiles before install
of the truenas, it is basically for me, useless until this is modified properly.

That was a long post and I wasn't entirely clear what your ask is. You are saying you want TrueNAS to allow you to configure your pool layout(s) before install? As others have pointed out, that's done after install, through the UI. The documentation for that is at ZFS Pools | TrueNAS Documentation Hub

As for 6 disks being configured as 3 mirror vdevs, that's by using the "Suggest Layout" button? Or how did that come about?

A debate can be had on whether the suggested layout for 6 disks should be mirror vdevs (better performance) or raidz2 (more space). Ultimately, there truly is no "one size fits all".

A possible improvement to "Suggest Layout" is to have that lead to a second screen that offers two or more layouts, with pros and cons listed, so users can choose. If that or some other improvement is what you want, you can open that as a suggestion in Jira, then advertise it ... forums, social media, friends ... to get 10 upvotes, at which point it will be discussed by ix for possible implementation.

My recommendation is to keep that Jira suggestion crisp. Aim for concise, and a clearly defined ask / set of features.

 

[ChrisRJ]

@Yorick , I also wasn't really clear on the actual intent behind the initial post. The impression was that of someone with a very specific setup in mind and new to TrueNAS, who now complains heavily that his particular use-case is not well supported and demands changes.

 

[Patrick M. Hausen]

But TN can create a RAIDZ2 from 6 disks. I really don't get what the OP is complaining about. It uses GPT partitions of type freebsd-zfs as anyone with half a clue about ZFS on FreeBSD would - and the OP demanded, I am not claiming him to be clueless - it automatically does proper 4k alignment ... in summary it does about everything the OP demands except "being zfsguru". And what the heck is that, anyway? Never heard of it and I am using ZFS since FreeBSD 7 as well.

End rant.

Edit: read that tirade again, I think now I get it. He chose to *install* TrueNAS on all of his 6 disks. Of course TN created a mirror. Reading the documentation helps.

Dear @amtriorix, TrueNAS demands a dedicated install disk or two or more of them and dedicated storage disks. You don't install on the disks you want to use for your data. After installation you have all the pool topology options at your disposal with a drag and a click. After installation ...

 

[Yorick]

He chose to *install* TrueNAS on all of his 6 disks.

I completely missed that. Mein Gott. Okay, thread solved, thanks!

 

[Samuel Tai]

ZFSguru appears to be an abandoned project for a GUI front-end to ZFS. Development ceased in 2011. No wonder we've not heard about this.

 

[amtriorix]

Wrong, the intent of this post is to improve the setup of TrueNas. Did I say that truenas/freenas is a disaster, no.
I only said it is time to have a better setup.
It does not really help to play game-like advocacy.

The fact zfsguru was clearly a one person project does not mean the project was stable and in fact
pretty well written in php, seen I am a senior php developer myself and I know a lot about ZFS.
If you look into the wayback machine zfsguru was active for a good period and used by those who want to quickly setup a zfs pool in
combo with a nas, of course the samba, nfs, etc etc stuff was not alike freenas at that time, but the setup method to have
a zfs pool was pretty awesome.

And that is the part I like to see integrated before actually installing truenas...the setup method to create the zpool using TUI.

It is not that difficult to threw zfs commands or cli command using php etc.
Btw we all know that clonezilla works alike.

I found in my library the latest version of zfsguru (usb) and I forked the git repo of someone who cloned the latest version.
Latest version to study for integration.:

375 MB file on MEGA

mega.nz

I cloned also a gitrepo of zfsguru2 for what it is worth:

GitHub - pehsa/ZFSguru2: Fork of the ZFSguru project http://zfsguru.com/

Fork of the ZFSguru project http://zfsguru.com/. Contribute to pehsa/ZFSguru2 development by creating an account on GitHub.

github.com

Do not understand me wrong here, I do belief in true nas, but it is only a bit work to merge the work of zfsguru inside the setup of truenas
and you have the best of both worlds.

 

[amtriorix]

But TN can create a RAIDZ2 from 6 disks. I really don't get what the OP is complaining about. It uses GPT partitions of type freebsd-zfs as anyone with half a clue about ZFS on FreeBSD would - and the OP demanded, I am not claiming him to be clueless - it automatically does proper 4k alignment ... in summary it does about everything the OP demands except "being zfsguru". And what the heck is that, anyway? Never heard of it and I am using ZFS since FreeBSD 7 as well.

End rant.

Edit: read that tirade again, I think now I get it. He chose to *install* TrueNAS on all of his 6 disks. Of course TN created a mirror. Reading the documentation helps.

Dear @amtriorix, TrueNAS demands a dedicated install disk or two or more of them and dedicated storage disks. You don't install on the disks you want to use for your data. After installation you have all the pool topology options at your disposal with a drag and a click. After installation ...

I know that and the project I do mention does install everything at the zfs pool, such like freebsd at root.
It is perfectly possible to create a bpool, rpool and dpool (boot,root,data) where you use mirror for the first two pools and a custom raidz1/raidz2/raidz3 for the data.

This has advantages because not only the nas software is protected by the zfs pool, you can also snapshot the installation
and most of all on a port with a few sata/sas ports you do not lose ports, because installing the nas os on USB is really a bad idea.

Alignement was a big issue years ago, it was not that easy to calculate the right setup for your slices because it had impact, but of course things got improved and it is done know using the right ashift value etc etc.

But also zfs has the option to select several nice tweaks such as the poor man raid 'copies', 'deduplication', 'prefetching or not', l2arc cache, compression type...native encryption and native NFS support....finally it has zfs sync/scrub to be used for backup or resilver etc etc.

The file system is too beautiful to not provide extended screens at install but also at post-configuration and tuning.

 

[Patrick M. Hausen]

You cannot create the storage pool at install time and you probably never will be able to do it. The entire idea is to install TrueNAS to a boot medium, boot that, then create your storage pool. The division between boot ("firmware") and storage like in any common appliance is most probably set in stone, because it makes sense. You are free to suggest pool creation at install time via iXsystems' JIRA, though.

Have you even made it to the "create pool" function of TrueNAS after installation, yet? What are you missing?

 

[HoneyBadger]

Did I say that truenas/freenas is a disaster, no.

It is a complete disaster at the point of setup. I really do expect better guys.

:thinking emoji:

If I can politely interject, my perspective on TrueNAS is that the "setup" portion continues after the TUI/initial installation. But from the point of ZFSguru, it appears that it does everything in that single step, including the initial pool provisioning.

Having never used ZFSguru, is it possible to install the "boot/setup" environment on one set of drive(s) and then have it create a separate pool afterwards with all of the same TUI configuration, or is that only available for the initial deployment?

 

[Patrick M. Hausen]

I know that and the project I do mention does install everything at the zfs pool, such like freebsd at root.
It is perfectly possible to create a bpool, rpool and dpool (boot,root,data) where you use mirror for the first two pools and a custom raidz1/raidz2/raidz3 for the data.

How do you start this software without a running system? It looks like a lot of PHP, so you need an installed FreeBSD first? Possibly I am not seeing the whole picture - can you boot in this thing somehow?

 

[amtriorix]

How do you start this software without a running system? It looks like a lot of PHP, so you need an installed FreeBSD first? Possibly I am not seeing the whole picture - can you boot in this thing somehow?

I did include the latest img i could find, try it out in a virtualbox/vmware/proxmox whatever with a bunch of 6 virtual hd.
Indeed, it does boot from the zfspool.

It was a combo of scripts, freebsd and the webinterface written in php.
There was also tuning of sysctl.conf (important)

Again I used it mainly to setup the right zfs pool, then I created in the gui a ZVOL and I used ctld to have an iscsi target with auth.
From that point on I used a server to access the iscsi and I could take advantage of the zfs to avoid bitrot etc.
Sharing, kerberos etc all that was done at server level.

version 0.1.7 found in my own archives (usb image, boot it to install), you can start with this:

375 MB file on MEGA

mega.nz

latest preview version 0.1.8, webinterfaces etc:

Google Code Archive - Long-term storage for Google Code Project Hosting.

code.google.com

And of course the links I posted already

 

[amtriorix]

:thinking emoji:

If I can politely interject, my perspective on TrueNAS is that the "setup" portion continues after the TUI/initial installation. But from the point of ZFSguru, it appears that it does everything in that single step, including the initial pool provisioning.

Having never used ZFSguru, is it possible to install the "boot/setup" environment on one set of drive(s) and then have it create a separate pool afterwards with all of the same TUI configuration, or is that only available for the initial deployment?

zfs is a filesystem that makes the disks in fact transparent. So the right approach is to create the proper pools
and it is possible to boot from zfs, that is why I suggest to have:
1) a mirrored bpool (boot and maybe inclusion of sync/scrub/maintain/tune)
2) a mirrored rpool (root file system with inclusing of truenas and settings etc)
3) a customized dpool (pool with the selected zXraid for data
4) optional, creation if ZVOL (to use in combo with iscsi or whatever)

 

[Patrick M. Hausen]

So it is a "storage OS" you can boot. Fair enough. For TrueNAS you have to install first (on a dedicated boot disk or two), but then you have a really pleasant UI to manage your storage disks, iSCSI etc. So to be fair, we should not compare the TrueNAS installer with ZFSguru but a running TrueNAS and its UI. If there's lessons that can still learn in terms of usability, I'm all for it.

Have you given the UI a try, yet? The UI definitely does not slam the whole of your 6 disks into a mirror ...