1st trueNAS build – Need pool,vdev,general install advice

[Arwen]

Their used to be an "Ideal RAID-Zx width", (which varies per 1, 2 or 3 parity disks).

However, one of the creators of ZFS said part of that went out the window with in-line compression:

How I Learned to Stop Worrying and Love RAIDZ

The popularity of OpenZFS has spawned a great community of users, sysadmins, architects and developers, contributing a wealth of advice, tips…

www.delphix.com

 

[Whattteva]

I never heard of this rule and all the recommendations I am aware of basically go for 2 more disks. So roughly not more than 5 for RAIDZ1, 8 for RAIDZ2, 11 for RAIDZ3; as always the details depend on the workload.

Can you point me to a source for the numbers you mentioned?

Why from none other than the co-creator of OpenZFS Matthew Ahrens himself! I guess I should qualify that it isn't exactly a hard rule, but should be followed if you want optimum performance.

Here's an excerpt from his blog:

For best performance on random IOPS, use a small number of disks in each RAID-Z group. E.g, 3-wide RAIDZ1, 6-wide RAIDZ2, or 9-wide RAIDZ3 (all of which use a..." of total storage for parity, in the ideal case of using large blocks). This is because RAID-Z spreads each logical block across all the devices (similar to RAID-3, in contrast with RAID-4/5/6). For even better performance, consider using mirroring.

EDIT: Looks like @Arwen is ahead of me.

 

[ChrisRJ]

Why from none other than the co-creator of OpenZFS Matthew Ahrens himself! I guess I should qualify that it isn't exactly a hard rule, but should be followed if you want optimum performance.

That makes sense. Especially since most people who go for RAIDZ? are more after net capacity than IOPS. So Matthew's post seems to cover a bit of an edge case (at least for me).

Here's an excerpt from his blog:
[..]

Thanks for the source.

 

[Chris Moore]

That makes sense. Especially since most people who go for RAIDZ? are more after net capacity than IOPS. So Matthew's post seems to cover a bit of an edge case (at least for me).


Thanks for the source.

Yes, I think most of the users in the forum are looking for the most capacity for the least cost. Where I work, they have ordered a solution that is rated for 500k IOPS. It uses SSDs. The old, disk based, solution they are replacing has 3,280 disks, but the new solution only has 456 SSDs. It is a matter of what performance you need.
What is your storage goal?

 

[oldtechie]

Well, I'm back with good news, I love TrueNAS. The only regret I have is I should have installed TrueNAS on bare metal and not virtualized it, and Scale instead of Core. It would be a better use of my hardware. Since I wanted to move off of VMWare onto open-source software. Learning how to make it work for me has been time-consuming with mixed results. For what I need my time would be better spent learning Scale VM creation.

So, why am I back? Well, it is because I would like to move my TrueNAS install from a virtual environment (i.e. Proxmox) to bare metal, without losing what I have configured in my TrueNAS VM. See, I have all my backups and auxiliary shares set up and working really well.

Is there a way I can do a clean bare metal installation of the TrueNAS Scale software without reconfiguring my data pool and all the datasets?
If this can be done, can someone point me to a script or procedure?

As always thanks for any help or suggestions,

 

[NugentS]

If you virtualised it properly - then yes
Make a copy of the config file, install to bare metal and restore the config

 

[Whattteva]

Is there a way I can do a clean bare metal installation of the TrueNAS Scale software without reconfiguring my data pool and all the datasets?

You should probably start a new thread and post more details on how exactly your virtualized setup is configured, so we can tell if it's feasible or not.

 

[WI_Hedgehog]

Ok, Well now it is time to buy some drives. I will let the forum how it going after a month of testing. Chances are if I'm back in a month it's all good.
Thanks much

Oh, you'll be back sooner than that.

Nine days later:

Well, I'm back with good news, I love TrueNAS. The only regret I have is I should have installed TrueNAS on bare metal and not virtualized it, and Scale instead of Core. It would be a better use of my hardware. Since I wanted to move off of VMWare onto open-source software. Learning how to make it work for me has been time-consuming with mixed results. For what I need my time would be better spent learning Scale VM creation. ...

And the hedgehog scores big!



(i'm going to pat myself on the back because nobody else is going to ... LOL)

 

[Arwen]

...

Is there a way I can do a clean bare metal installation of the TrueNAS Scale software without reconfiguring my data pool and all the datasets?
If this can be done, can someone point me to a script or procedure?

As always thanks for any help or suggestions,

As @NugentS said, backup your configuration file, install again and restore config.

However, their generally is one gotcha that can't be avoided, network configuration. You will need to get on the console of the newly bare metal TrueNAS and re-configure the network. It is not bad, but the device used by the VM config would almost certainly not be suitable for bare metal.

 

[oldtechie]

If you mean "virtualized it properly" all of my TrueNAS disks connected via an LSI 9211 HBA that I passthrough to the TrueNAS VM. The boot drive is a virtual drive provided by Proxmox.
On the network side of things should not be a problem.
So, I saved the TrueNAS core config, I should be able to import it into scale, right?

 

[Whattteva]

If you mean "virtualized it properly" all of my TrueNAS disks connected via an LSI 9211 HBA that I passthrough to the TrueNAS VM. The boot drive is a virtual drive provided by Proxmox.
On the network side of things should not be a problem.
So, I saved the TrueNAS core config, I should be able to import it into scale, right?

Yeah, that sounds like you should be able to import your pool just fine. Not sure about the config. I've personally never done the migrations (someone else can chime in here), but you should be able to recreate the configs from scratch... though it would require some manual tedious work on your part.

 

[oldtechie]

I would like to avoid recreating the config scratch, which could be very error-prone and tedious.
also, do you have any views on configuring a cache drive? I have a couple of small SSDs laying around.

 

[NugentS]

Whats your hardware - all of it please?

 

[oldtechie]

1. CPU Ryzen 7 5700G
2. MOB Asus x570 Pro
3. Memory 2X32GB DDR4 3200 ECC
4. LSI 9211-8i P20 IT mode (Passthrough from Proxmox)
5. Drives 3X18TB Seagate EXOS X18 in RAIDZ
6. Drives for Proxmox 3X2TB Samsung 870 EVO on mob Sata ports 1-3
Boot Drive Samsung 980 Pro SSD 1TB PCIe 4.0 (Proxmox boot drive) TrueNAS boot drive is a virtual drive from Proxmox.
The Sata ports on the mob are not visible to the TrueNAS vm.

 

[NugentS]

And how much memory does TN have?

 

[oldtechie]

32GB of memory allocated to TN.

 

[NugentS]

Some say that you want 64GB of memory assigned before using L2ARC, but as with all things your mileage will vary.

If your ARC Hit Rate > 90% its probably not worth it

 

[oldtechie]

My plan is to add 64GB to my server once the memory arrives. I will add 32GB more to TN vm