TrueNas Scale with SSD Namespace's for boot, slog, metadata and dataset for vm

[benda]

First i am new to TrueNas.

I have the following components:

• Mainboard: Kontron K3842-Q mATC
• RAM: 1x 16GB Crucial DDR5-4800 DIMM CL40 Single
• CPU: INTEL Core i3-14100T 2.7GHz FC-LGA16A
• Lüfter CPU: Noctua Kühler NH-L9i 1700
• PSU: Corsair SF600
• SSD: 2x Kingston DC1500M 1.92TB
• HDD: 3x Seagate Guardian BarraCuda ST5000LM000 - Festplatte - 5 TB
  

I plan to build a home server with less as possible power consumption.
I will run some docker container for media streaming, image backup and home automation.
The HDD‘s are for media files and images and the SSD for VM dataset and docker/Kubernetes volume mount.

I wanna split the SSD in 6 namespaces for boot, slog, metadata and l2arc (HDD pool), data set for vm and docker.
Both SSD have the same namespaces and sizes to use these in mirror.

Will this work?

The Kingston DC1500M supports up to 64 namespaces („virtual disk“).
I prefer to use 2x SSD with namespace, instead of separate ssd for each purpose, because of power saving.

The l2arc is to minimize the spin up for the HDD‘s and slog, to speed up the HDD pool.

 

[benda]

1. Is the idea with the SSD Namespace stupid? If yes why
2. Will the l2arc minimize the spin up of the HDD pool?
3. What else will help to reduce the spin up of the HDD pool?

 

[artlessknave]

Is the idea with the SSD Namespace stupid? If yes why

what is "Namespace"? this isnt a concept in truenas that I know of. it's not 'stupid', it just doesn't exist. you need to use terminology that is valid for us to understand what you mean.

wanna split the SSD in 6 namespaces for boot, slog, metadata and l2arc (HDD pool), data set for vm and docker.

assuming that you mean "pool", no, You cannot split any drives with truenas. each drive belongs to one pool . This is by design, as it makes maintenance far simpler and reliable.
you would need at least 1 more SSD for boot (smallest you can get, NOT USB 120GB-240GB is the most common)

I prefer to use 2x SSD with namespace, instead of separate ssd for each purpose, because of power saving.

SSDs cost very little power. none of what you are describing, even it it was possible with truenas, will save any power.

slog,

and slog, to speed up the HDD pool.

this not what SLOG does. if you do not have stats to prove you would benefit from SLOG. you are not ready for SLOG. a lot of mis-info mislabels SLOG as a 'write cache'; this is flat out incorrect, and it most cases will just do nothing at best or slow you down at worst.

Will the l2arc minimize the spin up of the HDD pool?

depends, however, you do not have enough RAM to bother. l2arc requires ram, ram that could, instead, be ARC, which is MUCH faster. 64GB-128GB before considering l2arc

Mainboard: Kontron K3842-Q mATC

truenas typically does not do well on random parts. never even heard of this company. seeing as it has intel graphics and an audio controller, im pretty sure it doesnt count as supported. ony 4 sata is low for a Matx board. 1 x16 and 1x1 slot is really low for a matx.

HDD: 3x Seagate Guardian BarraCuda ST5000LM000 - Festplatte - 5 TB

the numbers here indicate raidz1. absolutely not. no raidz1 on spinning drives larger than 2tb. even then, only if you know exactly what you are doing and have backups.

you do not have enough SATA ports to fix this build. you would have to buy an HBA at a minimum.

there are ways to add an SSD to the pool as special vdevs but your listed SSD would be wasted on that. far and away too big.

you need to do some serious reading first. fortunately, there are LOTS of relevant links in my signature!

what you are envisioning here is possible, but not with truenas.

 

[benda]

@artlessknave thank you for your detailed answer.

SSD namespace is an NVM Express feature. A somehow low level partitioning. The system sees it as separate drives.
Namespace explained
List of SSD with support for >1 namespaces


Kontron is a Austria company that make industrial electronic components.

To get more Sata ports, i can use a PCIe card like this for example.

 

[artlessknave]

o get more Sata ports, i can use a PCIe card like this for example.

no, those are garbage. actual garbage. they do not reliably present disks to ZFS, doing things like round robin disk access or just plain being slow, causing errors and problems. you need to use a sas HBA. the LSI ones are very well known to work properly and reliably (importantly, they work reliably and propery when things go wrong, like disks dying, which is very important for storage reliability), and are worth whatever it costs to get, though you have to watch out for fakes, as the counterfeit ones have the same problems as the sata card above.

Namespace explained

hmm. interesting, however, I would highly recommend to not use that for boot, don't bother with slog/l2arc, and there is no reason to have separate pools for VM and docker, they will run in separate datasets anyway. that leaves metadata. get separate disks for that. if anything goes wrong with your metadata, the POOL is lost.

an HBA will have 8 ports usually.


the problem with what you propose is that the more complex you make it, the more likely for something to go wrong. truenas upgrades, for example, were designed for dedicated boot disks and trying to change that tends to cause people issues. people have come to the forums because they cant replace any disks in their pool because they tried to hack partitions into the pools and that gave them blank serial numbers, and truenas WONT replace duplicate serial numbers...and if they are all blank, they show up as duplicate.

there are other nas distros that handle customization much better because they were NOT built as appliances. appliances are usually built such that every one of them works the same, making supporting them simple and reliable, and often trying to change that leads to dataloss.

 

[Etorix]

never even heard of this company. seeing as it has intel graphics and an audio controller, im pretty sure it doesnt count as supported.

Kontron is the German arm of Fujitsu. They make corporate and industrial motherboards.
This particular motherboard appears to be a desktop (or workstation-in-the-plant) on "corporate" Q670 chipset. I'd say it is OK-ish in the sense that is is guaranteed 100% free of gamer-class crap.
The Core i3 CPU does not support ECC anyway.

ony 4 sata is low for a Matx board.

Unfortunately typical of the latest generations. SATA is just falling out of fashion. Which highlights that an older board, with more SATA ports, and a more suitable distribution of PCIe slots (more x4 and/x8 slots, no x1), would be better.

the numbers here indicate raidz1. absolutely not. no raidz1 on spinning drives larger than 2tb. even then, only if you know exactly what you are doing and have backups.

Worse of all: These 2.5" Baracuda drives are… SMR. Not to be used with ZFS, especially in any form of raidz!

 

[artlessknave]

These 2.5" Baracuda drives are… SMR.

oh I didnt even think of checking that. that is, indeed, a massive, universe shaking, "no".

 

[artlessknave]

i will add that my goto example for cheap, usually decently easy to get, and definitely supported, is a simple x9scm-f/e3-1230v2/LSI 2008/2308 + 32GB ecc UDIMM from ebay. should be 200$-400$. you do need to update the bios though, due to a date bug.

 

[benda]

Kontron is the German arm of Fujitsu. They make corporate and industrial motherboards.
This particular motherboard appears to be a desktop (or workstation-in-the-plant) on "corporate" Q670 chipset. I'd say it is OK-ish in the sense that is is guaranteed 100% free of gamer-class crap.

Good to know. Was looking at the Kontron Inprint site. There is a Austrian address.

The Core i3 CPU does not support ECC anyway.

I know, but due to the low supply of motherboards and CPU for a reasonable price i decided that this is not a 'must'. It's also only a home server.

Unfortunately typical of the latest generations. SATA is just falling out of fashion. Which highlights that an older board, with more SATA ports, and a more suitable distribution of PCIe slots (more x4 and/x8 slots, no x1), would be better.

Many thanks for this hint. Did not checked it. Saw only the green label and Seagate. Will send them back.

Worse of all: These 2.5" Baracuda drives are… SMR. Not to be used with ZFS, especially in any form of raidz!

 

[benda]

hmm. interesting, however, I would highly recommend to not use that for boot, don't bother with slog/l2arc, and there is no reason to have separate pools for VM and docker, they will run in separate datasets anyway. that leaves metadata. get separate disks for that. if anything goes wrong with your metadata, the POOL is lost.

Have a 128GB SSD spare. Will use this for boot.
So slog/l2arc and pool for VM and Docker with namespaces on the mirrored ssd would be fine?

Can you recommend a size for slog and l2arc (10TB HDD pool)?

32GB RAM are fine, or 64GB is a must? slog, l2arc only to minimise "spin up" of the HDD's.

 

[LarsR]

A slog only helps with sync writes, e.g. iscsi and NFS. If all you do is use smb shares you dont need a slog.
L2arc is not recommended below 64gb of ram. So i'd say ditch the Idea of slog/l2arc and use the nvme Just AS dedicated fast Pools for Apps/vms

 

[artlessknave]

So slog/l2arc and pool for VM and Docker with namespaces on the mirrored ssd would be fine?

NO. SLOG IS NOT A CACHE.
do not just add slog because you think it might help; SLOG will NOT "minimize spin up of the HDDs" whatsoever.

SLOG is a backup of your synchonous writes; all writes go to the write cache in RAM (ARC) and then are written to the pool. enabling sync writes adds an extra step - dump the write to the ZIL in the pool ASAP (interrupting other writes), acknowledge the sync write as written to non volatile storage, THEN write again into the correct place in the pool, and then marks the ZIL used space as free. on spinners, as I think most can imagine, this is brutally slow, basically double, or more, the write time. a SLOG moves this from the ZIL in the pool to a special vdev, allowing that extra write to occur in parallel; the ZIL/SLOG is never read from unless there is a failure that causes the writes in the cache (RAM) to be lost.

SMB/CIFS cannot use sync writes; the protocol does not have the ability to interact with that functionally at all.

sync writes are generally for VM or Database storage, where any lost write can be detrimental. it is exceedingly rare that a home server can even benefit from this, and misconfiguring a SLOG can make your writes slower.

additionally, a SLOG is 100% percent writes, and will destroy SSDs that do not have very high endurance. trying to combine this with other writes is a recipe for failure.

as I said earlier, if you do not already have proof you need a SLOG, you do not need a SLOG; its an advanced vdev with very specific uses that generally do not apply to home use.

L2ARC is more of a grey area, but it will only stop HDD spin up if every file accessed fits into the l2arc.

 

[ChrisRJ]

@benda , what made you go for TrueNAS in the first place?

As you probably guessed from the earlier responses, some things are "special" with ZFS. You need to understand that ZFS is serious enterprise-grade storage. That means it comes with a level of power, and therefore complexity, which needs to be understood. So in my personal opinion you will need to spend a number of hours reading up on things.

If you don't want to do this, which is perfectly fine, other solutions may be a better fit.

 

[benda]

Thx@benda , what made you go for TrueNAS in the first place?

As you probably guessed from the earlier responses, some things are "special" with ZFS. You need to understand that ZFS is serious enterprise-grade storage. That means it comes with a level of power, and therefore complexity, which needs to be understood. So in my personal opinion you will need to spend a number of hours reading up on things.

If you don't want to do this, which is perfectly fine, other solutions may be a better fit.

Good point. I have read good things about ZFS and Truenas. It also seems very flexible for different purposes.
I watched and have read truenas related content, but due to the complexity and my goal to save some energy with less HHD/SSD questions were still open.

Thanks to the replies, they are answered. Thank you.

 

[anodos]

SMB/CIFS cannot use sync writes; the protocol does not have the ability to interact with that functionally at all.

If an SMB client opens a file with the SMB flag WRITE_THROUGH, then writes will be synchronous. This is generally uncommon, but IIRC MacOS clients may do it more frequently (especially during time machine backups). The MacOS caveat is part of why you see some "tuning guides" recommending turning off samba's handling for sync write requests. I suppose it probably needs to be stated that ignoring client requests for sync writes (especially during backups) is not a particularly wise course of action :)

[MS-SMB2]: SMB2 WRITE Request

The SMB2 WRITE Request packet is sent by the client to write data to the file or named pipe on the server. This request is

learn.microsoft.com

 

[ChrisRJ]

Good point. I have read good things about ZFS and Truenas. It also seems very flexible for different purposes.

No disagreement here ;-). But as you discovered this comes with complexity.

I watched and have read truenas related content,

Be aware that a lot of what you find outside this forum is bogus.

but due to the complexity and my goal to save some energy with less HHD/SSD questions were still open.

Saving energy is not exactly a sweet-spot of TrueNAS.