I have to waste an entire drive just for booting?

[jgreco]

jgreco submitted a new resource:

I have to waste an entire drive just for booting? - Why, yes, you do. Maybe even two or three!

With the advent of TrueNAS Scale, we seem to be getting a large number of people from the Linux community who are not quite grasping what TrueNAS is.

TrueNAS is not a "Linux distro" and it is not designed to try to meet your arbitrarily spec'd use cases. TrueNAS is an appliance firmware for storage, which happens to be built on top of FreeBSD and Linux as underlying operating systems. It is intended to be installed like firmware onto a storage server, which can then attach and manage a pool...

Read more about this resource...

 

[ChrisRJ]

I would like to add on the complexity argument. The latter is not only valid for the development side, as @jgreco mentioned. At least as important is the operational side. And mixing the OS with data does not help on that front. Yes, from a capacity point of view, there is considerable waste. But SSDs are so cheap now (I paid 500 Euros for my first 256 GB SSD back then), that this is not really a problem.

 

[Etorix]

Add option 3 bis: If the motherboard has an unused NVMe M.2 slot, or even an unused PCIe slot, boot from a cheap NVMe drive, and save a SATA port.

As of this writing, 16 GB Optane M10 drives can be found for $9.99 / 9.99E in second-hand listings…
In a sense that's even more a "waste" of ressources than putting a slower 120 GB SATA SS to the task, but the operative word is "cheap", not small. A home NAS is unlikely to have any use for a NVMe drive anyway: SLOG is likely of no use here, and one should not even think about L2ARC under 64 GB RAM.

 

[FJep]

Option 4 (manually partition boot drives) is not such a bad thing to fear.
I simply followed this guide HOWTO: setup a pair of larger SSDs for boot pool and data.
After that you have a boot pool of 16GB and the rest ist a fast SSD pool.

If you don't have two SATA ports left you could simply use an internal USB3 adapter and two USB SATA cases or cables like this.

 

[jgreco]

not such a bad thing to fear.

Option 4 is dangerous, because you have no way to deal with drive replacements through the GUI, and attempts to do so are actively hazardous to your pools. So, yes, fear it, and, plus, don't expect lots of sympathy in the forums when it goes sideways on you.

When you're dealing with an automated system like TrueNAS and you decide to go off-road adventuring, you run the very real risk of encountering conditions that the designers did not contemplate because they expected to be the only ones making these sorts of decisions. Making decisions that in fact collide with those design decisions is always risky. Fear it. It'd be stupid not to fear it.

 

[HoneyBadger]

This is where the "duality of TrueNAS" comes into play, and why I've often written, rewritten, and abandoned countless posts attempting to provide "blanket guidance" because there is rarely just One Answer To Rule Them All.

It's used everywhere from someone's dorm room to store their vast library of pirated media Linux ISOs where hacked-up solutions are great, all the way up to business use where it needs to retain all of that wonderful appliance-like behavior.

I don't think "fear" is the right concept, the original sentiment in the resource nails it pretty well as "understand the risks you're taking." If you're willing to risk your system deciding to not boot after an update, by all means partition it manually and best of luck.

 

[jgreco]

If you're willing to risk your system deciding to not boot after an update, by all means partition it manually and best of luck.

But it's worse than that, because it could also eat a component partition of a data pool.

And the problem is that we're seeing a huge influx of Linux users of the "I iz ooober linuxx gooroo" variety, who are shopping for the answer that they so desperately want to hear, rather than the correct answer. They tend to latch on to any hint of an answer that might vaguely hint that what they want is possible, whether this is the steady stream of H730 RAID card owners or the let's partition our boot device crowd, just to name a few.

The problem with this:

where hacked-up solutions are great, all the way up to business use where it needs to retain all of that wonderful appliance-like behavior.

is that it implies that hacking on the appliance is fine or even expected, and the problem is that it really isn't, and, worse, people have a tendency to show up here with broken systems having followed the hack-up-the-appliance strategy without bothering to own up to it.

I have tried to fairly and correctly explain the risks and tradeoffs in my resource. I don't really want or need people dismissing that and telling people not to fear it. There should be some fear. Sane, healthy fear.

 

[Ppriorfl]

Add option 3 bis: If the motherboard has an unused NVMe M.2 slot, or even an unused PCIe slot, boot from a cheap NVMe drive, and save a SATA port.

As of this writing, 16 GB Optane M10 drives can be found for $9.99 / 9.99E in second-hand listings…
In a sense that's even more a "waste" of ressources than putting a slower 120 GB SATA SS to the task, but the operative word is "cheap", not small. A home NAS is unlikely to have any use for a NVMe drive anyway: SLOG is likely of no use here, and one should not even think about L2ARC under 64 GB RAM.

So I have 2 x 2TB nvme consumer drives. I used one as the boot drive. I was going to use the other as a single stripe vdev/pool for installing apps/VM. Reading this comment I wonder if I should buy a cheap SATA drive and use it for boot and then mirror the two nvme drives for apps. But even then they will be far more space than I imagine I will ever need for apps and VM storage.

 

[Etorix]

The point of using a small (and cheap) NVMe drive to boot is to save a SATA port for a home NAS which uses no HBA, where every on-board SATA port matters for storage drives.
If you have a spare SATA port, go for a small SATA SSD and mirror your app/VM pool. 2*2 TB = 2 TB raw, best to use less than 1 TB for block storage (VM). If you won't even use that space, consider that all this extra capacity will help with endurance.

 

[Ppriorfl]

So is there a way to copy over my install from the NVMe drive to the SATA SSD or do I have to completely reinstall from scratch. Sounds like I should have installed on my SSD and setup the 2 NVMes as a mirror for other use. I installed on NVMe just because I figured it was faster and would make the system faster.

 

[jgreco]

Download your configuration database, reinstall on your new media, upload your configuration database.

 

[Ppriorfl]

@jgreco Sorry to be such a newb but I did that, installed onto the SATA SSD and imported my config. So it kept the old SSD (single disk VDEV) as the pool for apps. I can't see how to add a second disk as a mirror however. Do I have to change the apps (choose pool or unset pool) to something else and delete the pool and recreate it as a mirror? I thought there was a way to add second drive as a mirror, but doesn't seem to be an option I can find.

After all the reading I've done I'm trying to be VERY careful with anything I do in the storage section of ZFS!

And I'm super stoked how it automatically imported my data pool - that is so cool.

 

[Etorix]

I can't see how to add a second disk as a mirror however.

Storage > Pools > (gear) > Status > (disk) > (…) > Extend Expand

 

[Ppriorfl]

Storage > Pools > (gear) > Status > (disk) > (…) > Expand

@Etorix Thank you - Wow I never would have found that - when I get to the end of that pathway I see EXTEND but not Expand. Is that what you meant?

 

[Etorix]

Yes. Sorry for the error—it's the danger of writing from memory without opening an actual window to check the GUI.

 

[Ppriorfl]

Not at all - just wanted to make sure I wasn't confusing terms as I'm totally new to ZFS. It worked perfectly - Thank you so much!