Great thread!
Just wanted to chime in and confirm the slight mod on this approach works well.
I wanted to boot with dual NVMe drives, two SAMSUNG 980 M.2 2280 250GB. Have been doing some eval of TrueNAS 12, and was at a point to wipe and start over (again). So I did the following:
- Temporarily attached a 16 GB USB thumb drive
- Fresh install, and selected 3 drives for boot-pool: The 16 GB USB plus the two NVMe drives
- This created a 3-way mirror, but partition was limited to size of 16 GB USB drive
- After installed removed USB drive from pool
End up with proper UEFI boot, already built out mirror on NVMe for boot-pool.
Was now fighting with TrueNAS to setup dual 16 GB swap, one on each. Prior to then creating ZFS mirror vdev on remaining NVMe free space. But having issues because of course TrueNAS always wants to do it's own thing, and admitely I haven't yet figured enough of the "middle-ware" to successfully tame it. I did find this thread on scheduling a post init script to do swapon stuff:
FreeNAS by default will allocate 2GiB on each pool HD and use this as swap. Unfortunately, if a disk fails, your system may crash. If your boot devices is more reliable than a USB, and you have the space, it may be beneficial to relocate swap to...
www.truenas.com
If I recall, when I did install direct to dual mirrored NVMe, it automagically created swap and a 16 GB GELI device? Not sure if anyone has advice on how to recreate that after doing this workaround for boot-pool and usable zpool on dual NVMe / SSD. In such a way that it plays well with TrueNAS base config files, and is persistent between reboots, with out janky work arounds. Might be handy to have in the thread.
Given how much fighting it takes to get more advanced setup out of TrueNAS, not sure I will continue down this path. Only feedback I can give to any devs reading posts: More flexibility would be nice. Seems a large number of users would be more than capable of properly using advanced installs. Options for shared partitioning on boot devices. Ability to better control swap partitions. I see a lot of comments, threads such as this one, on how to hack your way around not only built-in, but practically forced limitations. Food for thought.