Hi,
While I was in the process of refreshing my hardware, @Ericloewe provided me with some great feedback in terms of a lean and energy efficient setup when it comes to boot devices. Up to then, I used to run a SuperMicro X11SCL-IF, 4 Seagate IronWolf ST4000VN008 connected through the onboard SATA controller and a 512GB Samsung 970 Pro M.2 which was basically sitting empty and serving as a jail/plugins dumpster. TrueNAS Core was booted from a separate USB stick.
The idea that came up was to eliminate the USB drive and partition the M2 SSD in a way that I could run TrueNAS off of it as well as have the rest of the space usable for jails/plugins.
Researching into what was already there I found @seltz thread. Also @Patrick M. Hausen and @mkabir contributed to this topic. However, finding all of the information was all over the place for my setup. In addition, preparing the EFI boot loader changed fundamentally with TrueNAS 12 (I believe). Therefore, I would like to share my steps and provide an up-to-date guide.
This guide is a fresh install. I recommend to backup your system's dataset and keys for pools you would like to add after going through this.
UEFI/BIOS settings prior booting:
- AMI NVMe enabled
- Boot Mode: UEFI
- fixed boot procedure: (1) HDD, (2) USB Key, ..., (8) Built-In UEFI shell
You will need:
- USB A: flashed with the .iso of TrueNAS
- USB B: empty, will serve as installation media)
1. Insert both USBs and install TrueNAS on the empty USB B, remove USB A
2. Boot vanilla TrueNAS from USB B
# If at any point you will encounter a 'device busy' error, I recommend booting USB A's installer and perform steps 3 - 11 with the installer shell.
3. Check your system's partition table
For this guide I will use the following:
4. destroy any existing partition table for the NVMe M2 - warning deleting data!!!
5. create a fresh partition table with gpt
6. add an EFI partition (nvd0p1) that will host the boot loader (size has to be at least 512 MB for FAT32, check here)
7. set a FAT32 filesystem with 1 sector per cluster for the newly created partition nvd0p1
8. mount the FAT32 filesystem to /mnt
9. make a directory according to the FreeBSD UEFI bootstrapping procedures
10. copy the loader.efi into the directory (which is on nvd0p1)
11. unmount
From here on, make sure to boot from the vanilla TrueNAS USB B
(optional: if you want to create swap partitions do so by running
12. add the TrueNAS boot or actual main partition, here 16 GB labeled root0 which will create /dev/nvd0p2
13. for the remaining space on nvd0 a jail partition will be created, labeled jail0 at /dev/nvd0p3
14. create the actual jail, unmount and export it
15. 'copy' the actual content of the TrueNAS main partition from the USB B stick and attach it to our newly created /dev/nvd0p2 on the NVMe
16. wait for resilvering to be completed. run
17. reboot the system, remove USB B from the slot and boot into TrueNAS directly from your NVMe
18. in TrueNAS GUI go to Storage/ Pools, click Add and select 'Import an existing pool'. The dropdown list should have your jail.
While I was in the process of refreshing my hardware, @Ericloewe provided me with some great feedback in terms of a lean and energy efficient setup when it comes to boot devices. Up to then, I used to run a SuperMicro X11SCL-IF, 4 Seagate IronWolf ST4000VN008 connected through the onboard SATA controller and a 512GB Samsung 970 Pro M.2 which was basically sitting empty and serving as a jail/plugins dumpster. TrueNAS Core was booted from a separate USB stick.
The idea that came up was to eliminate the USB drive and partition the M2 SSD in a way that I could run TrueNAS off of it as well as have the rest of the space usable for jails/plugins.
Researching into what was already there I found @seltz thread. Also @Patrick M. Hausen and @mkabir contributed to this topic. However, finding all of the information was all over the place for my setup. In addition, preparing the EFI boot loader changed fundamentally with TrueNAS 12 (I believe). Therefore, I would like to share my steps and provide an up-to-date guide.
This guide is a fresh install. I recommend to backup your system's dataset and keys for pools you would like to add after going through this.
UEFI/BIOS settings prior booting:
- AMI NVMe enabled
- Boot Mode: UEFI
- fixed boot procedure: (1) HDD, (2) USB Key, ..., (8) Built-In UEFI shell
You will need:
- USB A: flashed with the .iso of TrueNAS
- USB B: empty, will serve as installation media)
1. Insert both USBs and install TrueNAS on the empty USB B, remove USB A
2. Boot vanilla TrueNAS from USB B
# If at any point you will encounter a 'device busy' error, I recommend booting USB A's installer and perform steps 3 - 11 with the installer shell.
3. Check your system's partition table
gpart listFor this guide I will use the following:
da0 USB stick I am booting offnvd0 NVMe M2 drive4. destroy any existing partition table for the NVMe M2 - warning deleting data!!!
gpart destroy -F nvd05. create a fresh partition table with gpt
gpart create -s gpt nvd06. add an EFI partition (nvd0p1) that will host the boot loader (size has to be at least 512 MB for FAT32, check here)
gpart add -s 512m -t efi nvd07. set a FAT32 filesystem with 1 sector per cluster for the newly created partition nvd0p1
newfs_msdos -F 32 -c 1 /dev/nvd0p18. mount the FAT32 filesystem to /mnt
mount -t msdosfs -o longnames /dev/nvd0p1 /mnt9. make a directory according to the FreeBSD UEFI bootstrapping procedures
mkdir -p /mnt/EFI/BOOT10. copy the loader.efi into the directory (which is on nvd0p1)
cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi11. unmount
umount /mntFrom here on, make sure to boot from the vanilla TrueNAS USB B
(optional: if you want to create swap partitions do so by running
gpart add -s 16g -t freebsd-swap -l swap0 nvd0. This will create a 16 GB swap drive for nvd0p2) -- I have skipped this.12. add the TrueNAS boot or actual main partition, here 16 GB labeled root0 which will create /dev/nvd0p2
gpart add -s 16g -t freebsd-zfs -l root0 nvd013. for the remaining space on nvd0 a jail partition will be created, labeled jail0 at /dev/nvd0p3
gpart add -t freebsd-zfs -l jail0 nvd014. create the actual jail, unmount and export it
zpool create jail /dev/nvd0p3
umount /jail
zpool export jail15. 'copy' the actual content of the TrueNAS main partition from the USB B stick and attach it to our newly created /dev/nvd0p2 on the NVMe
zpool attach boot-pool /dev/da0p2 /dev/nvd0p216. wait for resilvering to be completed. run
zpool status to see the progress17. reboot the system, remove USB B from the slot and boot into TrueNAS directly from your NVMe
18. in TrueNAS GUI go to Storage/ Pools, click Add and select 'Import an existing pool'. The dropdown list should have your jail.
