- Joined
- Nov 25, 2013
- Messages
- 7,776
Moderator Warning: This HOWTO will damage the ability of your TrueNAS appliance to properly maintain itself, will break the ability to replace failed drives, and may cause problems with upgrades and updates. TrueNAS is not designed to do this sort of partitioning, and the developers are not interested in supporting this. You may be on your own if you do this. Further discussion of the topic is available here in this resource.
Hi all,
first of all: this is an unsupported configuration. I run it myself for quite some time and I'm confident that it will continue to work, but you proceed at your own risk.
1. Motivation
Current series of new SSDs frequently start at 250 GB or even 500 GB size. It is a waste of space and money to only put the boot pool on a drive this large. OTOH in a production installation it is highly desirable to use SSDs instead of USB thumb drives.
My system has two SSDs for the boot pool and another pool with virtual machines. The ZVOLs in the latter are regularly replicated to my main data pool (RAIDZ2) comprised of spinning disks.
2. Prerequisites
If you already run FreeNAS from one or two SSDs that you like to repurpose, you must perform a fresh installation. It is not possible to shrink a vdev, so we need to repartition.
Parts:
I'll assume UEFI boot throughout this post.
3. Preparation
As stated, if you are already running the system, a complete reinstall is necessary, so:
Use your preferred method (I frequently use VMware Fusion on my Mac) to perform a fresh FreeNAS install. Actually there's nothing that fundamentally prohibits what we are trying to achieve now, it is just the installer that won't let us partition the system drive(s) manually. Hence the workaround with a smaller initial drive.
5. Boot the USB installation
This should be easy using the boot menu of your EFI BIOS. The tricky part is that you want to make sure you are really running off the USB drive and don't import and mount the pool still on your SSDs. They are both named the same ("boot-pool"), so the system might get confused. I do not have the time to try it in a VM and do a transcript. Up to this point you will definitely not damage anything, stay calm ;)
6. Enable SSH and login
Configure the "new" FreeNAS installation to the point where you can get a shell via SSH. Then do
So we are running from USB (da0) ...
7. Wipe your SSDs
My SSDs are ada4 and ada5 (see above), ada0 - ada3 are the spinning disks that build my main storage pool. Danger, Will Robinson! We are deleting the old installation, now. Don't confuse the disk drives!
8. Transfer the new system to the SSDs
Copy the partition table:
Attach the devices to the zpool:
Copy the EFI boot partition:
Wait for the resilver to finish! (should not take long)
Detach the USB drive from the pool:
Result:
You should have quite some free space on your SSDs, now.
9. Shutdown, unplug the USB drive, boot from your SSDs, restore services
If the system boots alright, you might want to take the time to import your main data pool and restore your configuration, now.
Apart from the smaller partition size (which the installer refuses to create) there is absolutely nothing different from a regular install. So unless some future upgrade decides to wipe and repartition the system drives during the process this setup will definitely work like any other. We had that (wipe and repartition) once in the past when FreeNAS switched from UFS to ZFS for the boot drive. There might be a situation in the future when the boot loader changes (again) - you are on your own, watch the release notes for every major upgrade!
10. Create a new pool in the available space
Again we will need the command line, when we are finished the pool will be available in the GUI like any other:
Look for these in the output of the
Now create the pool - use the UUIDs from the previous step:
Last we export the pool from the command line:
11. Import the new pool from the GUI
Done! Enjoy :)
Patrick
Edit 20210514: replace "freenas-boot" with "boot-pool" for current versions
Hi all,
first of all: this is an unsupported configuration. I run it myself for quite some time and I'm confident that it will continue to work, but you proceed at your own risk.
1. Motivation
Current series of new SSDs frequently start at 250 GB or even 500 GB size. It is a waste of space and money to only put the boot pool on a drive this large. OTOH in a production installation it is highly desirable to use SSDs instead of USB thumb drives.
My system has two SSDs for the boot pool and another pool with virtual machines. The ZVOLs in the latter are regularly replicated to my main data pool (RAIDZ2) comprised of spinning disks.
2. Prerequisites
If you already run FreeNAS from one or two SSDs that you like to repurpose, you must perform a fresh installation. It is not possible to shrink a vdev, so we need to repartition.
Parts:
- 1 USB drive of suitable size, 16 or 32 GB
- 1, better 2 SSDs for the boot pool and additional data
Code:
ada4: <Samsung SSD 860 PRO 512GB RVM01B6Q> ACS-4 ATA SATA 3.x device ada4: Serial Number S42YNX0MA13733E ada4: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada4: Command Queueing enabled ada4: 488386MB (1000215216 512 byte sectors) ada5 at ahcich5 bus 0 scbus5 target 0 lun 0 ada5: <Samsung SSD 860 PRO 512GB RVM01B6Q> ACS-4 ATA SATA 3.x device ada5: Serial Number S42YNX0MA13643E ada5: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada5: Command Queueing enabled ada5: 488386MB (1000215216 512 byte sectors)
I'll assume UEFI boot throughout this post.
3. Preparation
As stated, if you are already running the system, a complete reinstall is necessary, so:
- Backup your configuration
- Export your data pools from the UI
Use your preferred method (I frequently use VMware Fusion on my Mac) to perform a fresh FreeNAS install. Actually there's nothing that fundamentally prohibits what we are trying to achieve now, it is just the installer that won't let us partition the system drive(s) manually. Hence the workaround with a smaller initial drive.
5. Boot the USB installation
This should be easy using the boot menu of your EFI BIOS. The tricky part is that you want to make sure you are really running off the USB drive and don't import and mount the pool still on your SSDs. They are both named the same ("boot-pool"), so the system might get confused. I do not have the time to try it in a VM and do a transcript. Up to this point you will definitely not damage anything, stay calm ;)
6. Enable SSH and login
Configure the "new" FreeNAS installation to the point where you can get a shell via SSH. Then do
Code:
zpool status boot-pool
pool: boot-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
da0p2 ONLINE 0 0 0
errors: No known data errors
So we are running from USB (da0) ...
7. Wipe your SSDs
My SSDs are ada4 and ada5 (see above), ada0 - ada3 are the spinning disks that build my main storage pool. Danger, Will Robinson! We are deleting the old installation, now. Don't confuse the disk drives!
Code:
zpool labelclear ada4p2 zpool labelclear ada5p2 gpart delete -i2 ada4 gpart delete -i2 ada5 gpart delete -i1 ada4 gpart delete -i1 ada5 gpart destroy ada4 gpart destroy ada5
8. Transfer the new system to the SSDs
Copy the partition table:
Code:
gpart backup da0 | gpart restore ada4 gpart backup da0 | gpart restore ada5
Attach the devices to the zpool:
Code:
zpool attach boot-pool da0p2 ada4p2 zpool attach boot-pool da0p2 ada5p2
Copy the EFI boot partition:
Code:
dd if=/dev/da0p1 of=/dev/ada4p1 dd if=/dev/da0p1 of=/dev/ada5p1
Wait for the resilver to finish! (should not take long)
Detach the USB drive from the pool:
Code:
zpool offline boot-pool da0p2 zpool detach boot-pool da0p2
Result:
Code:
zpool status boot-pool
pool: boot-pool
state: ONLINE
scan: scrub repaired 0 in 0 days 00:00:02 with 0 errors on Thu Jan 9 03:45:02 2020
config:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ada4p2 ONLINE 0 0 0
ada5p2 ONLINE 0 0 0
errors: No known data errors
gpart show ada4
=> 40 1000215136 ada4 GPT (477G)
40 2008 - free - (1.0M)
2048 524288 1 efi (256M)
526336 67108864 2 freebsd-zfs (32G)
67635200 932579976 - free - (445G)
You should have quite some free space on your SSDs, now.
9. Shutdown, unplug the USB drive, boot from your SSDs, restore services
If the system boots alright, you might want to take the time to import your main data pool and restore your configuration, now.
Apart from the smaller partition size (which the installer refuses to create) there is absolutely nothing different from a regular install. So unless some future upgrade decides to wipe and repartition the system drives during the process this setup will definitely work like any other. We had that (wipe and repartition) once in the past when FreeNAS switched from UFS to ZFS for the boot drive. There might be a situation in the future when the boot loader changes (again) - you are on your own, watch the release notes for every major upgrade!
10. Create a new pool in the available space
Again we will need the command line, when we are finished the pool will be available in the GUI like any other:
Code:
gpart add -t freebsd-zfs -a 1m ada4 gpart add -t freebsd-zfs -a 1m ada5 gpart list ada4 gpart list ada5
Look for these in the output of the
gpart list command:Code:
3. Name: ada4p3 [...] rawuuid: 25fe934a-19d6-11ea-82a1-ac1f6b76641c [...] 3. Name: ada5p3 [...] rawuuid: 3fc8e29a-19d0-11ea-9848-ac1f6b76641c
Now create the pool - use the UUIDs from the previous step:
Code:
zpool create ssd mirror gptid/25fe934a-19d6-11ea-82a1-ac1f6b76641c gptid/3fc8e29a-19d0-11ea-9848-ac1f6b76641c
Last we export the pool from the command line:
Code:
zpool export ssd
11. Import the new pool from the GUI
Done! Enjoy :)
Patrick
Edit 20210514: replace "freenas-boot" with "boot-pool" for current versions
Last edited by a moderator:
