The reason that I bring this up, is that this is something that can bring a system down quite easy... and what it even worse, is that you may start out with a mirror like I did... then if one of your drives fail and you replace it, you will not find out that your system is unable to boot... until you reboot of cause...
The system does create three partitions on both disks:
root@truenas[/]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 1M 0 part
├─sda2 8:2 0 512M 0 part
└─sda3 8:3 0 19.5G 0 part
sdb 8:16 0 20G 0 disk
├─sdb1 8:17 0 1M 0 part
├─sdb2 8:18 0 512M 0 part
└─sdb3 8:19 0 19.5G 0 part and the boot pool is ok:
NAME STATE READ WRITE CKSUM
boot-pool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sda3 ONLINE 0 0 0
sdb3 ONLINE 0 0 0I can see that / and /boot/grub is mounted from the boot-pool... which is mirrored..
root@truenas[/]# mount | grep boot
boot-pool/ROOT/22.02.RELEASE on / type zfs (rw,relatime,xattr,posixacl)
boot-pool/grub on /boot/grub type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system on /var/db/system type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/cores on /var/db/system/cores type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/samba4 on /var/db/system/samba4 type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/syslog-782595d4363048b99e75310cc20adc4f on /var/db/system/syslog-782595d4363048b99e75310cc20adc4f type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/rrd-782595d4363048b99e75310cc20adc4f on /var/db/system/rrd-782595d4363048b99e75310cc20adc4f type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/configs-782595d4363048b99e75310cc20adc4f on /var/db/system/configs-782595d4363048b99e75310cc20adc4f type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/webui on /var/db/system/webui type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/services on /var/db/system/services type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/glusterd on /var/db/system/glusterd type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/ctdb_shared_vol on /var/db/system/ctdb_shared_vol type zfs (rw,relatime,xattr,posixacl)
boot-pool/.system/cores on /var/lib/systemd/coredump type zfs (rw,relatime,xattr,posixacl)So my guess it that the boot-block/signature/whatever is not written to the replaced disk, which leaves the system unbootable.
As mentioned, this can be replicated by just creating a VM with two disks, installing TrueNAS Scale onto the two disks, and then simulate a disk failure of the first disk (sda). With the disk gone, the VM can boot from sdb, but as you do a replacement with a new virtual disk and do a replace on the boot-pool... it resilvers OK, (it then fails with the error described in my first post).. but even if I re-run the grub-install manually (which completed OK)... it is now unable to boot of sda.
Is it just a grub command that needs to be executed on the replaced disk in order to make it bootable?
Would be very nice if this just worked out of the box... is that too much to ask? Is it hard or complicated?
Might just end up mirroring the boot disks with a server RAID card if this doesn't work
/B
