Broken boot mirror after migrating to Truenas Scale

Broken boot mirror after migrating to Truenas Scale

A few months ago I jumped from Truenas Core 12 to Truenas Scale 22.02. The upgrade was supported at a button press, and has largely gone well despite my misgivings over using Kubernetes for home server tasks.

But I’m now encountering an unexpected rough edge: The boot environment cannot be healed after losing a disk in the 2-disk mirror pool. At first I thought this was a more general issue of disk size discrepancies that I’ve whined about previously. Instead it wound up being that same problem (not enough space on an identical disk to resilver) but created intentionally.

Here is my remaining, working boot disk:

[email protected]:~# fdisk -l /dev/sdm
Disk /dev/sdm: 111.79 GiB, 120034123776 bytes, 234441648 sectors
Disk model: CT120BX500SSD1  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: F51A048E-C72D-11EB-9F7B-F04DA2301444

Device     Start       End   Sectors   Size Type
/dev/sdm1     40      1063      1024   512K BIOS boot
/dev/sdm2   1064 234441607 234440544 111.8G FreeBSD ZFS

And this is what Truenas attempted to do with the new disk being swapped in to replace a failed one:

[email protected]:~# fdisk -l /dev/sdn
Disk /dev/sdn: 111.79 GiB, 120034123776 bytes, 234441648 sectors
Disk model: CT120BX500SSD1  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: E8A71C3D-4A58-45FB-91EB-EB0FB52432BA

Device       Start       End   Sectors   Size Type
/dev/sdn1       40      2087      2048     1M BIOS boot
/dev/sdn2     2088   1050663   1048576   512M EFI System
/dev/sdn3  1050664 234441614 233390951 111.3G Solaris /usr & Apple ZFS

So the system, post upgrade, will not be able to recover a boot mirror because it is attempting to create a partition scheme not used previously.

https://www.truenas.com/community/threads/migrating-to-scale-breaks-the-boot-pool.103340/#post-711129