Thanks to everyone who participated in this thread, it saved my bacon: I had 10 drives with
prot_en=1
, had no idea it was even an issue.
I had been using the drives with zfs on another distro, but repurposed them for a friend's restaurant, where I wanted them to have something more user-friendly and comprehensive, so TrueNAS Scale seems like a great fit.
Big picture question: Why is TrueNAS Scale concerned about DIF protection bits being set on the drives now? I'm sure I've used these same drives with Core a few years ago and it never complained - is this specific to TrueNAS, or newer versions of zfs? (feel free to reply with any reference I overlooked).
I suppose I should share my experience with
sg_format -vF $DEV_PATH
in case it helps someone. Overall it was very positive, but all I was doing was removing the prot_en=1 flag. Each took an excrutiating 7 1/2 hours per 4TB drive, but formatting all 10 in parallel had max 10-15 min deviation in period (aka
=< %0.03
), so must not be all that CPU or IO heavy.
I was learning it as I went, so I ended up doing it over 10 ssh sessions via wifi: It was pins and needles worrying my laptop would get booted from the network and kill the process right at 99% - recommend other people ignore my poor example and connect via ethernet. Thankfully, It all worked out.
According to sg_format -v, all that changed was this:
Code:
$ diff -u dev-sdj-start.txt dev-sdj-end.txt
--- dev-sdj-start.txt 2024-01-14 14:46:56.047681276 -0800
+++ dev-sdj-end.txt 2024-01-14 15:06:31.642292021 -0800
@@ -1,4 +1,4 @@
-sg_format -v /dev/sdj
+sg_format -v /dev/sdj
HGST H7240AS60SUN4.0T A2D2 peripheral_type: disk [0x0]
PROTECT=1
<< supports protection information>>
@@ -13,7 +13,7 @@
Block size=512 [0x200]
read capacity(16) cdb: [9e 10 00 00 00 00 00 00 00 00 00 00 00 20 00 00]
Read Capacity (16) results:
- Protection: prot_en=1, p_type=0, p_i_exponent=0
+ Protection: prot_en=0, p_type=0, p_i_exponent=0
Logical block provisioning: lbpme=0, lbprz=0
Logical blocks per physical block exponent=0
Lowest aligned logical block address=0
That was taken before and after the format. TrueNAS web UI let me make a pool after format, while before the process halted and exited due to
DIF
.
Here's a full output of
sg_format -v
on one of the drives after format:
Code:
# sg_format -v /dev/sda
HGST H7240AS60SUN4.0T A1B2 peripheral_type: disk [0x0]
PROTECT=1
<< supports protection information>>
Unit serial number: 001342E4THXX PBG4THXX
LU name: 5000cca03b08b604
mode sense(10) cdb: [5a 00 01 00 00 00 00 00 fc 00]
Mode Sense (block descriptor) data, prior to changes:
block count maxed out, set <<longlba>>
mode sense(10) cdb: [5a 10 01 00 00 00 00 00 fc 00]
<<< longlba flag set (64 bit lba) >>>
Number of blocks=7814037168 [0x1d1c0beb0]
Block size=512 [0x200]
read capacity(16) cdb: [9e 10 00 00 00 00 00 00 00 00 00 00 00 20 00 00]
Read Capacity (16) results:
Protection: prot_en=0, p_type=0, p_i_exponent=0
Logical block provisioning: lbpme=0, lbprz=0
Logical blocks per physical block exponent=0
Lowest aligned logical block address=0
Number of logical blocks=7814037168
Logical block size=512 bytes
No changes made. To format use '--format'. To resize use '--resize
I wasn't really sure what I was looking for in the above, but the
man page for sg_format helped narrow it down. There's more people's experiences on
servethehome forums if someone's still struggling by the end of this thread.
Tangentially, I learned bash can iterate alphabetically in
for
loops, so that is awesome, e.g serial and lun printing one-liner.
Code:
# for d in {a..z}; do echo $d; sg_format -v /dev/sd$d; grep -E 'serial|lun'; echo ' '; done >> luns-and-serials.txt
Aside: Anyone who's better at bash than I am know if the above could exit each
sg_format
invocation with
&
so a single script could be used to run command in parallel? e.g.
... sg_format -vF /dev/sd$d &; ...
<next disk>?
Thanks again!