How do I change a hard drive's logical sector size from 512 bytes to 4096 bytes?

[Daisuke]

The same disclaimer about "be careful not to destroy anything" applies.

I hear you. I'm going to take one disk offline from pool and reboot from that ISO mounted as virtual CD. I'll let everyone know if I made progress...

Should I run wipefs or anything fancy before, or just hdparm --set-sector-size 4096 --please-destroy-my-drive /dev/sda will suffice?

 

[artlessknave]

I never used Ubuntu, they don't provide a true Live CD so I'll be stuck downloading the packages when I mount their ISO on a Virtual CD.

downloading the packages is usually faster than loading a virtual CD though.....is it not?
I've never tested it. i use any network install over downloading a whole CD of old packages....

 

[winnielinnie]

Should I run wipefs or anything fancy before, or just hdparm --set-sector-size 4096 --please-destroy-my-drive /dev/sda will suffice?

Not sure if you need to wipe the drive first, as I believe reformatting the sector-size essentially does that anyways. (I've never tried to do what you're doing, as I've always left my drives at 4K/512e.) Someone else with more experience might be able to chime in.

However, there is one thing that makes me very uncomfortable with the above command:

/dev/sda

I would only ever try that if I physically unplug all the other drives, and make sure 100% that the sole remaining drive (still plugged in) is in fact the one I want to format.

But that's just me. I'm slow as a turtle, and just as safe.

 

[Daisuke]

But that's just me.

No worries, I'll be a guinea pig! I use the first option, right?

I was thinking to use this option:

 

[winnielinnie]

That option works too.

I don't like using the "copy system to RAM" option, since it takes longer to bootup.

 

[Daisuke]

That option works too.

I'll pick the first option. I'm going to remove all disks and leave only the one I want to test. I had to do this anyways to get the serial number for each slot, finally a good reason to do it! Will report back in an hour if I managed to to get the format starting. This is going to be a long ride...

The good thing is I'm directly connected to console, so I will not have any network surprises.

 

[winnielinnie]

Will report back in an hour if I managed to to get the format starting. This is going to be a long ride...

I'm reading around online about others trying to use sg_format and hdparm to set the sector-size of their drives, and it looks like a mixed bag out there.

 

[Daisuke]

I'm reading around online about others trying to use sg_format and hdparm

sg_format fails:



hdparm has the same result:



It has to be a way to format these babies, seriously.

 

[winnielinnie]

hdparm has the same result:

Does hdparm -I /dev/sdb correctly read the drive?

 

[Daisuke]

Does hdparm -I /dev/sdb correctly read the drive?

Yes, I get the same results like hdparm 9.60. I put back the drives and I'm rebooting back to Scale. /dev/sda is the Scale boot ssd.

 

[winnielinnie]

Perhaps the drive just doesn't support it?

Apparently (and it's a bit late now), but using --verbose would have likely revealed more information about the error.

 

[Daisuke]

but using --verbose would have likely revealed more information about the error.

Code:

# hdparm --verbose --set-sector-size 4096 --please-destroy-my-drive /dev/sda

/dev/sda:
APT: No idVendor found -> not USB bridge device
outgoing cdb:  85 08 0e 00 00 00 01 00 00 00 00 00 00 40 ec 00
SG_IO: ATA_16 status=0x0, host_status=0x0, driver_status=0x0
SG_IO: sb[]:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
incoming_data:  5a 04 ff 3f 37 c8 10 00 00 00 00 00 3f 00 00 00 00 00 00 00 4b 56 55 48 4a 53 58 4c 20 20 20 20 20 20 20 20 20 20 20 20 03 00 00 00 38 00 34 41 4c 47 30 30 33 30 55 48 37 48 38 32 38 30 41 30 45 4c 30 36 20 31 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 10 80 01 40 00 2f 00 40 00 02 00 02 07 00 ff 3f 10 00 3f 00 10 fc fb 00 00 71 ff ff ff 0f 00 00 07 00 03 00 78 00 78 00 78 00 78 00 18 0c 00 00 00 00 00 00 00 00 00 00 1f 00 0e 97 76 00 de 0c 44 00 fc 03 29 00 6b 74 69 7d 73 47 69 74 41 bc 73 47 7f 40 ed 81 01 80 fe 00 fd ff 00 00 00 00 08 00 ca 00 f9 00 10 27 00 00 b0 2a 81 a3 03 00 00 00 ca 00 00 00 03 60 87 5a 00 50 a2 cc d9 54 21 cd 00 00 00 00 00 00 00 00 00 00 00 00 00 00 dc 40 1c 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 21 00 0b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 45 58 30 32 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 07 00 4c 47 00 00 00 00 0e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 3d 00 00 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20 1c 00 00 00 00 00 00 00 00 7f 10 21 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 2a 81 a3 03 00 00 00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 a5 27
SG_IO: desc[]:  00 00
      ATA_16 stat=00 err=00 nsect=00 lbal=00 lbam=00 lbah=00 dev=00
oflags.bits.lob_all=0xfc, flags={ nsect lbal lbam lbah dev command }
oflags.bits.hob_all=0x3c, flags={ nsect lbal lbam lbah }
using LBA48 taskfile
outgoing cdb:  85 09 0e 00 00 00 01 00 00 00 00 00 00 e0 2f 00
SG_IO: ATA_16 status=0x0, host_status=0x0, driver_status=0x0
SG_IO: sb[]:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
incoming_data:  01 00 00 00 00 00 01 00 08 00 00 00 00 00 01 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 01 00 01 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 00 00 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 09 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 02 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
SG_IO: desc[]:  00 00
      ATA_16 stat=00 err=00 nsect=00 lbal=00 lbam=00 lbah=00 dev=00
READ_LOG_EXT(SECTOR_CONFIGURATION) failed: No such file or directory

 

[winnielinnie]

No additional useful information. But...

...according to another user on ServeTheHome, the "No such file or directory" message is a poorly written aesthetic bug, when in fact the drive's firmware simply does not support the specific low-level ATA command to set the sector size.

 

[Daisuke]

No additional useful information.

I'm into something, there is a WD utility called Hugo designed for these enterprise drives. I'm in the process of setting a virtual drive with all required files to run it. I'll let you know if I make progress.

 

[thalf]

OP here. I've been planning to post an update on what happened to my 20TB WD Red drives, and now seems a good time.

I couldn't get the sg_format (or hdparm when booting to Linux) commands to work, and I couldn't find a download link to WD's "Hugo" tool, so I contacted WD support and they sent me a download link for WDC Kit which, among many other things, can convert to 4Kn sectors. (If you contact them make sure to specify what you're running; on first attempt I got a Windows binary. And don't delay downloading, the link expires after a while (days/weeks, not hours).)

Before converting to 4Kn I had run a long S.M.A.R.T. test on both disks, and jgreco's solnet-array-test-v2.sh script. Full output from the solnet-array-test-v2.sh run:

Code:

# sh ./solnet-array-test-v2.sh
sol.net disk array test v2

1) Use all disks (from camcontrol)
2) Use selected disks (from camcontrol|grep)
3) Specify disks
4) Show camcontrol list

Option: 2

Enter grep match pattern (e.g. ST150176): WD201KFGX

Selected disks: ada1 ada2
<WDC WD201KFGX-68BKJN0 83.00A83>   at scbus1 target 0 lun 0 (pass1,ada1)
<WDC WD201KFGX-68BKJN0 83.00A83>   at scbus2 target 0 lun 0 (pass2,ada2)
Is this correct? (y/N): y
Performing initial serial array read (baseline speeds)
Sun Aug 28 15:29:46 CEST 2022
Sun Aug 28 15:34:16 CEST 2022
Completed: initial serial array read (baseline speeds)

Array's average speed is 268.32 MB/sec per disk

Disk    Disk Size  MB/sec %ofAvg
------- ---------- ------ ------
ada1    19074048MB    268    100
ada2    19074048MB    268    100

Performing initial parallel array read
Sun Aug 28 15:34:16 CEST 2022
The disk ada1 appears to be 19074048 MB.
Disk is reading at about 268 MB/sec
This suggests that this pass may take around 1187 minutes

                   Serial Parall % of
Disk    Disk Size  MB/sec MB/sec Serial
------- ---------- ------ ------ ------
ada1    19074048MB    268    269    100
ada2    19074048MB    268    268    100

Awaiting completion: initial parallel array read
Mon Aug 29 15:47:47 CEST 2022
Completed: initial parallel array read

Disk's average time is 86517 seconds per disk

Disk    Bytes Transferred Seconds %ofAvg
------- ----------------- ------- ------
ada1       20000588955648   85823     99
ada2       20000588955648   87212    101

Performing initial parallel seek-stress array read
Mon Aug 29 15:47:47 CEST 2022
The disk ada1 appears to be 19074048 MB.
Disk is reading at about 222 MB/sec
This suggests that this pass may take around 1434 minutes

                   Serial Parall % of
Disk    Disk Size  MB/sec MB/sec Serial
------- ---------- ------ ------ ------
ada1    19074048MB    268    222     83
ada2    19074048MB    268    221     82

Awaiting completion: initial parallel seek-stress array read
Mon Sep  5 22:19:18 CEST 2022
Completed: initial parallel seek-stress array read

Disk's average time is 580556 seconds per disk

Disk    Bytes Transferred Seconds %ofAvg
------- ----------------- ------- ------
ada1       20000588955648  536370     92
ada2       20000588955648  624742    108 --SLOW--

After another long S.M.A.R.T. test, I ran the WDC Kit tool to convert my two 20TB drives to 4Kn. Converting took almost 25 hours (I did them in parallel in two windows in a screen session). Output:

Code:

# ./wdckit format /dev/ada1 -b 4096
wdckit Version 2.14.0.0
Copyright (C) 2019-2022 Western Digital Technologies, Inc.
Western Digital ATA/SCSI/NVMe command line utility.
11/05/2022 16:14:41

Format on 1 device(s) started...
Progress: 100%
Success: Format completed on: [redacted]
/dev/ada1: Success


# time ./wdckit format /dev/ada2 -b 4096
wdckit Version 2.14.0.0
Copyright (C) 2019-2022 Western Digital Technologies, Inc.
Western Digital ATA/SCSI/NVMe command line utility.
11/05/2022 16:34:48

Format on 1 device(s) started...
Progress: 100%
Success: Format completed on: [redacted]
/dev/ada2: Success


130.815u 65.608s 24:41:38.45 0.2%       5268+462k 0+7458119io 0pf+0w

After converting to 4Kn sectors I powered off the nas, left it off a couple of minutes, then powered on again, and ran another long S.M.A.R.T. test and then the solnet-array-test-v2.sh script again. Full output from solnet-array-test-v2.sh:

Code:

# sh ./solnet-array-test-v2.sh
sol.net disk array test v2

1) Use all disks (from camcontrol)
2) Use selected disks (from camcontrol|grep)
3) Specify disks
4) Show camcontrol list

Option: 2

Enter grep match pattern (e.g. ST150176): WD201KFGX-68BKJN0

Selected disks: ada1 ada2
<WDC WD201KFGX-68BKJN0 83.00A83>   at scbus1 target 0 lun 0 (pass1,ada1)
<WDC WD201KFGX-68BKJN0 83.00A83>   at scbus2 target 0 lun 0 (pass2,ada2)
Is this correct? (y/N): y
Performing initial serial array read (baseline speeds)
Tue Nov  8 07:41:53 CET 2022
Tue Nov  8 07:46:23 CET 2022
Completed: initial serial array read (baseline speeds)

Array's average speed is 267.92 MB/sec per disk

Disk    Disk Size  MB/sec %ofAvg
------- ---------- ------ ------
ada1    19074048MB    267    100
ada2    19074048MB    268    100

Performing initial parallel array read
Tue Nov  8 07:46:23 CET 2022
The disk ada1 appears to be 19074048 MB.
Disk is reading at about 268 MB/sec
This suggests that this pass may take around 1185 minutes

                   Serial Parall % of
Disk    Disk Size  MB/sec MB/sec Serial
------- ---------- ------ ------ ------
ada1    19074048MB    267    268    100
ada2    19074048MB    268    268    100

Awaiting completion: initial parallel array read
Wed Nov  9 07:59:54 CET 2022
Completed: initial parallel array read

Disk's average time is 86517 seconds per disk

Disk    Bytes Transferred Seconds %ofAvg
------- ----------------- ------- ------
ada1       20000588955648   85824     99
ada2       20000588955648   87211    101

Performing initial parallel seek-stress array read
Wed Nov  9 07:59:54 CET 2022
The disk ada1 appears to be 19074048 MB.
Disk is reading at about 221 MB/sec
This suggests that this pass may take around 1437 minutes

                   Serial Parall % of
Disk    Disk Size  MB/sec MB/sec Serial
------- ---------- ------ ------ ------
ada1    19074048MB    267    222     83
ada2    19074048MB    268    221     82

Awaiting completion: initial parallel seek-stress array read
Wed Nov 16 10:31:32 CET 2022
Completed: initial parallel seek-stress array read

Disk's average time is 478491 seconds per disk

Disk    Bytes Transferred Seconds %ofAvg
------- ----------------- ------- ------
ada1       20000588955648  491168    103
ada2       20000588955648  465814     97

Then I ran a final long S.M.A.R.T. test.

After that I added the two 20TB drives to my pool (that already consisted of two 18TB drives; one Seagate ST18000NM000J and one Toshiba MG09ACA18TE). I don't remember, but I think I added them one at a time, and I don't remember how long it took to resilver after adding.

...and then I haven't gotten further. I've been planning since mid-november to shutdown and disconnect the two 18TB drives to check acoustic levels (which was my original reason for diving down this rabbit hole and buying too many expensive drives for a home user just to try them out) but haven't gotten around to it yet. Perhaps some evening this week, or this coming weekend, or during the easter holiday... no later than my summer vacation, though.

Looking at the numbers from the two solnet-array-test-v2.sh runs above, it appears that 4Kn is slightly faster. The load on the system was the same (i.e. virtually none) during the two runs so I think it's fair to compare the numbers. (As an aside, the time estimates in solnet-array-test-v2.sh are way way off; my guess is that they're for one run of dd, but the script runs six tests (dd's) in parallel.)

I've had no warnings or experienced any problems after adding my two 20TB 4Kn drives to the pool, despite the two existing 18TB drives in the pool being 4K/512e. So mixing 4Kn and 4K/512e appears to work fine (as winnielinnie wrote), at least with ashift=12.

(Edit: grammar and formatting.)

 

[Daisuke]

@winnielinnie I'm making great progress with Hugo. I created a resource with the software and also I added the HGST Ultrastar Firmware section into my guide. @thalf, with my procedure, I can format one disk in 30 seconds, then resilver it in 4 hours. I do this directly on my Scale server one disk at the time. Like @winnielinnie said very well, people should know what they are doing. The only scary moment was when I ran fdisk after first format, it was returning:

Code:

# wipefs -af /dev/sda
# hugo format --danger-zone --fastformat --simple-progress -b 4096 -g /dev/sda
Format device on 1 Device(s)...
Format on 1 device(s) started...
Progress: 100%
Success: Format completed on: [removed]

# fdisk -l /dev/sda
fdisk: cannot open /dev/sda: Input/output error

Then I realized I need to initialize the disk with a sg_format read, to make it visible to OS:

Code:

# sg_format -v /dev/sda
# fdisk -l /dev/sda
Disk /dev/sda: 7.28 TiB, 8001563222016 bytes, 1953506646 sectors
Disk model: HUH728080ALE601
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

You need to run wipefs first, otherwise Hugo will not like you and will keep the 512e logical volume in place, when you do a fast format. wipefs is also instant, Hugo probably does not understand the zfs format. All this is explained into guide.

Another thing I noticed, with S.M.A.R.T. enabled into disk options, Scale will throw constant errors and emails about your disk being locked, so you need to disable that into disk options, before you start the format process on that particular disk.

 

[winnielinnie]

Glad you got it working and shared how you did it!

I just don't understand why these hard drive manufacturers make you go through this "song and dance" to enable a practical feature of their hardware, when it could be as simple as this...