How to convert 512e to 4Kn using Fast Format (Seagate Exos X16 drive) ?

[dogwhistle]

thanks for this info. it helped me out.

nothing much to add, but i'll share my X18 experience.

i have some exos 18TB X18 SATA drives (model: ST18000NM000J).

go here

Seagate Technology - Download Finder

apps1.seagate.com

put your serial(s) and download the fw update package. in my case it was for SN02 fw: ExosX18-EvansBP-STD-SATA-5xxE-SN02.zip

i will refer to this as "the zip".

although it includes a command line tools dir, SeaChest_Format was unfortunately not included (they only give the SeaChest_Firmware util). you need this to switch to 4kN.

the easiest way to get it is to run the bootable tools usb bootmaker (SeaChest_RC_3.0.2_05-21-2021__14-15-15.usbBootMaker.exe, included in the zip) from a windows machine.
after you make that, you can either copy the fw file (EvansBPExosX18SATA-STD-5xxE-SN02.LOD) to the usb drive and boot it. or you can copy the command line utils folder (includes both the fw and format utils) off of the drive and use it directly from linux, which is what i did.

disturbingly, the readme pdf that comes in the zip includes says:

Note: The firmware above [SN02] does NOT support SAS, SED or any other default sector size other than 5xxE.
...
5xxE Advanced Format (512 byte emulation from 4k physical)

wtf? the datasheet clearly says this drive model can do 4kN. i was a little worried that i woudln't be able to do 4kN if i updated the fw, but it was fine.

to be safe though, i formatted to 4kN and then upgraded the fw on one drive as a test. then i did the same for the rest.

some commands to get you started:

Code:

# -----formatting-----

# list your drives. note that the devices are named differently like sg2 instead of sdb, so refer to this output.
sudo ./SeaChest_Format_x86_64-redhat-linux --scan

# check a drive's supported formats
sudo ./SeaChest_Format_x86_64-redhat-linux -d /dev/sg2 --showSupportedFormats
# you could also try
sudo ./SeaChest_Format_x86_64-redhat-linux -d all --showSupportedFormats

# if it says it can do 4096, then proceed on one drive as a test:
sudo ./SeaChest_Format_x86_64-redhat-linux -d /dev/sg2 --setSectorSize=4096 --confirm this-will-erase-data
# confirm the change worked:
sudo ./SeaChest_Format_x86_64-redhat-linux -d /dev/sg2 --showSupportedFormats

# if that worked, then do it for the rest of your drives. for convenience, you can do a bulk action based on model number:
sudo ./SeaChest_Format_x86_64-redhat-linux -d all --modelMatch ST18000NM000J --setSectorSize=4096 --confirm this-will-erase-data

# -----firmware upgrade-----

# list your drives. note that the devices are named differently like sg2 instead of sdb, so refer to this output.
sudo ./SeaChest_Firmware_x86_64-redhat-linux --scan

# upgrade one drive first as a test
sudo ./SeaChest_Firmware_x86_64-redhat-linux -d /dev/sg2 --downloadFW EvansBPExosX18SATA-STD-5xxE-SN02.LOD

# if that worked, then you can do the rest in a batch (select by model)
sudo ./SeaChest_Firmware_x86_64-redhat-linux -d all --modelMatch ST18000NM000J --downloadFW EvansBPExosX18SATA-STD-5xxE-SN02.LOD

and there you have it. all 4kN and updated to SN02 even though the release notes say SN02 doesn't support 4kN. probably a mistake in the documentation.

 

[diskdiddler]

What are the implications of not doing this and leaving the drives as default?

This has me particularly concerned:

How to convert 512e to 4Kn using Fast Format (Seagate Exos X16 drive) ?

I'm planning to purchase some Seagate Exos X16 (model ST16000NM001G) 16TB drives. They come formatted in 512e by default, but they support "Fast Format" to convert to 4Kn so that they appear as a true 4Kn to the OS. This is documented in the Seagate documentation, but they neglect to say how you...

www.truenas.com

 

[diskdiddler]

thanks for this info. it helped me out.

nothing much to add, but i'll share my X18 experience.

i have some exos 18TB X18 SATA drives (model: ST18000NM000J).

go here

Seagate Technology - Download Finder

apps1.seagate.com

put your serial(s) and download the fw update package. in my case it was for SN02 fw: ExosX18-EvansBP-STD-SATA-5xxE-SN02.zip

i will refer to this as "the zip".

although it includes a command line tools dir, SeaChest_Format was unfortunately not included (they only give the SeaChest_Firmware util). you need this to switch to 4kN.

the easiest way to get it is to run the bootable tools usb bootmaker (SeaChest_RC_3.0.2_05-21-2021__14-15-15.usbBootMaker.exe, included in the zip) from a windows machine.
after you make that, you can either copy the fw file (EvansBPExosX18SATA-STD-5xxE-SN02.LOD) to the usb drive and boot it. or you can copy the command line utils folder (includes both the fw and format utils) off of the drive and use it directly from linux, which is what i did.

disturbingly, the readme pdf that comes in the zip includes says:



wtf? the datasheet clearly says this drive model can do 4kN. i was a little worried that i woudln't be able to do 4kN if i updated the fw, but it was fine.

to be safe though, i formatted to 4kN and then upgraded the fw on one drive as a test. then i did the same for the rest.

some commands to get you started:

So format to 4kN BEFORE firmware change?

 

[Apollo]

So format to 4kN BEFORE firmware change?

Do firmware update first, then change to 4KN and then cycle power to HDD to see the change.

Hard Drive Burn-In Testing - Discussion Thread

/incoming appears to be empty on that FTP, do you know somewhere else to get the script? Don't bother with Chrome or Firefox, even if you fix the URL. They dropped FTP support. Quickest way to grab the file (directly): wget ftp://ftp.sol.net/incoming/solnet-array-test-v2.sh ✅ The above...

www.truenas.com

 

[diskdiddler]

Do firmware update first, then change to 4KN and then cycle power to HDD to see the change.

Hard Drive Burn-In Testing - Discussion Thread

/incoming appears to be empty on that FTP, do you know somewhere else to get the script? Don't bother with Chrome or Firefox, even if you fix the URL. They dropped FTP support. Quickest way to grab the file (directly): wget ftp://ftp.sol.net/incoming/solnet-array-test-v2.sh ✅ The above...

www.truenas.com

What are the implications of not doing it? There's a guy here reporting 2 drives died in changing it.

broken HDD after format change from 512 to 4096 4kn

Hi all, I used the SeaChest-Software on Linux system to fastformat from 512 to 4kn on a new Seagate Exos X16 10TB HDD. The topic was working for 2v4 brand new HDDs. But now 2 HDD seem to be fully broken. Following commands I used: SeaChest_Lite --scan SeaChest_Lite --device /dev/sg...

www.truenas.com

AND this issue:

Exos x16 - High head cycle count due to toggle between EPC idle_a and idle_b power states

I'm currently trying to build a new Storage Server based on TrueNas and having some trouble because the "load_cycle_count" of the drives gets increased every 3 minutes, which would lead to a load_cycle_count of 480 per day or 175200 per year. I don't think this is intended behaviour. But first...

www.truenas.com

So I'm kind of concerned here truth be told

 

[Apollo]

What are the implications of not doing it? There's a guy here reporting 2 drives died in changing it.

broken HDD after format change from 512 to 4096 4kn

Hi all, I used the SeaChest-Software on Linux system to fastformat from 512 to 4kn on a new Seagate Exos X16 10TB HDD. The topic was working for 2v4 brand new HDDs. But now 2 HDD seem to be fully broken. Following commands I used: SeaChest_Lite --scan SeaChest_Lite --device /dev/sg...

www.truenas.com

AND this issue:

Exos x16 - High head cycle count due to toggle between EPC idle_a and idle_b power states

I'm currently trying to build a new Storage Server based on TrueNas and having some trouble because the "load_cycle_count" of the drives gets increased every 3 minutes, which would lead to a load_cycle_count of 480 per day or 175200 per year. I don't think this is intended behaviour. But first...

www.truenas.com

So I'm kind of concerned here truth be told

I think, in the case of the so called broken disk, reverting to 512e would have fixed those drives. If HDD is writing to it it might corrupt some data.
Formatting the drives would also take care of it.
I have converted 4 of mine and I haven't seen any issues yet. They are currently part of my backup replication pool.

512e to 4Kn doesn't seem to be a problem and mixing 512e and 4Kn formatted disk within the same vdev didn't bother ZFS whatsoever.
ZFS does store the data as 4K block. I also didn't see any meaningful performance degradation or improvement.

 

[diskdiddler]

I think, in the case of the so called broken disk, reverting to 512e would have fixed those drives. If HDD is writing to it it might corrupt some data.
Formatting the drives would also take care of it.
I have converted 4 of mine and I haven't seen any issues yet. They are currently part of my backup replication pool.

512e to 4Kn doesn't seem to be a problem and mixing 512e and 4Kn formatted disk within the same vdev didn't bother ZFS whatsoever.
ZFS does store the data as 4K block. I also didn't see any meaningful performance degradation or improvement.

The problem is, if that guy is using TrueNAS- surely converting all 4 disks, killed all 4 for the pool?
His post seems to suggest 2 of them are 'dead' - loss of data is no big deal at all, because I haven't deployed them yet.

Which model do you have and can I ask are you suffering with this issue?

Exos x16 - High head cycle count due to toggle between EPC idle_a and idle_b power states

I'm currently trying to build a new Storage Server based on TrueNas and having some trouble because the "load_cycle_count" of the drives gets increased every 3 minutes, which would lead to a load_cycle_count of 480 per day or 175200 per year. I don't think this is intended behaviour. But first...

www.truenas.com

 

[Apollo]

The problem is, if that guy is using TrueNAS- surely converting all 4 disks, killed all 4 for the pool?
His post seems to suggest 2 of them are 'dead' - loss of data is no big deal at all, because I haven't deployed them yet.

Which model do you have and can I ask are you suffering with this issue?

Exos x16 - High head cycle count due to toggle between EPC idle_a and idle_b power states

I'm currently trying to build a new Storage Server based on TrueNas and having some trouble because the "load_cycle_count" of the drives gets increased every 3 minutes, which would lead to a load_cycle_count of 480 per day or 175200 per year. I don't think this is intended behaviour. But first...

www.truenas.com

You don't want to change format with disk containing live data and expect to reinsert it in the pool and hope your data is still safe. It might, but most likely may not (translation is done at the controller level and maybe just as transparent).
Only SATA communication will see a less efficient throughout due to overhead.

My EXOS X18 18TB are not currently connected to my backup replication system as I am doing replication to another set of drives.

I don't have the numbers on head cycle count but I didn't seems to see anything outstanding in that area.

 

[NugentS]

I used a boot USB to boot Seagate's tools and formatted the drives to 4Kn.
I have since formatted them back as some of my somewhat random collection of spares won't do 4Kn and you can't mix 4Kn and 512e in a vdev
BUt I didn't break anything, and upgraded the firmware whilst at it

 

[diskdiddler]

I used a boot USB to boot Seagate's tools and formatted the drives to 4Kn.
I have since formatted them back as some of my somewhat random collection of spares won't do 4Kn and you can't mix 4Kn and 512e in a vdev
BUt I didn't break anything, and upgraded the firmware whilst at it

Were all the disks Seagate?
I think the 16TB Exos def does 4Kn based on spec sheets, it just (possibly) ships in 512e mode (??)

I believe I'm buying all of the same model, ST16000NM001G so fingers crossed.

 

[NugentS]

All the ones I formatted to 4Kn were Seagate
I cannot format the Toshibas to 4Kn and the Hitachi (old drives) flat out don't seem to support it

 

[dogwhistle]

So format to 4kN BEFORE firmware change?

the documentation for my drives' fw update said 4kN was not supported (or rather that only 512e was supported). this worried me, but i figured it was probably an error in the documentation.

so on a single test drive, first i formatted to 4kN because the util reported that was a valid format option for it.
then i upgraded the fw.
and after noting that the drive was working fine with new fw and 4kN, i repeated this same procedure for the rest of the drives just because it worked. i.e. i formatted to 4kN and then upgraded the fw.

updating the fw first probably would have also worked fine.

 

[dogwhistle]

What are the implications of not doing this and leaving the drives as default?

you could read this for some insight:

https://kb.vmware.com/s/article/2091600#:~:text=4Kn%20is%20the%20advanced%20format,emulates%20512%20bytes%20sector%20size.

basically, 4kN could be more performant, but many logical 4k drives ship with 512 emulation mode for compatibility reasons with legacy crap.
idk what actually needs 512e. all my stuff works with 4kN just fine.

 

[dogwhistle]

also keep in mind that formatting to 4kN probably necessitates wiping them, so if you already have data on them, maybe it's better to leave them as 512e if you don't have a place to temporarily move the data.

as for concerns about bricking drives by upgrading fw or formatting... i had the same concerns, but after reading this thread in its entirety, it sounded like the problems happened from using the wrong utils or fw images. putting your serial into the seagate page linked repeatedly in this thread seems to not produce issues because it will give you the right stuff to use.

 

[diskdiddler]

the documentation for my drives' fw update said 4kN was not supported (or rather that only 512e was supported). this worried me, but i figured it was probably an error in the documentation.

so on a single test drive, first i formatted to 4kN because the util reported that was a valid format option for it.
then i upgraded the fw.
and after noting that the drive was working fine with new fw and 4kN, i repeated this same procedure for the rest of the drives just because it worked. i.e. i formatted to 4kN and then upgraded the fw.

updating the fw first probably would have also worked fine.

Are you using windows or Linux tools to identify if the disk is currently in 4kn or 512e?

I should probably grab all these tools and prepare before my 8 drives arrive.

I'm also really hoping that somehow Seagate continues to honour the 5 year warranty, from the refurb drives. Arguably they should if the manufacture date is less than 5 years ago

 

[dogwhistle]

Are you using windows or Linux tools to identify if the disk is currently in 4kn or 512e?

I should probably grab all these tools and prepare before my 8 drives arrive.

I'm also really hoping that somehow Seagate continues to honour the 5 year warranty, from the refurb drives. Arguably they should if the manufacture date is less than 5 years ago

i used the linux tools

read my original post carefully ;p

as for warranty, are you aware that they have a warranty checker tool? put in your serial, and it shows you the remaining warranty.

 

[diskdiddler]

i used the linux tools

read my original post carefully ;p

as for warranty, are you aware that they have a warranty checker tool? put in your serial, and it shows you the remaining warranty.

Thanks for the ongoing replies. I am just panicking and talking aloud - I know there's a serial checker, I won't know until one of the disks actually arrives.
According to Amazon feedback, I'm expecting up to 2 of them to be dead and that's ok with me, I allotted for that by purchasing 8.

I do know a bit about linux, not a lot - but I'll set up my old TrueNAS box with Windows and Linux dual boot, in preparation for manage the disks for testing for a week.

Is ubuntu 22.04 fine?

 

[Apollo]

@diskdiddler Just follow instruction from Seagate on EXOS update and you will be fine.
Don't trust Amazon regardless about meaningless info they provide.
You don't need linux, just make a bootable drive (most likely USB), which is going to setup the environment necessary for the firmware update.
If you search the Seagate website about sector format on EXOS drive, it will be given to you with all the bells and whistles.

 

[diskdiddler]

smartctl -i /dev/adax will tell you. Look for the line that states Sector Sizes. For example, on my WD Red /dev/ada1:

Code:

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD40EFRX-68N32N0
Serial Number:    WD-WCC7K7DDVSL7
LU WWN Device Id: 5 0014ee 21132bb0c
Firmware Version: 82.00A82
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Jun  8 20:20:59 2020 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

I feel like something is wildly wrong with my system, I've never seen this before.

Firstly:
root@truenas[~]# smartctl -i /dev/ada0
ada0% ada0p1% ada0p2%

My autocomplete per drive, is coming back with an odd % at the tail, I've never seen this before?


Secondly, all 3 of them will report the following:

/dev/ada0p1: Unable to detect device type
Please specify device type with the -d option.


Finally:
-d auto and -d ata don't work

Read Device Identity failed: Inappropriate ioctl for device
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.



Is this normal? These are plain old 8TB SATA disks that are actually working fine. I could've sworn I've issued similar commands before, without an issue of specifying device type, is this a 13 change or my system being rude?

 

[Patrick M. Hausen]

ada0p1 is the first GPT partition. That is not a SMART capable device. You must stick to ada0 ...