Help me understand the why of this difference in resliver

[AVB]

My server is just used for storage. No Jails, nor anything, about as straight forward as it gets. I have a pool with 2 Z2 eight drive vdevs. One vdev has 8tb drives and the other had 4tb drives. The eight tb drives are the famous or infamouse HGST SUN models the read 7.15tb in TrueNAS formatted as 512e. I bought 9 IBM-AIX ST6000NM0054 6tb drived to upgrade the 4tb drives I had. I do have a copy of the generic ST6000NM0054 firmware. But neither Seatools, or Seaflash or SG_write_buffer has been able to load it I just get an Illegal Operation error every time. The IBM drives are also 512e and since nothing within my ability that I threw at them would allow me to reformat or load new firmware on them so they remain 512e.

When I replaced the first 4tb drive it too 64 hours to resilver. The pool was only at 68% at the time. The other 7 drives took anywhere from 64-70 hours each to resilver . I have not added or erased any data from the pool since I began the upgrade.

As long as I was running up my electric bill I decided to reformat the HGST drives one at a time to 4096 block size. Luckily the standard sg format-size command worked just fine and 13 hours later I had a 4Kn drive. When I put that drive back into the same slot it came out of and did a replace it started the resilver process as expected. However, unless something goes seriously wrong in the next few hours it will be done in half the time (32 hours). Can this difference be attributed to 4K block size alone or do you think there something in the IBM firmware that slows everything down? Protection perhaps? I formatted my spare 6tb drive and stuck in my PC and it copies data at about the same rate as other 3 or 4tb drives I have but beyond that I haven't tested it.

Just looking for opinions to satisfy my curiosity since the server works at the level of performance I need. If it bothers me enough It will be fixed in about 4-5 years when I upgrade drives again.

 

[artlessknave]

I am quite sure you cannot "format" a drive into another sector type. that's done at manufacture. it's a physical characteristic of the drive. because of this, what you are saying doesn't make sense.

512, 512e, and 4kn will perform about the same( as long as your pool is ashift12, which pools are created as by default for like the last 10 years).
if your pool was ashift9, you would be unable to use 4kn drives, it would either fail or perform so badly you would just know.

ZFS knows which is which and how to use them appropriately. ZFS will write to 512e drives in 4b blocks anyway, because it knows all and sees all.

64 hours seems a little long but not unreasonable, especially if the server is busy.

trying to hack the firmware (it sounds like this is what you are doing?) seems like a terrible idea, and is definitely the first thing I would say you need to seriously reconsider....

 

[AVB]

I am quite sure you cannot "format" a drive into another sector type. that's done at manufacture. it's a physical characteristic of the drive. because of this, what you are saying doesn't make sense.

512, 512e, and 4kn will perform about the same( as long as your pool is ashift12, which pools are created as by default for like the last 10 years).
if your pool was ashift9, you would be unable to use 4kn drives, it would either fail or perform so badly you would just know.

ZFS knows which is which and how to use them appropriately. ZFS will write to 512e drives in 4b blocks anyway, because it knows all and sees all.

64 hours seems a little long but not unreasonable, especially if the server is busy.

trying to hack the firmware (it sounds like this is what you are doing?) seems like a terrible idea, and is definitely the first thing I would say you need to seriously reconsider....

I'm not sure what you mean by format to another sector type. If it is allowed in the firmware you certainly can change the block size from 520 to 512, 512e to 4Kn and a host of other things. None of my vdevs were manually ashifted ever so we can eliminate that. The end result performance was not what was in question. The drives do perform pretty much the same in the server as the other vdev. What I am trying to figure out is why these drives are so much slower at the resilver task then other 512, 512e or 4Kn drives I have when used in the same server, same pool, same vdev with exactly the same amount of information. There has to be some reason for that which is beyond my knowledge base - which could be a whole universe of things.

The only thing the server was doing was the resilver. It basically is just media storage so if I'm not watching or listening to something it is off. I'm not good enough to "hack" the firmware although I did start to look into what was involved but it was beyond my comfort zone. These are Seagate drives so I don't consider using Seagate firmware tools as hacking nor using SG3 utilities which can only change what the manufacturer allows you to change. By your outlook changing logical block size from 512 to 4096, as I mentioned above, is hacking since it is a change in the firmware. I don't think many will agree with you.

 

[artlessknave]

it's fully possible I misunderstood what you were saying. it looked to me like you were saying you were putting the firmware from another brand on in order to "format" the sector size, and then it was performing differently. if that is not the case then you can ignore that.

 

[AVB]

it's fully possible I misunderstood what you were saying. it looked to me like you were saying you were putting the firmware from another brand on in order to "format" the sector size, and then it was performing differently. if that is not the case then you can ignore that.

It's all good, at least you gave it a shot. I never would have come up with the ashift idea myself.