all my drives suddenly DEGRADED after months of working fine

[joeschmuck]

I'm curious why someone would purchase 10 year old drives, I'm more curious as to why all the drives failed at the same time. What is the output of 'zpool status' ?

I'm not familiar with the LSI controllers and the drive shelf but there are a few cables involved and as hard as it is to believe, cables do go bad even when they are left alone. Check your cables, might even replace them.

 

[elvisimprsntr]

and based on the purchase date of your drives, it's looking unlikely that they are SMR drives.

Not sure how one can come to that conclusion based on a product manual revision history that indicated that the recording technology changed to SMR for model ST4000DM004. You need to look at the product manual for model ST4000DM000, which according to Seagate's own product manual are TGMR (a form of SMR)

https://www.seagate.com/files/www-content/product-content/desktop-hdd-fam/en-us/docs/100782401d.pdf

TGMR recording technology provides the drives with increased areal density.

What is TGMR tech by seaget?

What is TGMR tech by seaget? Can any one please explain? Thanks. Ref: https://www.seagate.com/www-content/product-content/barracuda-fam/barracuda-new/en-us/docs/100804187a.pdf Page 10 "2.4 Recording and interface technology"

linustechtips.com

btw, "TGMR" should be considered to be a form of SMR, and generally avoided if you're interested in NAS/SAN (or similar) applications.

 

[winnielinnie]

Not sure how one can come to that conclusion based on a product manual revision history that indicated that the recording technology changed to SMR for model ST4000DM004.

OP’s drives were all manufactured in 2015, a year before the revision from the spec sheet in your link (2016).

I’m no expert, but usually the “all my drives went bad at the same time out of nowhere” finds a controller or cable (or PSU) as the culprit.

 

[elvisimprsntr]

OP’s drives were all manufactured in 2015, a year before the revision from the spec sheet in your link.

I’m no expert, but usually the “all my drives went bad at the same time out of nowhere” finds a controller or cable (or PSU) as the culprit.

The OP's drives are TGMR, which is a form of SMR. Its clearly listed in the following link

https://www.seagate.com/files/www-content/product-content/desktop-hdd-fam/en-us/docs/100782401d.pdf

Regardless of root cause, used desktop TGMR drives were a poor choice.

 

[neofusion]

Seagate's own FAQ post.

CMR and SMR Hard Drives | Seagate US

See the list of recording technology utilized on our internal hard drives

www.seagate.com

View attachment 56085

Data sheet states TGMR, which from what I have read is a form of SMR

https://www.seagate.com/files/www-content/product-content/desktop-hdd-fam/en-us/docs/100782401d.pdf

View attachment 56086

You apparently chose to buy used drives that are out of warranty.

View attachment 56088

Bottom line, you put used desktop SMR drives in a server application. Accept your error and buy enterprise class NAS CMR drives.

I'm not seeing it.
It's like saying that just because a certain vehicle has wheels, it must be a car. But that's not actually true. It could be, but it might actually be a bicycle instead.

TGMR does not automatically mean the drive is SMR. The use of TGMR does however not exclude the possibility. Some other method must be used to determine what kind of drive it actually is. It's almost like they deliberately choose names like this just to confuse people.

In an ideal world, Seagate would just say what it is.

The linked FAQ does not list how the SMR usage has changed throughout the years, it just shows what the status was at a specific moment in time.

 

[neofusion]

Not sure how one can come to that conclusion based on a product manual revision history that indicated that the recording technology changed to SMR for model ST4000DM004. You need to look at the product manual for model ST4000DM000, which according to Seagate's own product manual are TGMR (a form of SMR)

https://www.seagate.com/files/www-content/product-content/desktop-hdd-fam/en-us/docs/100782401d.pdf

What is TGMR tech by seaget?

What is TGMR tech by seaget? Can any one please explain? Thanks. Ref: https://www.seagate.com/www-content/product-content/barracuda-fam/barracuda-new/en-us/docs/100804187a.pdf Page 10 "2.4 Recording and interface technology"

linustechtips.com

Here's the comparable sections from this product sheet:

https://www.seagate.com/files/www-content/product-content/ironwolf/en-us/docs/202989100a.pdf

The wording (describing increased areal density) is the same as for TGMR as it is for the 20TB Ironwolf Pro model linked above, clearly a model not believed to be SMR.

Again, not saying that the drives in question can't be SMR. Just that them being TGMR is not conclusive evidence one way or the other.

 

[TheThomen]

Have you got decent cooling across the card?
Also post the result of zpool status please. Might need a -v in there as well

https://pastebin.com/Hdatjren results, I checked the files its flagged and there fine

 

[TheThomen]

I'm curious why someone would purchase 10 year old drives, I'm more curious as to why all the drives failed at the same time. What is the output of 'zpool status' ?

I'm not familiar with the LSI controllers and the drive shelf but there are a few cables involved and as hard as it is to believe, cables do go bad even when they are left alone. Check your cables, might even replace them.

Drives were very cheap i did run a smart test before using and had no issues, I got the drives years ago they have never given me issues over 3 years of uptime. I mirror to the cloud too so even if they did all fail i wont loose data.

 

[TheThomen]

also the cards not hot at all its inside a 1U server with buckets of air flow

 

[NugentS]

But is the airflow going over the card?
Have you run a zpool clear as the status is showing no errors, whilst complaining about too many errors

 

[TheThomen]

But is the airflow going over the card?
Have you run a zpool clear as the status is showing no errors, whilst complaining about too many errors

Thank you for that. Cleared it. I will let you know if I have any other issues. is there a way I can check the temperature from command line?

 

[joeschmuck]

Have you run a zpool clear as the status is showing no errors, whilst complaining about too many errors

Exactly where I was headed. I'm surprised everyone was jumping on the SMR thing vice asking for the zpool status first, that is the normal progress for troubleshooting.

@TheThomen
I would recommend that you run a scrub of your pool and then once complete, check the zpool status again. If you have errors then you should see an error message but I would do this for peace of mind. Also, whilst a SMART Long test is a good test to perform on new drives, it is only a read operations so it's not a very complete test. In the future you might want to run a more inclusive hard drive test that includes writing to the entire drive. i like badblocks myself but I guess it doesn't work great for very large hard drives in the 18TB range. Eh, I personally don't plan to ever need more than 5TB total storage. Just something to think about.