S.M.A.R.T. Test / Scrubs for large pool's

[Pointeo13]

Currently I'm running Long Self-Tests and Scrubs twice a month for my Raidz_2_SATA. It takes about a day to complete the Scrubs. As I added more storage, should I continue to be running each of these tests twice a month? The Short Self-Test are currently setup to run four times a month. I expect to have double the storage by end of this year.

Code:

pool: Raidz_2_SATA
state: ONLINE
  scan: scrub in progress since Mon Feb 15 01:00:22 2016
        56.1T scanned out of 104T at 1.09G/s, 12h26m to go
        0 repaired, 54.08% done
config:

        NAME                                            STATE     READ WRITE CKSUM
        Raidz_2_SATA                                    ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            gptid/bb42496c-ccbc-11e3-af9c-5cf3fce554e8  ONLINE       0     0     0
            gptid/fa3480c7-059e-11e5-a388-5cf3fce554e8  ONLINE       0     0     0
            gptid/9768b882-9910-11e4-9b38-5cf3fce554e8  ONLINE       0     0     0
            gptid/be170a12-ccbc-11e3-af9c-5cf3fce554e8  ONLINE       0     0     0
            gptid/bf1ed8e2-ccbc-11e3-af9c-5cf3fce554e8  ONLINE       0     0     0
            gptid/c01f53e5-ccbc-11e3-af9c-5cf3fce554e8  ONLINE       0     0     0
          raidz2-1                                      ONLINE       0     0     0
            gptid/1add5798-9882-11e4-b735-5cf3fce554e8  ONLINE       0     0     0
            gptid/1b977ed1-9882-11e4-b735-5cf3fce554e8  ONLINE       0     0     0
            gptid/1c4f94c6-9882-11e4-b735-5cf3fce554e8  ONLINE       0     0     0
            gptid/1d09d3df-9882-11e4-b735-5cf3fce554e8  ONLINE       0     0     0
            gptid/1dc3fb10-9882-11e4-b735-5cf3fce554e8  ONLINE       0     0     0
            gptid/1e7f4bb5-9882-11e4-b735-5cf3fce554e8  ONLINE       0     0     0
          raidz2-2                                      ONLINE       0     0     0
            gptid/6cedee97-d360-11e4-a407-5cf3fce554e8  ONLINE       0     0     0
            gptid/6db82092-d360-11e4-a407-5cf3fce554e8  ONLINE       0     0     0
            gptid/6e825870-d360-11e4-a407-5cf3fce554e8  ONLINE       0     0     0
            gptid/6f4d2f7f-d360-11e4-a407-5cf3fce554e8  ONLINE       0     0     0
            gptid/700c20b4-d360-11e4-a407-5cf3fce554e8  ONLINE       0     0     0
            gptid/70c3fed4-d360-11e4-a407-5cf3fce554e8  ONLINE       0     0     0
          raidz2-3                                      ONLINE       0     0     0
            gptid/1188ebb6-da2f-11e4-9cb2-5cf3fce554e8  ONLINE       0     0     0
            gptid/12ce6032-da2f-11e4-9cb2-5cf3fce554e8  ONLINE       0     0     0
            gptid/141b7d4f-da2f-11e4-9cb2-5cf3fce554e8  ONLINE       0     0     0
            gptid/156883c8-da2f-11e4-9cb2-5cf3fce554e8  ONLINE       0     0     0
            gptid/16c00f2a-da2f-11e4-9cb2-5cf3fce554e8  ONLINE       0     0     0
            gptid/182548b8-da2f-11e4-9cb2-5cf3fce554e8  ONLINE       0     0     0
          raidz2-4                                      ONLINE       0     0     0
            gptid/a171ff36-ea19-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/a23d7661-ea19-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/a2fd9001-ea19-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/a3cd595a-ea19-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/a491157c-ea19-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/a57817d9-ea19-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
          raidz2-5                                      ONLINE       0     0     0
            gptid/56c3e79f-ea1a-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/579a1f95-ea1a-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/58665265-ea1a-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/594b1439-ea1a-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/5a1a493c-ea1a-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
            gptid/5ad473a5-ea1a-11e4-8345-5cf3fce554e8  ONLINE       0     0     0
          raidz2-6                                      ONLINE       0     0     0
            gptid/2a5228d8-8a12-11e5-9905-3440b5b0233c  ONLINE       0     0     0
            gptid/2b8c0b23-8a12-11e5-9905-3440b5b0233c  ONLINE       0     0     0
            gptid/2cc53e86-8a12-11e5-9905-3440b5b0233c  ONLINE       0     0     0
            gptid/2e0403b9-8a12-11e5-9905-3440b5b0233c  ONLINE       0     0     0
            gptid/2f3e74bf-8a12-11e5-9905-3440b5b0233c  ONLINE       0     0     0
            gptid/3076d876-8a12-11e5-9905-3440b5b0233c  ONLINE       0     0     0
          raidz2-7                                      ONLINE       0     0     0
            gptid/70c3d74e-b33a-11e5-83e2-3440b5b0233c  ONLINE       0     0     0
            gptid/72c3e734-b33a-11e5-83e2-3440b5b0233c  ONLINE       0     0     0
            gptid/74c1d3e1-b33a-11e5-83e2-3440b5b0233c  ONLINE       0     0     0
            gptid/76a320ea-b33a-11e5-83e2-3440b5b0233c  ONLINE       0     0     0
            gptid/787028e4-b33a-11e5-83e2-3440b5b0233c  ONLINE       0     0     0
            gptid/7a4744e8-b33a-11e5-83e2-3440b5b0233c  ONLINE       0     0     0

errors: No known data errors

 

[Mirfster]

Dang you have a lot of drives... :D

I would say that you are good with the current routine you have as far as the Short Test goes. May want to consider doing the Long Test once a month though; but that is my opinion.

You can see what others do/recommend here: S.M.A.R.T smart best practice!

 

[Pointeo13]

Dang you have a lot of drives... :D

I would say that you are good with the current routine you have as far as the Short Test goes. May want to consider doing the Long Test once a month though; but that is my opinion.

You can see what others do/recommend here: S.M.A.R.T smart best practice!

You know what, I know I said S.M.A.R.T tests but I'm so tired I actually meant Scrubs and S.M.A.R.T tests. I have updated my post and fixed the title.

 

[Mirfster]

Now it makes sense, seeing as how you included info on the Scrub... :)

For that I would say the same, once a month. But I guess it comes down to how often the data is accessed or written (Read-Only vs New). Just make sure that you schedule properly so the SMART Long Test is not running while the Scrub is going.

 

[Pointeo13]

I would say 70% of it is archive data and I also made sure the Long Smart Tests are schedule properly so it doesn't run into the Scrub. My SMART tests and Scrub is based off the best practice scheduled I pull off this site, just wasn't sure what people do once they hit 100TiB-200TiB range.

 

[Mirfster]

Yeah, your storage situation is "above my pay scale".. :) Let's see what others say.

 

[Johnny Fartpants]

I find I can run long SMART tests on no more than 5 X 4TB drives to ensure they all complete within 12 hours. That way I can set them off at about 6pm when most people have gone home for the day and they'll all be completed by the time they return to work the next day. When you have a lot of disks its very hard to schedule this as its not an exact science. For example if someone stays late and starts writing to the pool or you replication takes a bit longer than normal due to the amount of data thats recently gone on the pool then your long SMART tests won't (in my case) complete within the desired 12 hours. Therefore if you have them scheduled your system will start to have a backlog of disks trying to run long SMART tests and theres no way of stopping that practically. Personally I run the long SMART tests manually and like I said for my system I tell 5 of them to get cracking and 99% of the time they're done within 12 hours. I cron my short SMART tests just no my long.

I'd be interested to hear if anyone else has a better idea however.