Device /dev/gptid/706e4706-a7bb-11ec-a49c-14dae9124c74 is causing slow I/O on pool pool

[jordanthompson]

So, this is new...
I ran:

Code:

zpool status -v
  pool: boot-pool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:11 with 0 errors on Sat Mar 19 03:45:12 2022
config:

        NAME        STATE     READ WRITE CKSUM
        boot-pool   ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada5p2  ONLINE       0     0     0
            ada4p2  ONLINE       0     0     0

errors: No known data errors

  pool: pool
 state: ONLINE
  scan: resilvered 503G in 17:42:17 with 0 errors on Sun Mar 20 09:15:41 2022
config:

        NAME                                            STATE     READ WRITE CKSUM
        pool                                            ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            gptid/578aed6e-9fe7-11ec-a153-14dae9124c74  ONLINE       0     0     0
            gptid/57a37727-9fe7-11ec-a153-14dae9124c74  ONLINE       0     0     0
            gptid/ec0f97a6-a3fb-11ec-92d3-14dae9124c74  ONLINE       0     0     0
            gptid/706e4706-a7bb-11ec-a49c-14dae9124c74  ONLINE       0     0     0
            gptid/579f560b-9fe7-11ec-a153-14dae9124c74  ONLINE       0     0     0
            gptid/9a0e23e3-a48b-11ec-92e7-14dae9124c74  ONLINE       0     0     0
            gptid/57d31ec2-9fe7-11ec-a153-14dae9124c74  ONLINE       0     0     0
            gptid/57e6c684-9fe7-11ec-a153-14dae9124c74  ONLINE       0     0     0

errors: No known data errors

what's up?

 

[HoneyBadger]

Check your SMART stats for that device, as well as see if gstat -dp shows one device that's unusually busy.

Are all drives the same make/model?

 

[jordanthompson]

Are all drives the same make/model?

LOL - I have a mish-mash of 2 TB drives and a lone 3TB drive. I think the S.M.A.R.T. results are OK, but I will check now...

 

[HoneyBadger]

Check to see if any of them are on the list of known SMR drives.

 

[jordanthompson]

Check to see if any of them are on the list of known SMR drives.

Pretty sure they are not SMR drives. The old ones are too old... ARGGHHH I just bought some Seagate 2TB Barracudas that I now see are listed as SMR!!! Not sure if these are the ones giving me a headache, but WTF?!?!? I thought that SMR's were history with the WD lawsuit!

 

[HoneyBadger]

Pretty sure they are not SMR drives. The old ones are too old... ARGGHHH I just bought some Seagate 2TB Barracudas that I now see are listed as SMR!!! Not sure if these are the ones giving me a headache, but WTF?!?!? I thought that SMR's were history with the WD lawsuit!

Seagate's use of SMR predates WD, I believe. Double-check to be safe.

 

[oregonspain]

I meet the same problem, with alarms of the same type in a zpool of NVME disks.
In my case, all 3 disks are exactly the same.
Inside the zpool, I have an SQL DB, which hangs every time I have a "slow I/O on pool" message.
In the "Pending I/O" graphs, two peaks are observed that clearly coincide with the period of storage inactivity.
What could be the reason for this to happen, during night periods without workload on the zpool?

Attached to the post, the screenshot of the graphs, and the configuration of zpool and disk:

help please

 

[oregonspain]

Mi disk info too:

 

[jgreco]

I meet the same problem, with alarms of the same type in a zpool of NVME disks.
In my case, all 3 disks are exactly the same.
Inside the zpool, I have an SQL DB, which hangs every time I have a "slow I/O on pool" message.
In the "Pending I/O" graphs, two peaks are observed that clearly coincide with the period of storage inactivity.
What could be the reason for this to happen, during night periods without workload on the zpool?

Attached to the post, the screenshot of the graphs, and the configuration of zpool and disk:

help please

Just a note that you should quite possibly be using mirrors if you're hosting databases. Consider a database a type of block storage.

Some differences between RAIDZ and mirrors, and why we use mirrors for block storage

ZFS is a complicated, powerful system. Unfortunately, it isn't actually magic, and there's a lot of opportunity for disappointment if you don't understand what's going on. RAIDZ (including Z2, Z3) is good for storing large sequential files...

www.truenas.com

 

[Etorix]

I thought that SMR's were history with the WD lawsuit!

What's history is SMR drives sneakily introduced into NAS product lines.
SMR drives are still around in non-NAS products.
SMR drives are still present in NAS lines—but now it's clearly listed in the specifications (i.e. "buyer beware").
And vendors can still be sneaky for other features…

 

[oregonspain]

The situation has been reproduced again, showing the following notice in logs :

The connection to the SQL break, and i have to restart TrueNas to restore service.