Hot-plugging a single disk offlines all storage

seb101

Contributor
Joined
Jun 29, 2019
Messages
142
Hi!

Specifications are as follows:
Board: ASUS Q87T
Boot drive: SanDisk SDCZ430-032G-G46
Processor: Intel i5 4590T
Ram: 16GB DDR3 non-ECC
Disks: (see log below)
Freenas: 11.2-U5

Whenever I hot-plug a drive, it offlines all my other storage on that bus. All disks are directly, discreetly attached to the motherboard SATA headers and I have hot-plugging enabled in the BIOS, with AHCI mode. In the following console snip I plugged the new drive in at 11:21:20, the existing disks (ada0, ada2) seem to get detached. Then everything gets re-attached. However I have to manually 'clear' the zpools (which I did at 11:21:33) to get the pools back online.

Is this 'normal'? I suspect this is a poor hot-plug implementation on the motherboard (it's not a server mobo) but thought I'd check here for greater wisdom. Is there a 'fix' for this?

Thanks.

Code:
Jun 30 11:21:20 nas ZFS: vdev state changed, pool_guid=6243027112979744156 vdev_guid=18288264170334121262
Jun 30 11:21:20 nas ZFS: vdev state changed, pool_guid=10252935323651497312 vdev_guid=5593706345413028243
Jun 30 11:21:20 nas ada2 at ahcich0 bus 0 scbus0 target 0 lun 0
Jun 30 11:21:20 nas ada2: <WDC WD20EARX-00PASB0 51.0AB51> s/n WD-WCAZAE467037 detached
Jun 30 11:21:20 nas ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
Jun 30 11:21:20 nas ada0: <WDC WD10EAVS-00D7B1 01.01A01> s/n WD-WCAU44636200 detached
Jun 30 11:21:20 nas (ada0:ahcich1:0:0:0): Periph destroyed
Jun 30 11:21:20 nas (ada2:ahcich0:0:0:0): Periph destroyed
Jun 30 11:21:25 nas ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
Jun 30 11:21:25 nas ada0: <WDC WD10EAVS-00D7B1 01.01A01> ATA8-ACS SATA 2.x device
Jun 30 11:21:25 nas ada0: Serial Number WD-WCAU44636200
Jun 30 11:21:25 nas ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Jun 30 11:21:25 nas ada0: Command Queueing enabled
Jun 30 11:21:25 nas ada0: 953869MB (1953525168 512 byte sectors)
Jun 30 11:21:28 nas ada2 at ahcich0 bus 0 scbus0 target 0 lun 0
Jun 30 11:21:28 nas ada2: <WDC WD20EARX-00PASB0 51.0AB51> ATA8-ACS SATA 3.x device
Jun 30 11:21:28 nas ada2: Serial Number WD-WCAZAE467037
Jun 30 11:21:28 nas ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
Jun 30 11:21:28 nas ada2: Command Queueing enabled
Jun 30 11:21:28 nas ada2: 1907729MB (3907029168 512 byte sectors)
Jun 30 11:21:28 nas ada2: quirks=0x1<4K>
Jun 30 11:21:33 nas ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
Jun 30 11:21:33 nas ada3: <WDC WD20EARX-00PASB0 51.0AB51> ATA8-ACS SATA 3.x device
Jun 30 11:21:33 nas ada3: Serial Number WD-WCAZAE473309
Jun 30 11:21:33 nas ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
Jun 30 11:21:33 nas ada3: Command Queueing enabled
Jun 30 11:21:33 nas ada3: 1907729MB (3907029168 512 byte sectors)
Jun 30 11:21:33 nas ada3: quirks=0x1<4K>
Jun 30 11:23:30 nas ZFS: vdev state changed, pool_guid=6243027112979744156 vdev_guid=18288264170334121262
Jun 30 11:23:38 nas ZFS: vdev state changed, pool_guid=10252935323651497312 vdev_guid=5593706345413028243
 
Last edited:

seb101

Contributor
Joined
Jun 29, 2019
Messages
142

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
or maybe to your pool design... please post output from zpool status -v
 

seb101

Contributor
Joined
Jun 29, 2019
Messages
142
I am not at home right now, but there is really nothing exciting about my pool setup. I have a single drive in one pool, and a matched pair of drives as a mirror in the other.

Error occurs when any of those drives are hot-plugged, or when hot-adding a new (unpooled) drive to the system. Basically every hot-plug activity on that SATA bus.
 
Last edited:

seb101

Contributor
Joined
Jun 29, 2019
Messages
142
For completeness here is the information requested:

Code:
% sudo zpool status -v
Password:
  pool: freenas-boot
state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2     ONLINE       0     0     0

errors: No known data errors

  pool: msata
state: ONLINE
  scan: none requested
config:

        NAME                                          STATE     READ WRITE CKSUM
        msata                                         ONLINE       0     0     0
          gptid/7216a665-99be-11e9-9745-1cb72cef6c5b  ONLINE       0     0     0

errors: No known data errors

  pool: primary
state: ONLINE
  scan: resilvered 1.29T in 0 days 05:11:15 with 0 errors on Sun Jun 30 17:12:47 2019
config:

        NAME                                            STATE     READ WRITE CKSUM
        primary                                         ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            gptid/79786c45-9ab8-11e9-b551-1cb72cef6c5b  ONLINE       0     0     0
            gptid/229a6fd0-9b67-11e9-b551-1cb72cef6c5b  ONLINE       0     0     0

errors: No known data errors

  pool: wd1tb
state: ONLINE
  scan: scrub repaired 0 in 0 days 01:27:14 with 0 errors on Sun Jun 30 12:50:44 2019
config:

        NAME                                          STATE     READ WRITE CKSUM
        wd1tb                                         ONLINE       0     0     0
          gptid/a87b92c6-9a85-11e9-8031-1cb72cef6c5b  ONLINE       0     0     0

errors: No known data errors
 
Last edited:

seb101

Contributor
Joined
Jun 29, 2019
Messages
142
Can any one guide me where I might look for more detailed logs of what is happening? dmesg is fairly unhelpful.
 

colmconn

Contributor
Joined
Jul 28, 2015
Messages
174
I'd investigate whether the board supports hot-plugging before you dive into the logs.
 

seb101

Contributor
Joined
Jun 29, 2019
Messages
142

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
The board does support hotplug. I enabled it in the BIOS explicitly.
Although the board may support hot-plug in the BIOS and with the supported OS list, FreeNAS/FreeBSD is not on that list... see page 1-22.

You could try turning up the logging by enabling the debug kernel under the system-advanced section in the GUI, then look in /var/log/debug.log.

Since you're not using a supported board for the OS, who knows if anything different will appear there.
 

seb101

Contributor
Joined
Jun 29, 2019
Messages
142
Although the board may support hot-plug in the BIOS and with the supported OS list, FreeNAS/FreeBSD is not on that list... see page 1-22.

You could try turning up the logging by enabling the debug kernel under the system-advanced section in the GUI, then look in /var/log/debug.log.

Since you're not using a supported board for the OS, who knows if anything different will appear there.

You'll struggle to find many board manufacturers who state their boards support FreeBSD. Both of the builds in your signature use Asus boards and neither states they support FreeBSD.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Perhaps my suggestions offended you (an unintended result).

You can find a good number of FreeBSD supported boards here:
https://www.supermicro.com/support/faqs/os.cfm

Since Supermicro is the key recommended board manufacturer, it makes sense that this would be the case, see here:
https://www.ixsystems.com/community/resources/hardware-recommendations-guide.12/

Regarding my personal selection of boards (since you raised it), I have found good success running those specific 2 models (3 boards), but I don't use hot-plug for the onboard SATA ports, so can't (and wasn't meaning to) offer help on the basis of personal experience having made it work.
 
Top