Attaching HDD via HotSwap degrades another volume

[Chiaki]

Hello everyone.

My volume setup looks like this:

4 WD Red in RAIDZ2 + 1 WD Red as Spare

1 Toshiba X300 Striped as HotSwap

When I start the server with every HDD attached, everything's fine. Unmounting the Toshiba volume and then detaching the Toshiba: still everything's fine.

Now when I attach the Toshiba back while the system is running for some reason the WD Red spare HDD becomes unavailiable, the volume status says it is removed. But this is not true, I didn't touch the HDD.

Restarting the server with the Toshiba attached or detached makes the spare work again.

Whenever I attach the Toshiba while the system is running the WD Red spare will appear as removed and the volume as degraded.

What's going on? Any ideas?

EDIT:
I turned the 4 WD Red + 1 WD Red RaidZ2 + Spare into a 5 WD Red RaidZ2. Now attaching the Toshiba doesn't degrade this volume anymore.
From this I deduce that this is indeed a FreeNAS bug and not related to my hardware.

 

[Bidule0hm]

Sounds like a power brownout.

 

[Chiaki]

Sounds like a power brownout.

Excuse me, but what is a brownout? =/ And how does it explain why the HDD in question is removed when it is used as a spare and not if it is used as part of the RAIDZ2?
Also, I'm using a 500W 80plus Platinum grade PSU from Enermax. I doubt that I don't have enough power.

 

[Bidule0hm]

A dip in the voltage.

Maybe ZFS is more sensitive to a drive dropping as a spare than as a part of an array.

Or maybe I'm just wrong.

 

[Chiaki]

Hmm... if it is a dip in the voltage is there a way to find out from the logs?

 

[Bidule0hm]

I'd say no because logs are slow usually. And besides that I never saw a log of the voltages anyway.

 

[Ericloewe]

Are you trying to hotswap disks without a backplane? That's a big, big no no.

SATA is simply not designed for hotswapping without a backplane. Backplanes have some pins shorter to ensure that the drives' inrush current limiting circuitry gets connected first. Additionally, proper backplanes also include additional filtering to more easily handle drive insertions.

I'd say no because logs are slow usually. And besides that I never saw a log of the voltages anyway.

IPMI should log voltage sensor threshold excursions, much like fans. Dunno how accurate the onboard measurements are though, they used to be looked at with suspicion on consumer boards a few years ago.

 

[maglin]

Chiaki what is your hardware? It's not a FreeNAS issues. It's clearly hardware related. Cheap backplanes/expanders can have issues when hotswapping. Works fine in say windows but in FreeNAS or ZFS for that matter it can cause issues. When you insert that drive up to 35 watts is being demanded to it's startup and this can cause enough voltage drop down the power chain of that backplane/expander to cause one or more drive to go offline for a split second. This will cause it to drop out of the vdev.

Now if you where using a Server enclosure with a proper backplane then I wouldn't be talking about this because it wouldn't be an issue. I have 15 drives in my 15 drive 3U server and I hotswapped a bunch of drives just this week running badblocks and SMART tests on them. I've never had issues. People with Notica (sp?) expanders usually have issues.

And I'm willing to bet that Toshiba drive is now at the end of the chain so it's not effecting the other drives when it's inserted.

And best solve for this issue is to not hotswap your drives. I wouldn't trust it after what has already happened.

 

[Mirfster]

I want to know more about the system Make/Model... Pretty sure (as everyone else mentioned) that more than likely the system does not support HotSwap nor has a proper BackPlane.

 

[Chiaki]

Hello everyone and thanks for helping up this far.

You can find my hardware config here:
https://forums.freenas.org/index.ph...-build-check-first-freenas.42502/#post-276298
(Mind that I used the more expensive config with the Xeon E5-2620v4 !)

I'm using this Hot Swap Dock for hot swapping: https://www.amazon.de/Delock-5-25-Wechselrahmen-SATA-Festplatten/dp/B00293D3N2

The Hot Swap Dock is hooked up internally to the mainboard SATA ports just like every other HDD.

I didn't know that actually using it this way would cause some dip in the voltage. I thought the hot swap dock would use some kind of capacitor to even out a sudden power demand from plugging a drive in. Was I wrong to assume this?

EDIT: Well... I'm starting to wonder if what I bought is not an actual hot-swap dock but rather just a dock that transforms a 5.25" drive bay into a 3.5" drive bay without any additional function. Is there a way to find out for sure by examining it?

 

[Ericloewe]

EDIT: Well... I'm starting to wonder if what I bought is not an actual hot-swap dock but rather just a dock that transforms a 5.25" drive bay into a 3.5" drive bay without any additional function. Is there a way to find out for sure by examining it?

If you can get us pictures of the PCB on the back and of the SATA connector, we should be able to tell the difference.

Basically, it's supposed to have an electrolytic cap or two and the SATA connector is supposed to have most pins recessed, except for a few.

 

[Chiaki]

If you can get us pictures of the PCB on the back and of the SATA connector, we should be able to tell the difference.

Basically, it's supposed to have an electrolytic cap or two and the SATA connector is supposed to have most pins recessed, except for a few.

Hi Eric!
You can find a lot of up-close pictures of it here:
http://www.delock.de/produkte/F_343_5-25_47191/merkmale.html

EDIT:
It says Hot Swap in the specification though!

EDIT2:
Here's an English datasheet: http://www.delock.de/produkt/47191/pdf.html?sprache=en

 

[Ericloewe]

Ok, so it's clearly missing filtering on the SATA power side. The PCB just seems to be a basic LED driver circuit that takes input from the drive's power connector (yeah, the activity indication comes from the power connector).

We've seen backplanes with the proper SATA connectors, with recessed pins, have problems with hot plugging drives - presumably because their filtering circuitry had been omitted.

I can't tell if they're using proper backplane connections, here.

tl;dr - it's a crappy product and it's probably to blame.

What's the PSU, by the way?

 

[Chiaki]

What's the PSU, by the way?

It's a Platimax 500W Enermax (80plus Platinum)

So far I didn't see any problems hot plugging the Toshiba while having all WD Reds as one zpool. Maybe I'll change my backup strategy then, just to be safe, like I mentioned in the last lines here: https://forums.freenas.org/index.ph...up-via-weekly-hot-swapping.44507/#post-298260

 

[maglin]

So it's a single dock? I was thinking it was a backplane of sorts. That does just look like a 5.25 to 3.5" bay with a tray. You can hotswap SATA if your BIOS allows. I've never bothered with SATA hotswap though. You can't expand SATA like you can SAS.


Sent from my iPhone using Tapatalk

 

[Chiaki]

That does just look like a 5.25 to 3.5" bay with a tray.

Oddly the official specification mentions Hot Swap, though. Strange marketing I'd say ^^.

 

[Stux]

Hot swap works great with AHCI, but the backplane needs to have capacitors to prevent a voltage drop causing issues with the other drives on the circuit.

I don't think this case has that, and that explains why it causes the other drives to drop.

It's a cold swap case.