BUILD Can't achieve 6 HDDs

[marcevan]

Seems like while my motherboard supports 6 drives, whenever there are 6 and all are in my ZFS Raidz2-0, one throws some bad behavior.

I had 6x 3TBs and one had repeated CRC and sector errors enough that it was time to replace it. So bought a 4TB and did a proper replace and silvering finished but low and behold another drive is now unavailable.

I believe the lowest cost action is to replace SATA cables and see if one of them is causing an issue (at MB or at HDD), but failing that next up is a full motherboard replacement which is on a par with a new Server replacement.

Not sure if it's possible to take a ZFS pool based on 6 HDDs and move that to a new setup.

 

[Chris Moore]

It is fast and easy to move to a new system board.


Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[marcevan]

Thanks Chris,

To be sure, if I had a new system build and put in the same 6 HDDs onto 6 motherboard SATA ports and used my current Freenas U11.5 USB I should have success? Or is it somewhat more complicated than that?

 

[Chris Moore]

Thanks Chris,

To be sure, if I had a new system build and put in the same 6 HDDs onto 6 motherboard SATA ports and used my current Freenas U11.5 USB I should have success? Or is it somewhat more complicated than that?

You don't even need to put the drives on SATA ports. FreeNAS detects hardware at each boot. If the drives are connected to a supported controller, SAS or SATA, FreeNAS will find them and import the pool. Moving to a completely different hardware platform is as easy as moving the drives over and booting back up. I have done it probably 5 times over the years.
Also, the drives don't need to be is a particular order so there is no need to remember what drive was plugged where.

 

[MrToddsFriends]

To be sure, if I had a new system build and put in the same 6 HDDs onto 6 motherboard SATA ports and used my current Freenas U11.5 USB I should have success?

There's even a forum resource about this:
https://forums.freenas.org/index.php?resources/moving-freenas-to-new-hardware.89/

 

[marcevan]

Thanks but I suspect that only works if you stick on process vendor.

I plan to move from AMD to Intel in a 3U rackmount enclosure with server level processor and motherboard. I already see that many places on my existing FreeNAS show AMD64 in them such as plugins.

 

[sfcredfox]

I'm not sure that matters for importing the pool. A ZFS pool is the same regardless of Intel/AMD. You could still try importing the pool on another platform as far as I know.

Someone might be able to confirm this for you.

 

[CraigD]

Listen to @Chris Moore

I was going to recommend trying a M1015 SAS PCIe controller, however your motherboard is not a good fit for a freeNAS

A new CPU, motherboard, and maybe even RAM will be needed (If you value your data) cost will vary depending on your workload and number of users

The Hardware Recommendations Guide is a good place to start

Have Fun

 

[Chris Moore]

Seems like while my motherboard supports 6 drives, whenever there are 6 and all are in my ZFS Raidz2-0, one throws some bad behavior.

I had 6x 3TBs and one had repeated CRC and sector errors enough that it was time to replace it. So bought a 4TB and did a proper replace and silvering finished but low and behold another drive is now unavailable.

I believe the lowest cost action is to replace SATA cables and see if one of them is causing an issue (at MB or at HDD), but failing that next up is a full motherboard replacement which is on a par with a new Server replacement.

Not sure if it's possible to take a ZFS pool based on 6 HDDs and move that to a new setup.

Did you make the move to new hardware or do you need suggestions on what to get?

 

[anmnz]

Thanks but I suspect that only works if you stick on process[or] vendor.

Not true. There is no reason at all you can't move a pool between machines with Intel vs AMD processors. I've done it before.

ZFS was even designed to be portable across processor architectures, so that you could move a pool between SPARC and x86 systems, no problem.

I already see that many places on my existing FreeNAS show AMD64 in them such as plugins.

"AMD64" does not refer to the processor vendor, but to the 64-bir x86 CPU instruction set, which is common to both the AMD and Intel CPUs we're familiar with. You'll still see "AMD64" when you're running FreeNAS on an Intel CPU.

 

[Chris Moore]

ZFS was even designed to be portable across processor architectures, so that you could move a pool between SPARC and x86 systems, no problem.

Since Oracle closed the source, there has been divergence between what they do and what Open ZFS is doing, so you need to be mindful of the flavor of ZFS.

Still, it is true, ZFS is portable. I have moved a pool from FreeNAS to Linux and back with no trouble. It isn't the CPU that matters.

You'll still see "AMD64" when you're running FreeNAS on an Intel CPU.

Interesting story... AMD and Intel actually had competing 64 bit architecture and this was one time where AMD beat Intel.
Look at: https://en.wikipedia.org/wiki/Itanium
Intel didn't exactly abandon their architecture, but it is only used in a relatively narrow band of high-end computing. Intel, and everyone else, adopted the AMD 64 bit architecture for general computing.

 

[anmnz]

Since Oracle closed the source, there has been divergence between what they do and what Open ZFS is doing, so you need to be mindful of the flavor of ZFS.

Yes of course, and it was only an anecdote!

But that said (and from a position of admitted ignorance) I'd expect this level of portability to be a matter of how the fundamental data structures are laid out on disk in terms of size, endianness, padding, etc. And I wouldn't expect those to change in a hurry. It would be surprising and disappointing to find OpenZFS developers had blithely broken such a level of (in-principle) platform independence.

I am not aware of OpenZFS working on non-x86 anywhere though.

 

[sretalla]

I'm not 100% sure it's helpful to raise, but only 430W power supply... maybe look at running the math on it (depends what else you have in the case and how old the PSU is).

 

[Chris Moore]

But that said (and from a position of admitted ignorance) I'd expect this level of portability to be a matter of how the fundamental data structures are laid out on disk in terms of size, endianness, padding, etc. And I wouldn't expect those to change in a hurry. It would be surprising and disappointing to find OpenZFS developers had blithely broken such a level of (in-principle) platform independence.

Well, I was more putting it out there for everyone than trying to tell you. I figured you might be aware.
Many of the developers departed Sun when Oracle took over and are still working on the OpenZFS project from their new places of employment.

I am not aware of OpenZFS working on non-x86 anywhere though.

I have not heard of one, but I agree that the way data is put on disk is intended to be hardware agnostic for the very purpose of being able to move the drives, with data, to a different platform because Sun had the Spark processor and moved to the x86 and they probably wanted to keep their options open for moving to the next platform later. They envisioned ZFS as being the last file-system that would ever need to exist.
https://www.cs.utexas.edu/~dahlin/Classes/GradOS/papers/zfs_lc_preso.pdf

 

[Stux]

There is cross platform compatibility,(Ie Oracle/Open) but only for v23 (iirc) pools and below,

So just like you can’t move a pool backwards after you upgrade it, the same applies.

OpenZFS pools are v5000 Oracle is up to v32 or something.

V5000 added feature flags ;)

 

[anmnz]

There is cross platform compatibility,(Ie Oracle/Open) but only for v23 (iirc) pools and below,

Version 28, and to be really really pedantic that's the filesystem version not the pool version. :-D