POOL lost with every reboot

[VladimirVasiljevic]

on the fresh SuperMicro 45bay server with latest FreeNAS-11.2-U4.1 things are going crazy
I know it's not a best solution, but we have made 3 hardware raid with AVAGO/LSI megaraid SAS, each with 12 disks
then a pool was created with a stripe over three really big virtual disks.
on first reboot pool disappeared and refused to import back
as it was still empty I have just recreated new pool and tried new reboot to found pool disappeared again?!?!

I'd like not to install a plain centos or ubuntu server to share this beast but if it continues loosing pools I'll have to

thanks

 

[Arubial1229]

Sigh...

Don't use hardware RAID with FreeNAS...ever.

 

[PhiloEpisteme]

ZFS and thus FreeNAS needs direct access to the drives. Using raid cards not in IT mode and virtual disks precludes that. I don't know for sure that this is the cause of your pool issues but it is likely that it can become a cause in the future. Can you clarify exactly which cards you're using, whether they are flashed to IT mode, and what your pool configuration is?

If the issue is your cards the solution may be as simple as flashing those RAID cards to IT mode or picking up a couple of HBAs.

 

[VladimirVasiljevic]

Could it be the sync command not propagating to hw raid correctly?
I have repeated the issue 3 times, every shutdown would corrupt metadata making Import from cli impossible

 

[PhiloEpisteme]

Could it be the sync command not propagating to hw raid correctly?

I'm not sure. How many drives total do you have? And what types of drives are they? And what is your workload? And what board do you have?

The reason I ask is that if you have 36 disks as your post above suggests you could purchase a combination of HBAs and Expanders for a total cost of ~$150. Doing so will accomplish two things.

First, if you swap out the megaraid drives with the HBA/Expander combos and find your problem resolved you just figured out that the raid cards were the problem and solved it.

Second, you'll be using hardware more appropriate for FreeNAS which will help ensure data integrity over the lifetime of your build.

I imagine that the $150 is worth the system stability, time spent for you to not have to set up ZFS on your own, and risk getting something wrong.

Do you have constraints etc preventing the use of HBAs?

 

[Chris Moore]

I know it's not a best solution, but we have made 3 hardware raid with AVAGO/LSI megaraid SAS, each with 12 disks

This is simply not acceptable. It will never work properly. Remove the hardware RAID and get a SAS HBA instead. You will never get this to work reliably.

 

[Chris Moore]

SuperMicro 45bay server

Please give exact hardware spec so we can know more about what you have to work with.
See guidance:
https://www.ixsystems.com/community/threads/forum-guidelines.45124