FreeNAS failure

[devour]

Hi, sorry for my perhaps bad english but it´s not my native language.

Anywho, I´m in a quite big mess now, my NAS (based on a Asus E35M1-I with 8Gb RAM and 5x2TB drives, with FreeNAS installed on a USB-stick) has gone belly up on me.
Last week I noticed my shares were missing and I went to check on the old NAS-box to see what´s up. I looked like t´his: https://www.dropbox.com/s/mpxo0yf6bvsf9h9/2014-01-20 19.05.13.jpg
So I rebooted and got the same result.

I´ve tried quite a few things.

• First I change the RAM, no success there.
• Tried to boot with just the USB-stick connected, no drives, and that worked, it booted up.
• Started to connect one drive and started up, no problem. Connected a second drive, no problem.
• When I got to the fourth the same error ocurred. Skipped the fourth and tried the fifth, same problem.Tried ANY combination of 4 drives, didn´t work, same error as before. Doesn´t matter it seems what drives are connected, when it´s more then 3 it failes.
• Made a new 8.x FreeNAS USB-stick, same.
• Made a new 9.2x stick, boots with 5 drives.
• Tried changing to another motherboard, cpu and RAM-sticks, same as the above.

So this is where I´m now. I´ve managed to boot the system up, connect to the WEB-interface.
If I try to auto-import the volume it crashes after a little while (within a minute atleast).

Does anyone have any suggestion on what to try? Is there any way to salvage this or I´m I toast? Got the suggestion to boot Solaris 11 Live USB and try to import the pool and scrub it. Haven´t managed to do that yet though.

Best regards, Daniel

 

[SweetAndLow]

It sounds like you have tried to cover all your possible hardware issues. I'm not sure but maybe there is some zfs metadata corruption. Since you aren't using ECC ram you could have had a bad memory stick and after so long you got some corruption in your pool. I would try and run memtest on the original stick of ram you were using and if it has errors then I would say your pool is corrupt.

 

[devour]

It sounds like you have tried to cover all your possible hardware issues. I'm not sure but maybe there is some zfs metadata corruption. Since you aren't using ECC ram you could have had a bad memory stick and after so long you got some corruption in your pool. I would try and run memtest on the original stick of ram you were using and if it has errors then I would say your pool is corrupt.

Forgot to add that aswell, I did run memtest on the sticks and it showed some problems with the sticks, so I´ve throwned them away now. So yes, I guess the data is corrupt. That makes it impossible to save?

 

[cmfisher4]

Yes, pretty much, especially if memtest is telling you the RAM is bad. Chalk up another data loss to not using ECC memory. cyberjock can add this to the list.
I hope you have (had) a backup solution.

Chris

 

[devour]

Yes, pretty much, especially if memtest is telling you the RAM is bad. Chalk up another data loss to not using ECC memory. cyberjock can add this to the list.
I hope you have (had) a backup solution.

Chris

Well, my primary backup was this box. I´ve also stored all my DSLR photos on a offsite server. Unfortunately some of my data were only stored on the FreeNAS box.

I have a new box ready to be started that I bought now that this went south. A HP N54L, equipped with 8GB of ECC memory atleast. Loading that box with 6x3TB WD Red drives.

 

[rm-r]

you should have 16gb min ecc ram for zfs of that size - probably 32 gb! check your motherboard setup but if you only have 4 slots - think ahead and get 2 x 8gb sticks - leaving the other 2 ready for the next 2 x 8gb you will need

 

[gpsguy]

The OP only has 2 slots. While HP says 8Gb is the maximum it will take, you can put 16Gb in them. I'm using this in my N54L.

Kingston 16GB (2 x 8GB) 240-Pin DDR3 SDRAM DDR3 1333
ECC Unbuffered Server Memory Model KVR1333D3E9SK2/16G

 

[SweetAndLow]

It will work just fine with 8gig of ecc memory. I would suggest getting 16gig of ecc though. I have 6x3tb reds in my server with 16gig ecc memory and have very nice pool performance of 425-475MB/s.