New TrueNas Core, critical error, all 8 drives in pool degraded, permanent file errors, noob needs advise

[Polynikes]

I've got a 5U rackmount case with a bunch of fans, I don't think that should be an issue. I ordered a genuine intel 10 gig nic instead as well.

I'll do the memory dance, recreate the pool, while transferring data I'll do the finger heat test on HBA. If any issues I will switch to the mobo sata ports and see how that works.

 

[NugentS]

OK - so heat shouldn't be the issue here. I assume the server sounds like a hover craft and is hidden in a garage / basement so you can get some sleep (or the roar helps put you to sleep)

 

[Polynikes]

Haha well I have 4x noctua 120mm fans in it, and it's in an electronics closet (with ventilation). They don't make that much noise and the ambient temp is like 75.

 

[NugentS]

Warm, but not ridiculous.

Did you replace the original fans with the Noctua's. Thats often not a good idea as it reduces airflow through the case. I recently built a new NAS in a Fractal Design R5, added an extra fan (same as what came with the case, which were silent fans) at the front. So 2 fans blowing air through the HDD's and 1 exhaust.

I had to shut it down as the drives were getting toasty under test. I have now replaced the 2 front fans with high static pressure fans on a fan controller (running at medium speeds) to get much more airflow over the HDD's which works well. Just saying that silent fans (and you don't say what type your fans are) tend not to push much air which has consequences

 

[Polynikes]

One of the four RAM sticks was bad. Failed in all 4 slots. All others passed (although only like 20 min at a time each so far.)

 

[Polynikes]

Oh also, the HBA heat sink is quite warm to the touch but I can keep my finger on it without burning. I'm going to strap a fan to it with some zip ties.

If I want to continue with this hardware, can I simply delete the pool and recreate and transfer data back over, or should I reinstall truenas?

 

[NugentS]

Uncaught bad memory could have done almost anything. I would do a new install, but try keeping the config file if there are lots of config changes. If the config is simple to recreate - I would start again (but then I am a bit paranoid)

Good news on the memory - now you have three good sticks you need to test them for at least 24 hours (assuming you can test all 3 at the same time) some boards will run with 1, 2 or 4, just not 3 which does make some sense.

You also get the fun of replacing the stick of memory with identical memory!!

I think I would test memory for 24 hours, rebuild the NAS and then run badblocks on all disks at the same time. That should show up any errors as it thrashes the crap out of the disks and HBA



Then you should be able to use it with some degree of certainty.

ECC would still be better!!

 

[Polynikes]

Well I did a fresh install, installed my replacement RAM, ran memtest without errors, transferred over about 10 TB of data and ran a scrub, no errors. I did also use the mobo to increase the fan speeds and added a new fan pointing right at the HBA. So far so good!

I'll definitely be doing more extensive testing of my pc parts in the future...at some point I may still swap out the gaming parts for server parts, when I have more time.

Random aside question, I am deciding what cloud backup solution to use with select data from my NAS, and popular choices seem to be backblaze B2, Amazon s3/glacier, Storj, and to a lesser degree Google Drive and Onedrive. However, I never see Mega mentioned despite it having the lowest cost/TB, integration in TrueNas, full zero knowledge encryption, and a nice web interface and full featured device apps. Is there some reason people are avoiding Mega? I can't figure it out.