SMB Connection drops - IPMI errors - now unplanned reboots

[MacToFreeNAS]

First post, so trying to include all the info, but I am sure I will have missed something. System info should be in my signature if all works...

Started with SMB connections dropping on large files, now getting these errors:

Code:

Mar  4 13:25:22 arc ipmi0: KCS: Reply address mismatch
Mar  4 13:25:22 arc ipmi0: KCS error: 5f
Mar  4 13:25:22 arc ipmi0: KCS: Command mismatch

Rebooted by itself 3 times yesterday...

Starting to think the motherboard is having issues. Do you have any insight?

Trying to figure out where/how to attach debug file....

Thanks, - Robert

 

[artlessknave]

isnt that the board that eats itself due to a bios bug?
if so, you want to ask asrock or ixsystems directly, since its a freenas mini
i think they had a fix, or were fixing the motherboards, or...something?

Reddit - Dive into anything

www.reddit.com

 

[MacToFreeNAS]

I am 3 years out from purchase, so not thinking there is any warranty options. It has the updated IPMI / Bios which was supposed to fix the issue. Symptoms match the stuff I am reading from 5 years ago about the issue though.

I am trying to decide if I build a new system or buy a new iXsystems Mini X+. Would like to move to 5 drives RAID-Z1 as it seems to optimize space from what i have read.

I have no problem putting parts together, but don't have time to do all the research on what I would buy. Are there any lists of all the parts for a good quality TrueNas server?

Thanks

 

[artlessknave]

RAID-Z1

no. raidz2 is the minimum unless you know exactly what you are doing and accept the increased risk of the whole pool dying while rebuilding a failed disk.
in general, the more drives in the vdev, the better your efficiency (within reason, raidz2's over 10 or so drives is reaching no recommended)status, but the porformance of raidz vs mirrors is usually not as good.

ultimately, the parts you choose can only be chosen by you; only you know what you can get access to, what requirements you need to fill and your budget.
I have, in my signature, my bothers nas. it is about the closest you can get to a miniXL.

I am trying to decide if I build a new system or buy a new iXsystems Mini X+

you cant really go wrong buying an X+, if you are so inclined. doing it youself requires the research to do so, and if you are unwilling or unable to put in the effort, paying someone else for that effort is the only option. remember, the forums are voluntary. we will critique a build for mistakes, but we are highly unlikely to do your work for you.

there are a large number of links in my signature. read them and decide.

 

[MacToFreeNAS]

Thanks for all the info, and the math on MTF on RAIDZ1. So going to RAIDZ2 would mean going to 6 drives minimum 4+2 as I understand it for optimal space utilization on 4K drives. So Mini XL+ instead of X+ if I go iXsystems route, which I may go with due to lack of time.

Now to look into how best to migrate from 4 drive Z1 to 6 drive Z2 Raid. I think I can use the 2 extra bays in the 8 bay box to move the data between pools. Will look into the options and do some searching.

 

[artlessknave]

Now to look into how best to migrate from 4 drive Z1 to 6 drive Z2 Raid

the best way to migrate is to make the new pool with all its disks in the new server, and replicate everything over.
you can use the left over disks to make a stripe backup server or something. remember RAID(z) ! = backup.
since you cannot (yet) modify a raidz, you ideally want to fill the 8 bays if you only want the one pool. making it 6 drives wide will limit you to 6 drives wide. there IS a radiz migration in the pipe, but afaik it is still a ways out.
one way to do this is to buy a bunch of cheaper, smaller drives that give you sufficient space to migrate everything, while also fillout out your planned pool size, and then start replacing the drives, again, using the replaced drives for backups or other stuff.

 

[MacToFreeNAS]

The best way to migrate will not work, as current pool in current box is off line/unreliable at best. I am thinking I throw the existing 4 disk pool in the new 8 bay box, replicate to a 2 drive pool/mirrored, and add two disks to the 4 and create the new pool, then migrate from 2 drives to the 6.

From https://www.truenas.com/community/threads/raidz2-number-of-disks.16153/ is seems that 4+2 is idea for RAIDZ2 and going to 8 will be slightly slower and waist space. I realize I will be stuck at 6 drives (until/if they add feature to expand dives in pool) but not worried at this point, I just upgraded the drives from 4tb to 6tb, and I will get an additional data disk out of this, so should be good for quite some time.

I do cloud (iDrive) back up of my pool, and agree raid != backup, so have that covered.

Just looking forward to a stable server again... and hope this one does not eat its motherboard.

 

[artlessknave]

no, 8 is fine. that post you link is VERY old. I am quite sure that the stripe waste that that refers to is no longer a thing.
you will get far more space efficiency out of 8 wide vs 6 wide raidz2 than you will waste to the metadata. and by slightly slower, on relatively modern systems, that means like 10ns vs 20ns. unless you are running it on an actual potato, you will struggle to identify the difference.
again, ultimately up to you, but I would *highly* recommend just doing the max drives for the chassis and be done with it.

the raidz modification upcoming acually only reflows data, it doesnt truly restripe it until it gets rewritten organically.

 

[MacToFreeNAS]

OK, thanks for the info. Good to know 8 drive Z2 works well.

If I use all 8 I still have the migration issue to resolve as I would have to load data from existing pool onto external storage that I do not have to make that work. I think I am going to with the 6 so I can make this work without a lot more hardware, and do not project to outgrow that space for many years.

Again, appreciate all the input and information. Very helpful!

 

[artlessknave]

I read the details of the raidz vdev expansion a little closer, and it looks like it works better than my understanding of it was, so if you think 6 is fine, go with that.
if you want to expand it later, basically, existing data stays at the previous efficiency % (raidz2x6), with any data new or rewritten being at the new % (raidz2x7).
there is, as of yet, no convenient method to reflow the existing data to the new layout, but there was mention of interest in making one, with the benefit of that also likely applying to compression type changes as well.

 

[MacToFreeNAS]

Wanted to close the loop on the IPMI errors and reboots.

The IPMI errors in the console interface and in the IPMI web interface were all related to a battery which was failing. I finally got the battery error days later. I replaced the battery on the motherboard, and the IPMI errors and reboots are gone, system has been stable for a week now.

My SMB dropping connections on large file transfers is still an issue, so that seems unrelated to the other issues. I do not know if the SMB issue is hardware or software.

 

[artlessknave]

could your storage controller be overheating? is the board getting enough cooling for the non fan heat sinks?

 

[MacToFreeNAS]

No overheating based on temps reported on the board. The transfer seems to drop connection as it is figuring out how big the file is before it starts to actually transfer data. Small files work fine, just the larger ones....

I have tried this from two different Mac laptops, wired and wifi. Not sure if there is a Mac / SMB timeout which is being hit, or something else. I guess I can try rsync to see if that works. This used to work, not sure what may have changed.

 

[MacToFreeNAS]

With some testing, I am able to move large single files, the issues is coming up when I am trying to move my Apple Photos library (500GB), which is really a package directory of many folders and files. That is still failing when it is discovering how many files it needs to move.

Is there an SMB limit on number of files or folders or path length that I might be running into?