Where to start troubleshooting

[KempelofDoom]

I have a setup that seems to be giving me trouble. Things have been fantastic for some time. Only recently have I had issues with latency and I'm not sure where to begin. Regardless if I'm using sshfs or samba, it can take upwards of 30~60 seconds before a folder will finally open. I started off with a round of 8 3TB drives in RaidZ1, then added another round of 8 3TB drives in RaidZ1 and used it to expand the current pool. I was down to about 200GB of free space when things started getting wonky. I added another 8 4TB drives in RaidZ1 and used it to expand the pool. Performance remained spotty and I figured that things were getting bad because I was running out of space. So then I decided to upgrade from 8.3.1 to 9.2.1.3. Now I have some issues with smb but performance is still spotty even over sshfs.I'm fairly unfamiliar with FreeBSD but I am comfortable with Linux. Where can I start viewing logs and determine if it's the network adapter, the drive controllers, smb, smart, or something else? Does anyone have a suggestion that would help me improve performance? I have a separate plex server that scans this file server over smb and it will sometimes scan quickly and sometimes it will choke over and over. Specs below.

Asus F2A85-M/CSM
A8-5600K
32GB RAM
16 x 3TB
8 x 4TB
ARC-1260
Another disc controller to make up for the 1 missing sata port the mobo has only 7

Appreciate any suggestions as I would like to make sure my baby is running like a top.

 

[cyberjock]

If this is for home use, your RAM is probably fine for that size. If this is business, your problem is probably RAM cache is too small for the pool size. Even I'd be questioning that little RAM for a pool of that size for home use. I will say that RAID controllers + ZFS, aside from the potential for data loss, adds a certain amount of potential for performance problems(and in some cases performacne gains). You probably don't want to hear this, but you'd be better of with a dumb HBA than that controller.

I used an Areca 1224(or something like that.. don't remember the model number) and I can say that Areca's are NOT recommended, even when forced into jbod mode.

Your NIC is also a Realtek, and those are not recommended in the slightest. They have been known to cause random problems and other issues that just go away with an Intel. They have been known to work fine for a while, then suddenly not work fine for no reason whatsoever. Presumably its because the hardware is crap and the driver is hacked. Nobody really gives a crap about Realtek suport, and someone is bound to show up and say something like "so you are using realtek and complaining about performance? Are you dumb?" in the next few hours.

 

[KempelofDoom]

LOL, thank you for the quick response! I was reading another post you responded on about the Areca. The main reason I went with that was because I have a few on hand and it could run 16 drives for me. Can you suggest something that can run 16 drives or even 8? I do have an intel NIC so I'll get that installed right away. It is for home use and I'm the only user so it's load should not be that great. I've wondered if I were to swap out the 2nd set of drives I installed with new ones since that 2nd round was rushed and I used some drives that we not brand spanking new. Though I did run all the tests I could to feel safe in using them. Do you know how ZFS data is written across such a setup? I wondered if it was like filling the drives up one at a time and it hit one that was bad. That shouldn't be the case if RaidZ1 works anything like traditional Raid5. Just trying to imagine all possible scenarios so I can eliminate them and find the culprit.

 

[KempelofDoom]

Would I be able to make due by adding an SSD to the mix? Since it's unlikely that I'll be able to get RAM in 16GB sticks anytime soon. I'll eventually get a real server motherboard so I can expand to 256GB but that's down the road. I replaced the network card and things appear to be moving quicker but still have some lag.

 

[cyberjock]

As for recommendations for a controller to use, check out the forum stickies. We make a bunch of recommendations. The M1015 is the controller of choice and can do 8 drives out of the box. With a SAS expander(s) you can go up to 32 drives.

Check out my noobie guide(link in my signature). It's a great start for people with questions like yours.

As for SSD, don't do it. Using an SSD is NOT a substitute for RAM. Using an SSD increases the stress on your ARC(which is in RAM). If you don't have alot of RAM to start with you can actually slow the system down. I see it every day where people won't buy more RAM but throw in an SSD and wonder why it got slower.

 

[KempelofDoom]

Thanks again for the advice! I read your guide and see that I just need to get myself a server grade mobo, cpu so I can rock lots more ram than what I am now. What kind of ratio should I try to stick with? I thought it was 1GB ram to 1TB hd but that's not taking into account the 6GB base so maybe 1.1GB to 1TB? So with 128GB of ram, I would maintain 116TB of storage with performance. I also see that I need to start using RaidZ2.

Since SMB is single threaded and that's the service that will be used most do you think that this processor is sufficient?

mobo
proc

Then I would start with 128GB of memory since that is the largest I can get until they make 32BG sticks.

Will the M1015 work in non IBM systems? I assume yes but prefer to be sure before I hunt one down. I also don't have experience using an expander but I do recall seeing that you had one for sale. Would that expander work for me? I currently have this case and will compliment it with this case when I need to expand the amount of storage.

Again, thank you for your assistance!