CIFS Write Performance

[SK Enigma]

I am stuck on limited write performance with CIFS on FreeNAS 9.3-RELEASE-P13

Same hardware with Cent OS and Samba 4 I get 100MBs transfer rate through CIFS, Freenas I get 30MBps

I have tried several tunings i could find and I still can't get CIFS to stay above 30MBps consistently. However DD on the local storage gets 200MBPS

Network connections are 1Gbps and 10Gbps

I have attached my debug information

I am almost at my whits end.

 

[SweetAndLow]

have you tried iperf to test your network throughput?
how are you transferring the file? remote dd or transfer an actually file?
how large is the file?
did you do something stupid like jumbo frames or LACP?
Have you tried creating an actually raidZ configuration instead of single disks?
set your cifs max protocol to smb2 and see if that improves performance.
disable all tunables you think you need ans samba config, they won't help you at this point.

 

[cyberjock]

I knew the problem right away, and your debug confirmed it.

You put ZFS on hardware RAID. Nobody has a 60TB hard drive, so window1 is *definitely* hardware RAID. When you do hardware RAID + ZFS you have performance anomalies because ZFS tries to manage reads and writes with respect for time, then your RAID controller tries to do the same thing.

S0 yeah.. READ our manual.. FOLLOW our recommendations, and STOP using hardware RAID!

I have no idea how many disks you have in your "window1", but as a comparison my 6TBx10 array can do almost 1GB/sec with dd tests. No doubt beating your 200MB/sec by a landslide victory.

You also appear to have removed a crapload of files from the debug zip. So I'm not too inclined to provide any more time on this topic as you basically already decided I'm not privvy to read your log files and such.

 

[SK Enigma]

Thank you for your replies. The hardware controller for the disk drives, does not give me a way to NOT have them in an array. Even when I had 24 JOBD arrays I got the same throughput.

The only thing removed from the debug files was the IP addresses or domain names and they were replaced with REDACTED nothing else was changed, replaced or otherwise omitted. If I ran the debug command wrong, I would be happy to run it again.

So on these systems (I have 4) I cannot not use the raid controller. I have read the manual and am working with what I have, and I have followed many recommendations here in the forums.

 

[SK Enigma]

I knew the problem right away, and your debug confirmed it.

You put ZFS on hardware RAID. Nobody has a 60TB hard drive, so window1 is *definitely* hardware RAID. When you do hardware RAID + ZFS you have performance anomalies because ZFS tries to manage reads and writes with respect for time, then your RAID controller tries to do the same thing.

S0 yeah.. READ our manual.. FOLLOW our recommendations, and STOP using hardware RAID!

I have no idea how many disks you have in your "window1", but as a comparison my 6TBx10 array can do almost 1GB/sec with dd tests. No doubt beating your 200MB/sec by a landslide victory.

You also appear to have removed a crapload of files from the debug zip. So I'm not too inclined to provide any more time on this topic as you basically already decided I'm not privvy to read your log files and such.

No files were deleted from the Debug zip. Any assistance is more than welcome, if you have a more detailed debug dump command I can get you, I will be happy to.

 

[titan_rw]

The hardware controller for the disk drives, does not give me a way to NOT have them in an array.

Then you have the wrong controller.

 

[cyberjock]

No files were deleted from your debug? There's a whole directory of files missing from that debug file. Did you make that file from the "Save debug" button under system -> advanced -> save debug?

+1 to what titan_rw said. If you have to do RAID, then your hardware isn't appropriate for FreeNAS. If the data is important, this is non-negotiable.

 

[SK Enigma]

No files were deleted from your debug? There's a whole directory of files missing from that debug file. Did you make that file from the "Save debug" button under system -> advanced -> save debug?

+1 to what titan_rw said. If you have to do RAID, then your hardware isn't appropriate for FreeNAS. If the data is important, this is non-negotiable.

No, I didn't see that option, I ran #freenas-debug -ChTNtze {email address}

 

[cyberjock]

Yeah, please use the save debug button in the future.

In the meantime, getting rid of your RAID controller is a necessary first step.

 

[SK Enigma]

Yeah, please use the save debug button in the future.

In the meantime, getting rid of your RAID controller is a necessary first step.

The RAID controller MegaRAID SAS 2208 is the card connecting the server to the drive chassis. I don't have the authority to replace it :(

 

[cyberjock]

The RAID controller MegaRAID SAS 2208 is the card connecting the server to the drive chassis. I don't have the authority to replace it :(

Ok, then I guess you need to find another OS to use. :(

 

[SK Enigma]

Yeah, please use the save debug button in the future.

In the meantime, getting rid of your RAID controller is a necessary first step.

Here you go

 

[cyberjock]

Thanks for the debug. It really doesn't change the fatal flaw with your box (using hardware RAID with ZFS). If you aren't replacing you RAID controller you shouldn't be using FreeNAS. Even with that aside, your performance is going to be poor because you are using the RAID controller.

So from my perspective it's an open-and-shut case for two reasons.

 

[SK Enigma]

Thanks for the debug. It really doesn't change the fatal flaw with your box (using hardware RAID with ZFS). If you aren't replacing you RAID controller you shouldn't be using FreeNAS. Even with that aside, your performance is going to be poor because you are using the RAID controller.

So from my perspective it's an open-and-shut case for two reasons.

I am trying to get my boss to approve, but would this be a better card?

http://www.amazon.com/gp/product/B007Q1TYTE/?tag=ozlp-20

 

[Ericloewe]

Unlikely.

You want something with an LSI SAS 2008 or 2308 controller.

 

[cyberjock]

I am trying to get my boss to approve, but would this be a better card?

http://www.amazon.com/gp/product/B007Q1TYTE/?tag=ozlp-20

Nope. Check out our hardware recommendations stickies for recommended cards (it's the same that Ericloewe mentioned above).

Virtually anything else that "would work" is going to cost more than the recommended cards.