We have done some testing on the follwing hardware.
sidenote: we are not able to install Freenas using the CD to an attached USB Stick
First Server
Dell PowerEdge 2900
6x 73 gig 15k 3.5" drivesm, RAID 10 (software) {Does not count system drives, no spares} {15k 3.5" ZIL drive}
16 gig RAM
Dual dual core 3.2 Ghz Core2Duo CPUs
The t610 hardware configuration-
dual hex core 2.8 ghz CPUs (HT enabled)
144 gig 1333 mhz RAM
h200 controller
4x 7.2k ES drives, software RAID 0
We are using Raid 0 only for testing to emulate a possible Raid10 config for the final server
maybe a R720xd from Dell
First test with the PE2900 was saw the following
Copying a 2 gig file over the network to the lab SAN and to the FreeNas box shows they run at the same speed, or at least close enough (~100-120 meg/sec).
Copying a 2 gig file file from the lab san over the network goes at about 65 meg/sec. The same file copying from the freenas box over the network to the same server goes MUCH slower- I stopped the first attempt because it was running in kb/sec. The second run started at 5 meg/sec but sped up as it got through the file to 36/meg sec, almost like it had to load the file in to a buffer before it could serve it properly
Likewise copying files between the two SANs seems inconsistent- it will also be fast copying files to the lab SAN but not always fast going to the Freenas server. But sometimes the performance is the same; This seems to be related to the size of the data set- under 2 gig is fast in both directions, but much over 2 gig is slow going to the Freenas system.
Deleting a large file from both units- the lab san deleted a 2 gig file pretty much instantly, but it took 15 seconds on the freenas box.
Some of the Intel tests show that while the lab SAN has consistent performance, the raw throughput to the freenas box is faster for many workloads but is very bursty- e.g., it will move a lot of data and then do nothing for a bit, then move another large amount of data.
Running an iometer over the network shows that the freenas box is 4x faster than the lab SAN in terms of raw IOPS.
I tried copying very large files, though, and that's where we see the biggest difference; a 7 gig file copies at ~90 meg/sec to the lab SAN but after the first gig or two of the file to the freenas box starts slowing down, and keeps getting slower. It looks like a buffer can fill up and take the system to it's knees. At 40% of the way through the copy I stopped it because it was down to 17 meg/sec and dropping.
All of this, by the way, is using NFS so this isn't samba tuning.
on the T610
Copying our 7 gig reference file does basically the same thing in the same place as the 2900 based Freenas server- it gets a little more than 2 gig in to the file and the copy speed drops precipitously. Left to itself, by the time it finishes copying it is running slower than a copy to a slow USB thumb drive (10 meg/sec)
any Idea what is going wrong here, that Freenas slows down ?
sidenote: we are not able to install Freenas using the CD to an attached USB Stick
First Server
Dell PowerEdge 2900
6x 73 gig 15k 3.5" drivesm, RAID 10 (software) {Does not count system drives, no spares} {15k 3.5" ZIL drive}
16 gig RAM
Dual dual core 3.2 Ghz Core2Duo CPUs
The t610 hardware configuration-
dual hex core 2.8 ghz CPUs (HT enabled)
144 gig 1333 mhz RAM
h200 controller
4x 7.2k ES drives, software RAID 0
We are using Raid 0 only for testing to emulate a possible Raid10 config for the final server
maybe a R720xd from Dell
First test with the PE2900 was saw the following
Copying a 2 gig file over the network to the lab SAN and to the FreeNas box shows they run at the same speed, or at least close enough (~100-120 meg/sec).
Copying a 2 gig file file from the lab san over the network goes at about 65 meg/sec. The same file copying from the freenas box over the network to the same server goes MUCH slower- I stopped the first attempt because it was running in kb/sec. The second run started at 5 meg/sec but sped up as it got through the file to 36/meg sec, almost like it had to load the file in to a buffer before it could serve it properly
Likewise copying files between the two SANs seems inconsistent- it will also be fast copying files to the lab SAN but not always fast going to the Freenas server. But sometimes the performance is the same; This seems to be related to the size of the data set- under 2 gig is fast in both directions, but much over 2 gig is slow going to the Freenas system.
Deleting a large file from both units- the lab san deleted a 2 gig file pretty much instantly, but it took 15 seconds on the freenas box.
Some of the Intel tests show that while the lab SAN has consistent performance, the raw throughput to the freenas box is faster for many workloads but is very bursty- e.g., it will move a lot of data and then do nothing for a bit, then move another large amount of data.
Running an iometer over the network shows that the freenas box is 4x faster than the lab SAN in terms of raw IOPS.
I tried copying very large files, though, and that's where we see the biggest difference; a 7 gig file copies at ~90 meg/sec to the lab SAN but after the first gig or two of the file to the freenas box starts slowing down, and keeps getting slower. It looks like a buffer can fill up and take the system to it's knees. At 40% of the way through the copy I stopped it because it was down to 17 meg/sec and dropping.
All of this, by the way, is using NFS so this isn't samba tuning.
on the T610
Copying our 7 gig reference file does basically the same thing in the same place as the 2900 based Freenas server- it gets a little more than 2 gig in to the file and the copy speed drops precipitously. Left to itself, by the time it finishes copying it is running slower than a copy to a slow USB thumb drive (10 meg/sec)
any Idea what is going wrong here, that Freenas slows down ?