Performance with new system

[VSXi-13]

Hello,

I just recently put together my first freenas box, after lurking on these forums for quite some time (prior to making an account.) After my initial burn-in testing, I started to load my freenas box with my files. A bit to my surprise, my files ended up transferring fairly consistently between 80-85 MB/s (using CIFS). I have gigabit connections across the board, so I was expecting results in the mid-to-high 90 MB/s range.

NAS:
Asrock C2750D4I
16 GB Crucial 102472BD160B ECC UDIMM
4x 4GB WD Reds in RaidZ2
Silverstone DS380 + Silverstone 450 watt power supply (not that those should effect this.)

PC:
i5 3750K Intel (Windows 7 Pro)
32 GB 1866 G.Skill RAM
Asrock Z77 Extreme6/TB4
Drives: 2TB Seagate, 1TB WD Green, 500GB Samsung Evo

Switch: DLink DGS-1100-16

To rule out drive speed on my PC, I transferred some of my files to the Samsung Evo, then transferred them to my NAS. These were videos ~3GB each. The average transfer speed was around 82 MB/s. One thought I have to the cause of this would be the Broadcom NIC doesn't quite do true 1Gbps speeds?

My other thought was that the 4 discs could not handle that writing any after than that. As such, once the files were loaded, I transferred them back to PC. The performance was similar (80-85 MB/s).

I'm thinking these results are as good as I am likely to get, but I wanted to check. In reviewing a ton of posts both on the Freenas forums and others, I've seen plenty of results much worse than what I'm getting. I wanted to get a feeler out there for others who have a similar build to me to see what their CIFS transfer speeds are.

In the meantime, my next step will be to test the CIFS speeds using a Linux Mint laptop wired in, as I have read a couple of posts from Cyberjock stating that CIFS works better from a linux client than windows.

Many thanks in advance.

 

[cyberjock]

I hate to break it to you, but pretty much anything over 75-80MB/sec is typically considered "good speeds" here. You could try doing Intel on both the PC and server, but I really wouldn't fret your speeds. Most people get 75-80 in best case. Only those that chose to do "everything" right get higher speeds. Typical "everything" means a sufficiently powerful network switch, properly setup NIC settings on the desktop, etc.

 

[VSXi-13]

Cyberjock,

Thank you for your confirmation on that. I thought that was likely the case, but I did want to get the opinion of a more experienced member, such as yourself.

Thanks!

 

[mjws00]

Heh. He has that board in his mini and it did 350 MB/s. 80MB/s is about where I'd be looking for an increase. Takes a few minutes to fire up iPerf. Plug in a network cable directly. Or fire up another OS as you plan. I thought that was a surprisingly pragmatic response. Especially from cyberjock who has seen that board "scream." Personally I'd be pissed and on the warpath looking for more. ;) You have good gear. Is 65% 'good enough' for you? Hope I've planted a seed that will gnaw at your core as it would mine (and I suspect Mr cyberjock). :)

But it's about comfort level looking for tweaks and speed. Day to day when will you feel it? Almost never.

I look forward to hearing how you like that case. I've thought more than once about throwing that exact combo together. If it stays cool with 8 drives that would be awesome.

 

[VSXi-13]

Heh. He has that board in his mini and it did 350 MB/s. 80MB/s is about where I'd be looking for an increase. Takes a few minutes to fire up iPerf. Plug in a network cable directly. Or fire up another OS as you plan. I thought that was a surprisingly pragmatic response. Especially from cyberjock who has seen that board "scream." Personally I'd be pissed and on the warpath looking for more. ;) You have good gear. Is 65% 'good enough' for you? Hope I've planted a seed that will gnaw at your core as it would mine (and I suspect Mr cyberjock). :)

But it's about comfort level looking for tweaks and speed. Day to day when will you feel it? Almost never.

I look forward to hearing how you like that case. I've thought more than once about throwing that exact combo together. If it stays cool with 8 drives that would be awesome.

mjws00,

You're quite right that I do plan on continuing to look into it. I did see Cyberjock's review of the mini and he performance when he put in a 10GbE card into it, which helped me make the choice between the C2750 vs E3 builds. I will admit that I am fairly new to FreeBSD. I took a dive into Linux over the past 3 months, but as it's been mentioned, FreeBSD <> Linux. I would like to squeeze as much performance out of it as I can. The majority of my data is over onto my pool now, so now begins the testing/tinkering stage. I'll run iperf and see what results I get and post back.

In regards to the Case, so far I do like it, however the fans are a wee bit loud. I may be replacing them with Nocturas. During my initial data load, smart reported that my drives were around 41 degrees (I have a notification sent for 40 degrees currently, as I haven't quite managed to work my way around the command line for the smart utility.) It'll be quite awhile before I fill the other 4 hotswap drives, so I'm afraid I can't help you with the normal temps :)

 

[cyberjock]

For me, I got 115MB/sec+ on that board... with Linux on the client. Windows was fluctuating, but generally in the 70-90MB/sec range. If you read my review of the FreeNAS Mini (which uses that board) Linux always outperformed Windows. I also have horribly overpowered network gear, Intel NICs in all of my devices except my laptop, cat 6 everywhere, etc. I've got everything going for me. :P

 

[VSXi-13]

@cyberjock,

I must have missed your gigabit portion of your FreeNAS Mini review. I do remember reading in their that your Linux results were better and saw the differences when using 10GbE card. I'm planning on testing it out using a Linux Mint boot partition on my laptop to see if the speeds are any better (Thinkpad T420S 2.5 GHz i5, 16GB RAM, dual SSD's.)

I also need to do some review to see how close to line speed my DLink DSG-1100-16 switch is to line speed. I feel better knowing that you're also getting between 70-90 MB/s, but I want to see if there is anything else left I can tweak.

 

[cyberjock]

I didn't include values for the Mini with 1Gb since plenty of other people do that. But I use mine at home for testing purposes, so it was trivial for me to run the test just now and give you numbers. ;)

Keep in mind that switch speeds and stuff often matter based on cable length and cable type. I originally had cat5e installed in my house and with one of those "green" switches. My LAN is centered on the south side of the house and I had 2 cables that ran to the north side. Total length was probably 50 feet or so. With "green" switches on both sides I could only get a 100Mb link. As soon as I replaced both with non-green switches everything was 1Gb. I did frequency tests down the length of the cables because I had access to the equipment and I like to know "why" things don't work. The end result is that the green switches don't seem to provide enough voltage and current to make it to the other end.

So when discussing LAN speed problems, you're going to quickly learn that not all is "as it appears". That's why I said when "everything" is perfect. It's not an exact science and takes alot of trial and error to figure out what is wrong (assuming something is wrong).

We've had people that used a program called "Teracopy" that claim to use big buffers and stuff to allow for faster network speeds, but in practice it was slower.... like 30%+ slower.

So yeah, you're pretty much going to have to find out what in particular about your setup is your limiting factor. That's not something you're going to get an answer from here. ;)

 

[anodos]

Your workload will also affect the speeds you see. 4 thousand 1 megabyte files will almost always transfer slower than a single 4GB file. The important thing is stability and adequate speed for your needs. As an adjective 'green' is a synonym for 'sucks'. :)

 

[VSXi-13]

Well, no matter what I did with my laptop, I could not get the transfer rate above 50 MB/s. (Linux Mint, Windows 8.1, changed cables, enabled high power profile.) I'm not certain what the Linux commands would be for adjusting how the adapter behaves. My PC doesn't have a Linux partition currently, so I can't use that for testing.

As for my network cables, they all are <8 ft long CAT5e cables. My entire network is layed out around my desk, with all components nearby. I'm half tempted to get myself an Intel NIC for my PC, but I'm not certain it's worth it.

As for my workload, all of the testing I was doing for transfers were files over 1GB in size. Mostly movies and TV shows in 720p. I'm not quite certain if my switch would be considered a "green" switch, but there are no active fans for cooling and it's not rack mountable (13" vs 19").

A side note: I was backing up my VMware Workstation VM's that I currently have to my NAS box (not to be ran from, simply for a file backup). When I transferred my VMDK, the transfer was going at about 15-20 MB/s, which surprised me. Workstation was completely off and all of the VM's had been shut down prior to me closing out of Workstation. I'll have to do some further research on that portion.

I'll have to keep tinkering away at it. Thanks for all of the replies and tips!

 

[cyberjock]

there's a good chance your VMs are badly fragmented, which would hurt transfer rates.

Also laptops have always and will probably always suck for transfers. The LAN cards that are installed in them aren't meant to be particularly fast. I rarely see more than 50MB/sec, and never seen over 70MB/sec on my current laptop.

I would definitely go with Intel NICs in your desktop. I deliberately buy motherboards with them built-in or I add one myself. I just wish adding Intel NICs to laptops were easy. ;)