Performance Question - VS traditional NAS Hardware

[DeWebDude]

Hello All,

I want to compare the performance of lets say of a Synology or Buffalo type NAS, lets say some of their mid range stuff, 12 bays etc.
Versus running FreeNAS on dual Quad Core Server with 32GB of RAM, is there even a comparison?

Now my gut says, FreeNAS will smoke it, but here are some things I'm considering.
A standard HP server, with whatever on board is well built, and relatively fast vs the pre-boxed NAS devices which are more tuned to serve one function, be a NAS. Most of that may be about software in which case both are focused on being a NAS.
Assuming same drives, same RAID methods etc, so the drives themselves are not a factor.

Are we really speeding up the process because of our CPU & Backbone vs these devices to the point that it's something measurable, or is this only a way to utilize equipment that we have laying around and solve a need?

Hopefully the question makes sense.

Thanks!

 

[Chris Moore]

Are we really speeding up the process because of our CPU & Backbone vs these devices to the point that it's something measurable,

The advantage of FreeNAS is ZFS, whether or not it is faster depends on the hardware used and what you are doing (iSCSI, NFS, SMB) not every thing you can do will work the same on the same hardware.

or is this only a way to utilize equipment that we have laying around and solve a need?

FreeNAS is absolutely not about being able to use equipment you have laying around. In many situations, the equipment you have laying around is not going to work well with FreeNAS.
You need to tell us what you are trying to do and what hardware you have and people here that have experience will be able to make informed suggestions on whether or not it will work and what you can do to make it work better.

 

[DeWebDude]

Thank you Chris, those are fare statements and questions.
I'm already using FreeNAS on an HP Dual QuadCore server, with the proper controller etc, in a raid 10 configuration.
It's being utilized 100% for iSCSI to support 2 VMWare Hosts, with a handful of servers running on it.

I'm being asked why we used something like FreeNAS vs using something like a Synology.
Because the thought is, that there is a small instruction set for the controller, the argument is that all the extra CPU etc, is not utilized.

This is where my question comes from, to better understand the benefits from the performance perspective.
I do understand the benefits of ZFS, but technically someone could put it on some firmware or even an OS and run the same basic hardware that any NAS manufacturer is using.

Thanks!

 

[Chris Moore]

It's being utilized 100% for iSCSI to support 2 VMWare Hosts, with a handful of servers running on it.

Do you have a SLOG device in this server?
The advantage of FreeNAS is that it allows for a greater degree of customization. At work, we have two QNAP systems, not the little desktop units either, these are the rack mount units with expansion shelves that we changed to run FreeNAS instead of the software that came with them because FreeNAS has ZFS for the integrity of our data and we can configure them in ways that the QNAP software did not permit.

 

[danb35]

the argument is that all the extra CPU etc, is not utilized.

It's probably fair to say that most FreeNAS servers are pretty badly overbuilt when it comes to CPU, but there's really no such thing as too much RAM.

 

[Elliot Dierksen]

It's probably fair to say that most FreeNAS servers are pretty badly overbuilt when it comes to CPU, but there's really no such thing as too much RAM.

The only reason my FreeNAS boxes have dual CPU's is so I can use all the DIMM slots!

 

[danb35]

The only reason my FreeNAS boxes have dual CPU's is so I can use all the DIMM slots!

My box has dual CPUs because it came that way, though a single one would still be far more horsepower than I need. I'm certainly not discouraging beefy CPUs, but FreeNAS as such doesn't need them.

 

[Chris Moore]

I do understand the benefits of ZFS, but technically someone could put it on some firmware or even an OS and run the same basic hardware that any NAS manufacturer is using.

No. ZFS is baked into the OS in the case of FreeNAS. That doesn't matter. ZFS has significant hardware requirements that are about making it work well. ZFS could work with less, but it would impact performance and baking it into firmware wouldn't change that. ZFS uses RAM to cache read and write operations. You might be able to get along with a low power CPU, but that might impact the speed of network communications at some point.
None of the other companies (that I am aware of) are using ZFS in a NAS. I especially don't like the hybrid RAID that Synology uses and QNAP is just using mdadm and a custom version of Linux. In either case, your performance is not because they baked something into a chip, it is because the made a custom OS and run it on purpose built hardware. I don't know about the Synology hardware, but several of the QNAP hardware solutions can run FreeNAS and do a reasonably good job of it.

Depending on the hardware you are using, it might be possible to visualize FreeNAS so they can have those idle CPU cycles for something else and still have FreeNAS for storage, but I don't understand exactly what the argument is that you are fighting.

 

[Chris Moore]

Thank you Chris, those are fare statements and questions.

I thought I would share this with you.
Working my system as hard as possible with regard to disk access, the CPU utilization is fluctuating between 30 and 40%.

The CPU is not usually where the bottleneck lies with FreeNAS.

 

[DeWebDude]

I thought I would share this with you.
Working my system as hard as possible with regard to disk access, the CPU utilization is fluctuating between 30 and 40%.

View attachment 24775

The CPU is not usually where the bottleneck lies with FreeNAS.

Thank you, What application are you using to obtain this info/graphic?

 

[DeWebDude]

Some view the extra work of using FreeNAS as not providing that much more performance vs an off the shelf mid-range NAS.
In my configuration I have seen good performance, but I was unable to compare it to the off the shelf models.

I have not implemented SLOG, which I take it is different from cache drives.
Which method do you recommend to take into consideration as an improvement for the next version of the NAS I'm using?
And in either scenario, I need to use 2 drives for redundancy correct?

( Currently running several disks in raid 10, and will probably re-do that and add some additional disks, so thats where my next version comes in to the conversation ).

Thank you!

 

[Elliot Dierksen]

I have not implemented SLOG, which I take it is different from cache drives.

Yes. The SLOG matters if you are using something that does synchronous writes (like ESXi when accessing FreeNAS via NFS).

 

[Chris Moore]

Thank you, What application are you using to obtain this info/graphic?

In FreeNAS 11.1, there is a service called netdata. I don't think it was available in previous versions of FreeNAS, but don't quote me.
http://doc.freenas.org/11/services.html#netdata

I have not implemented SLOG, which I take it is different from cache drives.
Which method do you recommend to take into consideration as an improvement for the next version of the NAS I'm using?
And in either scenario, I need to use 2 drives for redundancy correct?

Here are some resources that tell more about what SLOG is:

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

The ZFS ZIL and SLOG Demystified
http://www.freenas.org/blog/zfs-zil-and-slog-demystified/

Testing the benefits of SLOG
https://forums.freenas.org/index.php?threads/testing-the-benefits-of-slog-using-a-ram-disk.56561

 

[DeWebDude]

Thank you @Elliot Dierksen & @Chris Moore.
I'm going to look into the SLOG functionality and look to implement it.

 

[Chris Moore]

Some view the extra work of using FreeNAS as not providing that much more performance vs an off the shelf mid-range NAS.
In my configuration I have seen good performance, but I was unable to compare it to the off the shelf models.

There is really no way that I know of to do an 'apples to apples' compairison between FreeNAS and any of the pre-built systems from Synology or QNAP. Partly because they don't give details about their hardware, but also because they custom fabricate the hardware. Part of the reason they custom fabricate is to make it cheaper for them to build. Based on what I found with the QNAP systems that we converted to FreeNAS, it is mostly compatible with standard server gear, but there are a number of corners cut that make it less than ideal. If you are buying a new server to put FreeNAS on or a new appliance from Synology or QNAP, the price will probably be about the same and the only way to measure the difference in performance would be to buy them both and test. For me, it boils down to ZFS; the other guys don't use it and FreeNAS does.

 

[DeWebDude]

From a networking perspective, I wanted to ask this performance question.
Scenario:

Server A - NetCard1A - on subnet 10.1.1.0
- NetCard2A - on subnet 10.1.2.0

Server B - NetCard1B - on subnet 10.1.1.0
- NetCard2B - on subnet 10.1.2.0

FreeNAS - NetCard1FN - on subnet 10.1.1.0
- NetCard2FN - on subnet 10.1.2.0

For MPIO we connect all the Cards named 1
- 1A, 1B to 1FN all on subnet 10.1.1.0
- 2A, 2A to 2FN all on subnet 10.1.2.0

This gives the servers each 2 paths from each server to the NAS

Scenario 2:
In many other NAS environments the subnet is the same, so everything on 10.1.1.0

This allows me the following:
- 1A, 2A to 1FN and 2FN on subnet 10.1.1.0
- 1B, 2B to 1FN and 2FN on subnet 10.1.1.0

This gives me 4 paths from each server to the NAS

Based on that, we should have better performance in scenario 2, now I understand lets say these are all 1GB cards and thats our main bottleneck, but there are performance benefits from the MPIO perspective.

Should this even be a factor?
Is there something measurable here?

Thanks!

 

[Elliot Dierksen]

From a networking perspective, I wanted to ask this performance question.

(detail on scenarios removed for brevity)

There are some pretty big 'yeah but's on this. The first is that any IP connection has a single source and a single destination. It is certainly possible within an application to create something the multiplexes across multiple sources and destinations, but that is very application specific. In the scenario with multiple NIC's on the same IP network, you have to do some kind of aggregation and a LAGG is the ideal way to do this. However, a LAGG does not actually bond channels together. What it does is load balance across the multiple paths so that you SHOULD have an aggregate bandwidth of all the links, but any single conversation can get no more than the bandwidth of a single link. I have ranted a bit in this subject in some other threads, but let me try to distill it down as much as I can. I would leave aggregation out of your testing. Figure out what you can do to try and approach 90-95% utilization on a single link. If you can achieve that, you are doing pretty darn well. If you are testing, you need to have some testing that doesn't rely on the storage. Something like iperf is good, and I am sure there are other alternatives. You are only as fast as your slowest component. It could even be that the slow down could be writing data to the disk of the client machine (server A and server B in your example). First see what you can do just on the networking side. Then start bringing in the storage components.

I hope that all makes sense.