FreeNAS8 performance vs Synology

[Bulletproof]

I picked up a Synology DS1512+ (5 Bay Dual Ethernet) which works really well. I am able to saturate a single Ethernet while using RAID 6 and only WD 3TB Red Drives, coming from Raptor/WD Black only this seems surprisingly good. With nic aggregation, I'm able to get over 200MB/s consistently and about 105-116MB/s with a single operation.

I am finding somethings with the device that are bothering me. Most importantly lack of snapshots typically native to ZFS. The built in backup options are very basic and don't do a good job for my needs. You can root the box and customize it but I don't think there is an option to get snapshots or ZFS.

I used FreeNAS ages ago and I liked it, but it never seemed to perform all that well. It seems to have matured quite well over the years and ZFS is a very attractive option over just rolling a box and configuring everything from scratch.

I am looking for Windows, Mac, NFS/iSCSI (most likely just NFS).
I have five WD 3TB Red Drives I would want to use. Will I be able to get similar performance with FreeNAS & ZFS?
Any suggestions on a good small form factor 5+ bay hot swap box with cpu/memory. I really like the Synology form factor, and one of the reasons I didn't roll my own solution. But before I get too dependent on the box, I want to look into getting something with ZFS snapshots and better backups (encrypted rotating USB3/eSata external).

Thanks!

 

[jgreco]

You're not going to find something competitive with the Synology in terms of form factor. Sorry, that's just the way the PC world is. For four bays with trays, there's stuff like the ES34069. For six bays without trays, there's the Fractal R2 Mini, which is about as optimum as you can get for disks-per-cubic-inch for PC's. For eight bays, there's the Fractal R2. Personally, I prefer trays but I also prefer rackmount, which gives me a different set of much more expensive options, but when you compare to the NAS vendor rackmounts, still MUCH cheaper.

A device like a Synology has been designed and tuned to provide good general performance in its expected configuration, but FreeNAS has no such luck: the possible configurations are endless. Performance with FreeNAS is always going to be highly dependent on the choices you've made, the gear you've picked, the way you've configured your pool, etc.

For example, right now, on the bench, I'm trying to recreate and mitigate some performance issues noticed in #1531. A relatively big box, a Xeon E3-1230, 32GB RAM, with four 2005-vintage Barracuda 400GB drives. The system is capable of completely slagging and saturating the drives, but doesn't tune itself accordingly, so you get some really poor performance characteristics (drives each capable of ~50-60MB/sec, 4 of them in RAIDZ2, so ~100-120MB/sec theoretical thruput, and I'm seeing 90, but seeing periods where system is in txg flush mode for a minute or two at a time!)

Your WD Red drives should be well north of 100MB/sec each for read capability, and quite possibly writes too, so given the right configuration in FreeNAS, you ought to be able to do very well assuming no other bottlenecks. There's endless descriptions of the systems people have built and their performance results laying around.

 

[Bulletproof]

You're not going to find something competitive with the Synology in terms of form factor. Sorry, that's just the way the PC world is. For four bays with trays, there's stuff like the ES34069. For six bays without trays, there's the Fractal R2 Mini, which is about as optimum as you can get for disks-per-cubic-inch for PC's. For eight bays, there's the Fractal R2. Personally, I prefer trays but I also prefer rackmount, which gives me a different set of much more expensive options, but when you compare to the NAS vendor rackmounts, still MUCH cheaper.

A device like a Synology has been designed and tuned to provide good general performance in its expected configuration, but FreeNAS has no such luck: the possible configurations are endless. Performance with FreeNAS is always going to be highly dependent on the choices you've made, the gear you've picked, the way you've configured your pool, etc.

For example, right now, on the bench, I'm trying to recreate and mitigate some performance issues noticed in #1531. A relatively big box, a Xeon E3-1230, 32GB RAM, with four 2005-vintage Barracuda 400GB drives. The system is capable of completely slagging and saturating the drives, but doesn't tune itself accordingly, so you get some really poor performance characteristics (drives each capable of ~50-60MB/sec, 4 of them in RAIDZ2, so ~100-120MB/sec theoretical thruput, and I'm seeing 90, but seeing periods where system is in txg flush mode for a minute or two at a time!)

Your WD Red drives should be well north of 100MB/sec each for read capability, and quite possibly writes too, so given the right configuration in FreeNAS, you ought to be able to do very well assuming no other bottlenecks. There's endless descriptions of the systems people have built and their performance results laying around.

Thanks for the response. I am ok with rackmount if I have to go that route, I remember there were some good options years ago when I was building storage servers. It is surprising there still isn't a lot of good options. Sounds like performance may or may not be as good. I am going to try to do some tests here with freenas when I get a chance to move the data and pull the drives out. Most of the cases I am seeing are quite expensive.

 

[louisk]

You should read up on how link aggregation works in FreeNAS/FreeBSD before you make your decision. Many people don't read and then complain that it doesn't do what they imagined it would.

 

[Bulletproof]

You should read up on how link aggregation works in FreeNAS/FreeBSD before you make your decision. Many people don't read and then complain that it doesn't do what they imagined it would.

Are you referring to the fact one connection can't saturate both NICS in a single command? Or is there something else?

 

[cyberjock]

Are you referring to the fact one connection can't saturate both NICS in a single command?

That! Too many people don't understand that. :)

 

[Bulletproof]

That! Too many people don't understand that. :)

I understand it, and I have teamed nics in my workstations as well.

 

[louisk]

More that you can't get more than 1 links worth of throughput between any 2 hosts on the network, regardless of how many links are on each end.

 

[cyberjock]

I understand it, and I have teamed nics in my workstations as well.

I'm thinking you don't actually understand it if you have teamed NICs at the workstations as well as the servers. That's what alot of people do, and they all have it wrong.

 

[jgreco]

Well, that depends... two ways to make this work for some scenarios at least.

One can hash based on layer 3 hash calculations (IP only) or layer 4 hash calculations (IP and port). A quick glance at if_lagg.c suggests FreeBSD only does layer 3, which is fine because that's also all that most switches support.

So if you configure your FreeNAS with two IP addresses, and configure some of your connections to go to one IP and some to another, that will give you the possibility that some traffic gets routed over one link and some over the other. The switch may hash things differently, but it is probably a static computation, so if you find a setup that works, you have a better-than-average chance it'll keep working. However, traffic over any single NAS connection will still be limited to gigE.

Or you can configure your FreeNAS to do round-robin lagg. That should get FreeBSD sending packets out both ports, but is essentially mostly-useless unless you have a special case like a direct dual gigE connect to a client, or a switch that'll hash based on L4, or a 10gE client, or some other reason to think that your client can get multiple gigE traffic through your network infrastructure. On a standard switch, using round-robin lagg on FreeNAS merely means you've removed intelligence from the traffic-generating end and are now reliant on packet loss in your network to arbitrate a client's speed, so don't do that.

So in general louisk/noobsauce80 are correct, but there are ways to make specific subsets of the problem work in special cases.

 

[cyberjock]

So if you configure your FreeNAS with two IP addresses, and configure some of your connections to go to one IP and some to another, that will give you the possibility that some traffic gets routed over one link and some over the other.

But that's not what link aggregation is. ;)

You can take 2 dissimilar branded NICs and make that work. In fact, this is exactly what I do at home! It also works far better than LACP could ever have worked for me.

 

[jgreco]

Okay, I'll be pedantic. You can take two interfaces. You can build a lagg0 interface on top of them. You then assign that interface two IP addresses.

I'm pretty sure that the middle bit IS link aggregation. The rest of it is how to engineer around the deficiencies of link aggregation. Since the FreeBSD implementation, and most switch silicon, forms a hash based on the source and destination IP address, one solution to "fixing" link aggregation is to find a way to artificially use more IP addresses. That doesn't mean one IP address on one interface and another on a different one. I meant what I was saying in the context of the link aggregation discussion.

Those of us who use link aggregation in production networks pretty much understand the ins and outs. Generally speaking, the home-user-single-client-NAS-user situation sucks pretty badly because you're looking at that static hash result as a controlling factor. That doesn't mean that there's no way around it though.

 

[c32767a]

It looks like the sinology DS1512+ is a Intel based platform. If you're willing to upgrade the RAM, you might be able to boot FreeNAS on the box and use it that way. I can't tell from the product pictures if they give you a video port.

I took a Qnap TS459 Pro II and converted it to FreeNAS. It was pretty simple to do, I just added an extra 1G DIMM so that I had 2GB of RAM. It performs better and is more stable than it was with the Qnap software. Opening the case and adding the RAM voided the Warranty, but I'm OK with that.. :)

 

[SirHaxalot]

What kind of data are you storing? Do you expect many random reads/writes or mostly sequential? In the later case you will most likely find out that ZFS isn't your friend anymore with NFS. Performance is absolutely terrible, especially for sequential writes. SSD ZIL will be a requirement unless you disable ZIL altogether.

 

[c32767a]

Mostly sequential reads and writes... Home stuff, backups, media files, etc.

Nearly all my devices use CIFS or AFP. Haven't done much at all with NFS on the QNAP platform.