BUILD Ideas for a 4-Bay NAS box

[grunnsat]

I'm contemplating building a new 4-Bay NAS box to replace my Netgear Readynas NV+. The Readynas is a fine box, but it does not support drives > 2TB, and it lacks iSCSI and IPv6. Furthermore I think ZFS is a better storage solution than ext3 on raid-5.

I've been looking for the hardware components to use, and this is what I'm planning to use:

• The NAS case. There are quite a number of Mini-ITX cases that can be used for a NAS box, but just a few are specifically meant for a NAS box. I've looked at cases from Lian Li and others, and although they were fine cases they didn't offer anything more or better than the well known CFI A7879 case. This box has four bays for disk trays, a small 200W power supply that is sufficient for a NAS box, a slot for a PCI bracket, and two USB 2.0 connectors on the front. It would have been nicer if these USB connectors would have been USB 3.0 connectors, but I don't think there is any NAS box with those.
• The Motherboard. I have not found any Intel motherboard (not just Mini-ITX!) with more than two SATA III 600MB/sec. connectors. All other connectors on such boards are SATA II 300MB/sec versions. However I found the Jetway NF82 Mini-ITX board for AMD processors that has six SATA III 600MB/sec. connectors. It also has two GBe ports, not just one.
• The CPU. The NF82 board has been designed for the AMD R-series APUs (APU = Accelerated Processing Unit = CPU + GPU on one chip). There are two dual core and two quad core versions. he R-464 is the fastest APU with four cores and 4MB cache. The GPU is quite powerful, and it would be very nice if the many integer calculations for ZFS could be done by the GPU. I suppose that is not possible now, maybe in the future?
• the RAM. This boards supports two DDR3 SODIMM slots, so it supports a maximum of 16GB and that is what I want to use.
• The Disks. here are only two major disk manufacturers left, Seagate and Western Digital. Western Digital has two families of drives that are specifically meant for use in NAS storage. The first one is the Red series, but unfortunately there is no 4 TB disk (yet). The second series is the Enterprise class RE series, but those are much more expensive. I hope there will be a 4TB Red series drive.
• The cache drive. ZFS can use an SSD drive for caching. I found a drive that seems very suitable for this purpose. It is a Supersspeed S301 60GB drive, and it is the only family of drives that I know of that uses SLC chips instead of the normal MLC chips. SLC chips are far more resilient against wear, so I expect that this drive is far more durable than any MLC version. I hope to use it as ZIL log device and L2ARC cache device.
• The boot disk. It is possible to use a USB stick as a boot device, but I don't like this as a permanent sollution. Instead I found the DeLock Compact Flash reader with a SATA connector. The CF card will behave like any normal disk.

This is what I came up with so far, and now I would like to have your comments please. Do you think this a good configuration?

 

[jgreco]

SATA II 300MB/sec versions.[...] SATA III 600MB/sec. connectors.

The obvious question, aside from "why are you posting this in Feature Requests", is why you would obsess about this. Since the average hard drive is capable of maybe 150MB/sec, SATA II is fine. Even for an SSD, because with only one or two network ports, your NAS is not serving files at a rate that is hitting the transfer limit of even a single SATA II port.

There are three major disk manufacturers left.

FreeNAS doesn't support mixing ZIL and L2ARC on a single device (though you can make it work). However, the use of SLC is probably a nonissue for a ZIL if you're smart about it. MLC is a ton cheaper for L2ARC, and a competent selection for ZIL is the Intel 320 (about $2/GB) - the trick is to make a small ~2GB partition and use that for ZIL, and then let the drive's wear leveling cope with the endurance issue. This specific product has a lot going for it, including a super-capacitor-equivalent power loss design.

 

[grunnsat]

The obvious question, aside from "why are you posting this in Feature Requests", is why you would obsess about this. Since the average hard drive is capable of maybe 150MB/sec, SATA II is fine. Even for an SSD, because with only one or two network ports, your NAS is not serving files at a rate that is hitting the transfer limit of even a single SATA II port.

Sure, a drive is 'only' capable of 150MB/sec, but that is not a continuous flow of data between the platters and the SATA port. Let's take a read IO as an example. After accepting the read command the disk has to position the heads, wait on avarage half a revolution, read the data from the platters to the disk cache, write the data from the disk cache to the SATA port. So the same data is transferred twice, once from the platters to the cache, and once from the cache to the SATA port. If both operations take the same amount of time, then the data has to be transferred with 300MB/sec per operation in order to get an average data flow of 150MB/sec. And then I even don't take into account the time spend on positioning the heads and waiting for the first block to be read. The WDC RE drives have two processors, so maybe these drives are capable of transferring data between the SATA port and the cache as well as between the cache and the platters at the same time. That can be useful with a queue of disk operations. Writing data to and from a disk is a bursty operation, and the shorter (faster) the bursts are, the more efficient a disk drive can work. So 600MB/sec. ports do make sense.

Writing data to a parity protected file system takes several disk reads and writes, so writing with 100MB/sec. to the network ports will result in a much larger data flow to and from the disks involved.

There are three major disk manufacturers left.

And the third is Toshiba?? Not really a big player on the disk market as far as I'm aware.

FreeNAS doesn't support mixing ZIL and L2ARC on a single device (though you can make it work). However, the use of SLC is probably a nonissue for a ZIL if you're smart about it. MLC is a ton cheaper for L2ARC, and a competent selection for ZIL is the Intel 320 (about $2/GB) - the trick is to make a small ~2GB partition and use that for ZIL, and then let the drive's wear leveling cope with the endurance issue. This specific product has a lot going for it, including a super-capacitor-equivalent power loss design.

The price of these Supersspeed drives is also $2/GB (64GB - $140, 128GB - $250). The SLC cells wear out at about 100.000 operations, the MLC cells at about 15.000 operations, and SLC is much more reliable. You should use a UPS with a NAS box. I don't know if Freenas will enable the disk write cache when there a UPS connected to the NAS box, but that would increase the performance as well.

 

[JaimieV]

I'm contemplating building a new 4-Bay NAS box to replace my Netgear Readynas NV+.

This is exactly the journey I took a couple of years ago. Have you seen the HP Microserver range? They're ideal.

Aside from that, three comments:
* You definitely won't need an SSD ZIL *or* L2ARC (as jgreco says you can't do both in one device) if you're putting it into the same job that an NV+ was good enough for. You can max out GigE without one, and you can painlessly add it later if for some reason it would actually be useful. Generally it won't be unless you have many concurrent users or similar high concurrent loading.
* Don't bother using CF for the boot disk, three quarters of the adapters and/or CF's just don't work out. Use a USB stick like any normal person. Get a mobo with a USB socket inside the box if that concerns you.
* Your argument about the SATA port really isn't valid, it's just not that important unless you have a device capable of shifting data at faster than the port will. So for an SSD, certainly ensure it's on SATA-III. For HDDs, SATA-I is (currently) enough - not that you'd ever get SATA-I any more, nothing new comes with less than SATA-II so you can assume NCQ will be present. Remember that all disk transactions are buffered through ZFS's RAM cache and own queuing method and the 1% of the time that you're losing out on a moment's speed is unnoticeable.

My biggest recommendation: Don't treat a NAS as an opportunity to get all "PC enthusiast build" - aim for *solidity and stability*, then tweak in software for speed if you need to.

 

[jgreco]

Sure, a drive is 'only' capable of 150MB/sec, but that is not a continuous flow of data between the platters and the SATA port.

You're confusing the external connectivity and the internal transfer rate of the disk.

Let's take a read IO as an example. After accepting the read command the disk has to position the heads, wait on avarage half a revolution, read the data from the platters to the disk cache, write the data from the disk cache to the SATA port. So the same data is transferred twice, once from the platters to the cache, and once from the cache to the SATA port. If both operations take the same amount of time, then the data has to be transferred with 300MB/sec per operation in order to get an average data flow of 150MB/sec.

Which would be true if drive operations were carried out synchronously, but that's simply not the case these days. NCQ has been available on high end drives for about a decade now, DMA for even longer. Try it. You can easily get a lot more than 150MB/sec on a SATA-II port, would expect at least 200-250MB/sec sequential.

And then I even don't take into account the time spend on positioning the heads and waiting for the first block to be read. The WDC RE drives have two processors, so maybe these drives are capable of transferring data between the SATA port and the cache as well as between the cache and the platters at the same time.

You do realize that a single processor can be engaged in multiple activities, yes? Even embedded ones?

That can be useful with a queue of disk operations. Writing data to and from a disk is a bursty operation, and the shorter (faster) the bursts are, the more efficient a disk drive can work. So 600MB/sec. ports do make sense.

In much the same way a Bugatti Veyron gets you across the country faster, yes. It may be capable of more speed, but the practical realities don't actually result in a net improvement (bad pun!).

Perhaps I'm the only person that sees someone obsessing over whether a board has SATA-II vs SATA-III, but has picked a board with DUAL REALTEK GIGABIT ETHERNETS.

Writing data to a parity protected file system takes several disk reads and writes, so writing with 100MB/sec. to the network ports will result in a much larger data flow to and from the disks involved.

ZFS works differently. Some other ways in which it works also impact some of the other misconceptions you appear to have.

And the third is Toshiba?? Not really a big player on the disk market as far as I'm aware.

Well, update your awareness. They're a large player, especially on the high end, enterprise, and notebook drives. Just because you don't see them when you go to Best Buy doesn't mean they aren't significant.

The price of these Supersspeed drives is also $2/GB (64GB - $140, 128GB - $250). The SLC cells wear out at about 100.000 operations, the MLC cells at about 15.000 operations, and SLC is much more reliable.

If you can find it for sale at that sort of price, then yes, it seems interesting. I might still prefer the Intel product because of the proven reliability of the product (consider OCZ's track record for example), but I have to concede, that's a promising-looking product.

JaimieV's post is pretty excellent, except that I would say for iSCSI, I could see the need for more CPU than the MicroServer, and there could be a need for ZIL if you wanted to go sync=always for your iSCSI, which we discuss around here every now and then.