Small footprint ... what's the buzz?

[EasyGoing1]

Hello,

... So I ALMOST pulled the trigger today on a $1200 Synology box then I stumbled on truenas.com and remembered playing around with FreeNAS back in the mid 2000's I think ... so I figured I would check it out, downloaded the ISO, and loaded it up in VMWare Fusion and I have to say I'm fairly impressed with it. Looks like it even comes with a virtual machine server built-in? Or is that just a management interface for virtual servers?

Anyway ... I want to look at specing out a small footprint box - like something similar in size to the Synology DS1621 ... or the Drobo 5 bay or the QNAP 5 bays ... that kinda size ... what's the buzz right now on that kinda hardware? The box will just be for me and my MBP 16" ... I was going to go with a 10Gbe NIC in the Synology and a Thunderbolt 3 10G adapter for the mac since Synology doesn't do or support DAS ... but perhaps it might be possible to build a truenas with a fast DAS interface instead of ethernet? After all ... thunderbolt is a heck of a lot cheaper than 10G ethernet.

Any feedback or links would be much appreciated.

Thank you,

Mike

 

[Samuel Tai]

You should look at the TrueNAS Minis: https://www.truenas.com/truenas-mini/.

 

[Patrick M. Hausen]

DAS instead of Ethernet is not an option. An alternative to buying from iX is the Supermicro SC721TQ-250B which can take

• 4x 3.5" HDD in hot-plug trays
• 2x 2.5" HDD or SSD internal
• 2x NVME M.2 on the special AOC-SLG3-2M add-on card if the mainboard supports that
• 1x NVME or SATA M.2 on the mainboard if the mainboard supports that

for a total of 9 devices in a small "cube" case.

See the "Main private NAS" in my signature for all details. The Supermicro case is available separately to fit with a Mini-ITX mainboard of your choice or as a barebone with various mainboards. For my main NAS I picked the 5028D-TN4T barebone, while my backup NAS is the mentioned case that I fitted with the A2SDi-8C+-HLN4F mainboard myself.

The former features 2x 1G and 2x 10G, while the latter comes with 4x 1G, only.

HTH,
Patrick

 

[Samuel Tai]

If you're still leaning towards DAS, you should look at https://www.owcdigital.com/products/external-storage. In particular, the ThunderBays. The OWC Callistos are storage servers which appear to be running a modified FreeNAS 9.x UI, and run ZFS under the hood.

 

[EasyGoing1]

If you're still leaning towards DAS, you should look at https://www.owcdigital.com/products/external-storage. In particular, the ThunderBays. The OWC Callistos are storage servers which appear to be running a modified FreeNAS 9.x UI, and run ZFS under the hood.

My issue is that my MBP is a 2019 model ... and I have an internal 1TB SSD and my virtual machines are kept on an external Samsung SSD that uses USB-C and the transfer speeds are off the hook blazing fast ... my current external 8TB drive got partially corrupted to where I had to use recovery software to get back the data that just ... vanished ... and now I'm in a situation where I need to make a choice. I can't seem to repair all the damage that happened on the external APFS volume... I've been trying ... but nothing seems to be able to fix it, so I need to move the data to a healthy partition, and there's a lot of data, so naturally, something like RAID-5 just seems to make sense.

Having it directly attached means I can keep my transfer speeds to what I'm used to ... but at the same time, I can't imagine that 10G Ethernet would - in any way - suck ... so if I'm going to go with an Ethernet solution, then it's going to be a 10g solution without exception. If I'm going to go with DAS then it has to support modern USB-C speeds ... theoretically, 10Gbe and 10G-USB-C shouldn't be all that different when it comes to everyday use and how the data speeds "feel"...

I like the idea of having a NAS and if it's done right, I can offload my virtual machines to the NAS and upgrade my MBP to an M1 (though I'm hesitant to do that) ... having a NAS would also - in theory - give me some headroom when I need to run processes that can sometimes take days to complete. But I haven't had a NAS in over 10 years so ... I'm kind of used to not having one ... so at this point, I think that price will be my deciding variable. If I can build a decent NAS that can run one or two virtual machines without a hiccup and give me data transfer speeds at or near USB-C levels for around the same price that I can buy a DAS for (which would not give me the extra capability- yet would offer nice transfer speeds) ... then I'll go for the NAS - which I seem to be leaning towards anyways ... the other nice feature of the NAS is that my laptop would not be involved in maintaining the volume integrity ... and I like that idea ... a lot... since that seems to be what got me in this mess in the first place.

 

[EasyGoing1]

DAS instead of Ethernet is not an option. An alternative to buying from iX is the Supermicro SC721TQ-250B which can take

• 4x 3.5" HDD in hot-plug trays
• 2x 2.5" HDD or SSD internal
• 2x NVME M.2 on the special AOC-SLG3-2M add-on card if the mainboard supports that
• 1x NVME or SATA M.2 on the mainboard if the mainboard supports that

for a total of 9 devices in a small "cube" case.

See the "Main private NAS" in my signature for all details. The Supermicro case is available separately to fit with a Mini-ITX mainboard of your choice or as a barebone with various mainboards. For my main NAS I picked the 5028D-TN4T barebone, while my backup NAS is the mentioned case that I fitted with the A2SDi-8C+-HLN4F mainboard myself.

The former features 2x 1G and 2x 10G, while the latter comes with 4x 1G, only.

HTH,
Patrick

Hi Patrick,

I like that case ... I've been looking on Amazon for HOURS and I can't seem to find a case that is similar to what the commercial NAS products offer... but that 250B looks pretty decent. I've also got my NVME drive from my 2013 MBP ... the motherboard went sound on me earlier this year prompting me to get a 2019 model, but the SSD drive is still good and half a TB ... it would be a decent front end cache for spinning metal for sure.

 

[rvassar]

A couple thoughts:
1. The jump from 1GbE to 10GbE is significant. 10Gbe is generally faster than most non-SSD pools set up for performance in the device counts you mention. A 4 to 5 device RAIDz/RAIDz2 style ZFS pool isn't going to get anywhere near 10GbE write speeds. The NVMe drive by itself isn't going to accelerate a spinning rust pool. By itself it's a single point of failure, and there's no "write-back" cache mechanism in ZFS beyond system RAM.

2. I'm always amused at the thought everyone puts into cases, and this is almost certainly on me. I just want something that works and holds the parts I have. TrueNAS is an appliance built on FreeBSD. Once you set it up and have it running, it's very much like the Synology & QNAP devices you mention. You almost never need the console. Pick something cheap, with the drive bays you need, set the BIOS to boot without a keyboard, and hide it in a corner. I picked a cheap gaming case with lots of airflow / fan mounts. I leave the VGA cable attached to mine, so I can bring a monitor over and connect it when I need to look.

 

[ChrisRJ]

@rvassar , in principle I agree with you on cases. But many people have severe constraints in terms of space, optics, noise because accommodation cost is closer to Manhattan than where we live ;-)

 

[danb35]

If you can live with four bays rather than five, the HPE MicroServer Gen10+ is a nice little piece of kit. It doesn't do hot-swap though.

 

[EasyGoing1]

A couple thoughts:
1. The jump from 1GbE to 10GbE is significant. 10Gbe is generally faster than most non-SSD pools set up for performance in the device counts you mention. A 4 to 5 device RAIDz/RAIDz2 style ZFS pool isn't going to get anywhere near 10GbE write speeds. The NVMe drive by itself isn't going to accelerate a spinning rust pool. By itself it's a single point of failure, and there's no "write-back" cache mechanism in ZFS beyond system RAM.

2. I'm always amused at the thought everyone puts into cases, and this is almost certainly on me. I just want something that works and holds the parts I have. TrueNAS is an appliance built on FreeBSD. Once you set it up and have it running, it's very much like the Synology & QNAP devices you mention. You almost never need the console. Pick something cheap, with the drive bays you need, set the BIOS to boot without a keyboard, and hide it in a corner. I picked a cheap gaming case with lots of airflow / fan mounts. I leave the VGA cable attached to mine, so I can bring a monitor over and connect it when I need to look.

I'm right there with you ... aesthetics are pointless in a NAS ... performance matters but maybe even more than that is RESILIENCY ... they run 24/7 so they better hold up and be reliable ... but I live in a 1 bedroom apartment and I've already got too much crap as it is ... so compact is a BONUS for me. And I've been in IT for ... well... a long time now ... and if there is one thing I've learned - the hard way - is that you can NEVER go overboard with your bandwidth - be it Internet or your data bus ... the more room you have the better the hardware performs - it's just the way it is and it might defy common sense or conventional thinking because the numbers are the numbers ... but you actually FEEL what a computer is like to use that was designed right at its rated numbers then you go use one that has way too much bandwidth in it all around and you tell me which one is more pleasant to use... My MBP internal data speeds test out at ... well ... ill put up a screenshot... and when you get used to how a computer feels at those speeds, and you're gonna drop a grand or more into some storage ... I'd kick myself for not going 10G - I know I would.

 

[EasyGoing1]

If you can live with four bays rather than five, the HPE MicroServer Gen10+ is a nice little piece of kit. It doesn't do hot-swap though.

Heeyyyy ... thats a NICE server ... they even have a 16 gig ram version for just under $900. And yeah, I think I can live with 4 bayds, you only need three for a RAID-5 and I planned on dropping in 12TB drives into the unit .... if I can't live with 30TB of storage then someone needs to have an intervention with me cause there would be something wrong with me for sure.

Hot swapping ... I'm not hosting a million user banking site... I think I can afford to turn the thing off if I need to pull a drive... the Nas is only for me so ...

We can get a single 3.5" 20TB hard drive today for less than a thousand dollars ... I bought my first hard drive in 1991. I paid $320 for it and it was 20 MEGABYTES ... at that price per meg, a 20TB drive in 1991 would cost close to 300 MILLION dollars ... just stupid amazing how far we've come.

I'll look at that HP some more ... the price and the size are right... if It can support 10Gbe and hold up well with a couple of virtual machines running, that might be the winner.

 

[danb35]

if It can support 10Gbe

It can; I have a 10 GbE card in mine. I'm not using mine as a NAS; I'm using it as a small node for my Proxmox cluster. It is only four cores, with no hyperthreading, so plan appropriately.

I'm also using a Microserver Gen8 as a NAS. It isn't quite as small as the Gen10+ (same basic footprint, but taller), but can be considerably cheaper. Again, four bays, no hot-swap. And it is running a Ubuntu VM for Pi-Hole. Both are pretty quiet, though not silent.

 

[rvassar]

the more room you have the better the hardware performs - it's just the way it is and it might defy common sense or conventional thinking because the numbers are the numbers ... but you actually FEEL what a computer is like to use that was designed right at its rated numbers then you go use one that has way too much bandwidth in it all around and you tell me which one is more pleasant to use... My MBP internal data speeds test out at ... well ... ill put up a screenshot... and when you get used to how a computer feels at those speeds, and you're gonna drop a grand or more into some storage ... I'd kick myself for not going 10G - I know I would.

I'm ex-Sun Microsystems, ex-HGST, ex-a couple cloud companies, and currently employed in storage R&D, etc... I have 10GbE here at home in my homelab / home office, which goes mostly under utilized. Much like you, I did it because it wasn't that expensive, and I would have kicked myself if I didn't...

Understand, I'm trying to convey a very specific technical point. RAIDz/RAIDz2 is not really that different from RAID5/6 in terms of the performance rate limiting metrics. Yes, ZFS will make heavy use of the ARC, do CoW, etc... But at the end of it, I see you trying to match your ~2700 MB/sec SSD read speed to a NAS with 4 or 5 spinning disks. Parity stripe reads will tend to a fractional multiplier of the number spindles required to reassemble the data, less some calculation overhead. So if your drives can read at 200MB/sec, your three drive RAIDz/RAID5 like pool will top out somewhere around perhaps ~300 - 350 MB/sec on a good day. But the devil is in the write performance. Your 3 disk RAIDz/RAID5 has to commit each sector written to all devices in order to meet redundancy requirements. This means in effect you end up with the write speed to the NAS that's the same as the speed of a single device in the pool. If the slowest drive in your pool writes at 120MB/sec, that's all you're going to get. 1GbE ethernet gets you maybe 70MB/sec, and 10GbE maybe 700MB/sec if it's tuned right and everything keeps up. So you'll have ~6+ times as much network bandwidth as your NAS can utilize, and still fall an order of magnitude short of your Mac's NVME SSD. 10GbE on small spindle count single vdev ZFS pools hits the wall really really quick.

 

[Patrick M. Hausen]

Can the Microserver Gen10 connect a bootable small SSD in addition to the 4 removable disks? In the Gen8 all you have is the internal USB header, the CD/DVD-ROM SATA port does not support boot from an SSD, IIRC.

 

[danb35]

Can the Microserver Gen10 connect a bootable small SSD in addition to the 4 removable disks?

The Gen10+ has an internal USB-A port, just like the Gen8, so I'm sure the same kind of USB-SSD adapter could be used. But there isn't a dedicated bay or anything similar.

 

[Patrick M. Hausen]

No unused SATA port? Or M.2? One can dream, right?

 

[danb35]

No unused SATA port? Or M.2? One can dream, right?

Nope, neither of those. It'd be nice, but I'm not sure there's the spare motherboard real estate for the latter, and there definitely isn't the former.

 

[EasyGoing1]

So you'll have ~6+ times as much network bandwidth as your NAS can utilize, and still fall an order of magnitude short of your Mac's NVME SSD. 10GbE on small spindle count single vdev ZFS pools hits the wall really really quick.

300MB/s read for external storage is JUST FINE ... We use to have to tolerate 30 MB/s for internal storage not that long ago. Frankly, I don't understand the point of even discussing 1Gig ethernet vs 10 Gig when clearly, 10G will make sure that nothing is under-utilized.

 

[EasyGoing1]

Nope, neither of those. It'd be nice, but I'm not sure there's the spare motherboard real estate for the latter, and there definitely isn't the former.

It lists all of its USB ports as 3.2 ... or is that 3.0 Gen2? hard to keep up anymore... but I read that as 10 gig bandwidth on the USB ports ... and I've got a couple of external USB-C SSD drives that click along ... well ... here's the numbers on those drives ...Those are the speeds I got on my 2013 MBP internal storage ... well close ... Something like that as boot drive would be fine I would think.

There are two things about the HP server that are at least points of interest ... first one is ... no hardware raid? and second, is how they will handle virtual machines. But I'm wondering if hardware RAID is even something that home NAS boxes typically even do? Is it pretty much software raid these days?

 

[danb35]

no hardware raid?

If you're going to be running TrueNAS, you don't want hardware RAID.

how they will handle virtual machines.

I'm running a couple on mine without a problem--but again, only four cores, with no hyperthreading. I'm considering a CPU swap for mine at some point.