With a m.2 drive hitting $70 (USD), would you consider one as a boot drive for your build?

[Mark Holtz]

Here is some food for thought...

As we all know, as part of a FreeNAS configuration, you have to dedicate a device as a boot drive, usually a USB stick or a SATA drive. As currently implemented by FreeNAS, that drive can't be used for anything else, so if you (accidentally) set up one of your 5TB NAS drives as the boot drive, you can essentially kiss over 4.9 TB of storage goodbye.

Some of the newer server motherboards, including the Supermicro X11SAE-M, include an m.2 port (PCIe x4 only) as well as the SATA ports in their configuration. Now, until recently, the m.2 drives have been a tad expensive, and only worth it if you are going to use all of the available SATA ports for NAS drives.

However, something caught my eye this morning while reviewing the emailings from Newegg. There is now a Intel SSD 600p m.2 drive that Newegg is advertising for $69.99 as of this writing. This definitely caught my eye. I use a m.2 drive in my build, but the Samsung SM951 cost me $100. The drive shows up as a nvd0, and does what I need to do.

Again, the $70 (yes, I'm rounding) price point is catching my eye. The two comparable alternatives is either using a SATA DOM (I see a 16GB Supermicro for $47) or a SATA SSD drive. (Prices around $30-$40 for a smallish drive). However, those take up a SATA port, which means one less port for a NAS drive.

My own experience with a m.2 drive in my build has been mostly positive. The reason why I went with one in the first place is that I wanted to have eight NAS drives in my build, thus no SATA ports available for a boot drive. I didn't want to go near a USB boot drive. One of the biggest drawbacks is that the smallest size for a m.2 drive (PCIe x4 is the only ones that are pinout compatible for the motherboard) is 128GB. That means that over 120GB is "wasted" and unused. As far as I know, there isn't a way to partition this drive so that I can use part of the drive as a 16GB boot drive and the other 112GB as a write cache (not that I need that anyways for my setup). Also, I don't reboot the FreeNAS box on a regular basis beyond what is required for system updates, so the benefit of faster boot drives isn't there. And, looking at the FreeNAS logs, that m.2 drive is rarely touched (for what reason? Dunno) during normal operations.

Still, food for thought. If you were building a new FreeNAS box, would you consider a m.2 drive as a boot drive? Perhaps, some thoughts can be put into the build stickies as well.

 

[Mirfster]

While I don't use these drives, I would venture to say that if they work there is nothing wrong with that. I personally prefer to use SSDs (mirrored) and am willing to sacrifice the ports/slots.

However if one is maintaining regular configuration backups there is really nothing wrong with having only one boot device. Unless, it is a "Production System" and up-time is of significant importance.

At $70.00 a pop, it does seem a little high to me when (I can get two used 40GB SSDs for ~$44.00). Again though that is due to my willingness to sacrifice ports.

*** Side note, since I use all Dell C2100/FS12-TY Servers with HBAs the 6 Motherboard SATA Ports are not used for my Storage Drives. Mainly this is due to the fact that they are 3Gbs instead of 6Gbs that the backplane and HBAs run at.

All in all, the beauty and curse of FreeNAS is that it is so darn versatile that each person's build can be a unique configuration unlike anybody else.

 

[joeschmuck]

My opinion is I currently prefer the traditional SATA port connection because I can move the drive to other computers without some interface adapter. Being able to reconfigure hardware is important to me. From a mirrored boot drive perspective, I'm against that entirely, keep a backup of your configuration file like you should and you can easily rebuild your system. The mirrored drives thing came in because of failing USB Flash drives and this was the best a person could do but there is no real booting failover. I have installed a true RAID card for booting my ESXi server and a mirrored pair of SSDs for booting and running the system. The RAID card will allow a true failover should a drive failure occur. And again, this is my opinion, I'm not throwing stones at anyone who does otherwise.

Taking my preference out of the picture, when it comes to M.2 interfaces, well there are three types that I'm aware of which have PCI-E, SATA, or both. Some motherboard will kill a SATA port when you use M.2 card, some will not. As far as I know, if you use the PCI-E port and card, then your SATA ports should be unaffected however I heard last week that someone has a PCI-E M.2 slot only and it still took away a SATA port so I'm not sure what is going on there, plus I didn't personally verify the hardware in use.

If someone is trying to make as small of a footprint as possible, an M.2 card may help.

Cost wise, the M.2 cards still cost more.

So my recommendations are to stay away from M.2 unless you do your homework to ensure compatibility issues with the motherboard and M.2 card, and will it really benefit you.

 

[Arwen]

One thing that can help with reliabilty for largesh single boot drives, is ZFS "copies=2".
That can be a pain to setup with FreeNAS, but whence setup on the top level dataset,
"freenas-boot/ROOT", (or even "freenas-boot"), it should continue to be used through
out any standard updates.

Of course this maybe lost during a FreeNAS train change, like 9.10 to 10.x.

 

[Mark Holtz]

Taking my preference out of the picture, when it comes to M.2 interfaces, well there are three types that I'm aware of which have PCI-E, SATA, or both. Some motherboard will kill a SATA port when you use M.2 card, some will not. As far as I know, if you use the PCI-E port and card, then your SATA ports should be unaffected however I heard last week that someone has a PCI-E M.2 slot only and it still took away a SATA port so I'm not sure what is going on there, plus I didn't personally verify the hardware in use.

This is something that I checked three times in the docs prior to completing my install.

For the Supermicro X11SAE-M, which I am using with my NAS build, no SATA ports are used with the m.2. All eight ports have 5TB drives labeled ada0 through ada7, while the m.2 interface is labeled as nvd0. I had verified functionality during my build.

For my Gigabyte GA-Z97X-UD5H, according to the manual, "M.2, SATA Express, and SATA3 4/5 connectors can only be used one at a time. The SATA3 4/5 connectors will become unavailable when an M.2 SSD is installed." This, of course, is an older consumer-level motherboard released in April, 2014 using the DDR3 memory. DDR3 non-ECC memory. I'm currently running a PNY CS2211 960GB Pro Gaming SSD which cost me $279. A 1TB m.2 drive would cost me $769, while a ~500GB m.2 would cost $280-$310. Yes, I'll take the 1TB SSD please. And, we are still waiting for "SATA Express".

But, then again, we are talking about two different situations. The FreeNAS box is a file server with Transmission and Plex services bolted on the side, and essentially a big fat storage device. My computer has multiple uses (including game playing). Two different universes.

 

[Stux]

If you put the m.2 on a carrier PCIe card (quite cheap of eBay) it won't use any Sata ports and you can pop it in any 4 lane PCI slot and probably get full speed vs perhaps an older mobos 2x m.2/Sata slot