First Ever Server Build & FreeNas Newbie

[John3-6teen]

Greetings all from a FreeNas and Server Building Newbie. I am ready to order the components for my first ever home server, but I wanted to run it past this group first for comments and a few answers. I'm not new to computers or building them, just the whole server world. Over the past few months I've been reading, and reading, and reading, and then some more reading, so much that although I now have a fairly good understanding (I think) of what I'm doing I am also beginning to question the validity of some of what I think I've learned. It's all started to run together. Sooo....

I am building this server for my home to share files, backup laptops, tablets, phones, etc. Double as a Home Theater with Plex, Emby, CouchPotato, torrent streams, etc., and our own photos and home videos so it must be able to stream to several devices at once. I would also like to install a personal cloud to access files away from home. I don't mind a little overkill and would rather buy the right stuff the first time that will last for a while and something I can grow with if I decide to do more as I learn what can be done.

Here are the components I'm looking at so far:
MOTHERBOARD: SuperMicro X11SSH-LN4F
I was thinking about the X11SSM-F board, but I can get this one for less than $20 more.
Is there that much advantage to moving up?
Is there any disadvantage to the LN4F board? I know you loose one PCI-e 3.0 x8 slot in exchange for the M.2 connector and with this board you also have the addition of two Intel i210-AT GbE controllers.
Is it better to have the M.2 connector or the additional PCI-3 slot? (thinking of using for boot)
Is the M.2 connector capable of boot?
What are the uses for the additional two GbE controllers for a total of four?

PROCESSOR: Xeon E3-1240 v5 Processor
I was first looking at the E3-1230 v5, but again I can get this one for less than $20 more.
It appears to me that the only difference is in the speed and for <$20 I don't mind the upgrade.
Are there any other difference or known issues with this processor?
I'm still a little confused about the Core i3 vs the Xeon. Is the Xeon the better choice?

MEMORY: Samsung 16GB (1x16) DDR4-2133 (M391A2K43BB1-CPB)
I was going to start with just the one stick and probably add another later. Is there any harm in this approach?
Are there any cheaper options for RAM that are known to work with this board and FreeNas?

BOOT STORAGE: Transcend MTS800 32GB M.2-2280 Solid State Drive
OR: Supermicro SSD-DM016-PHI 16GB SATA DOM
The DOM is about $15 more, not a big deal. I understand that USB as boot is not an option at this time.
Which is the better approach for the boot drive M.2 SSD or SATA DOM?
Is there a better option than either of these?
If M.2 is bootable, and the better choice, is this Transcend SSD a good choice?

I will re-utilize (for now) an ATX case that I have on hand with a 450W PSU.

I also have (2) WD 4TB Red HDD's. I know I will need to pick up some more drives, but I don't quit grasp all the RAID nuisances yet. Is it better to have an odd or even number of drives? Which RAID solution is best suited for my situation? Any truly critical files I plan to also backup somewhere off the server.

I would like to re-utilize some other smaller drives I have sitting around for the Home Theater to use for those movies, tv-shows, etc. I have no desire to keep around after watching them i.e. I don't need to store them for backup or anything else. Is it possible to have these drives operate outside the drive pool for my regular RAID server?

After months of reading and trying to educate myself I'm now anxious to make the purchases and start learning first hand. I plan to build the system and play with the system for awhile before I start storing the important stuff on it in earnest at which time I will reformat and start from scratch if need be.

Thanks for any help and suggestions. Please go easy on the new guy.

 

[joeschmuck]

MEMORY: Samsung 16GB (1x16) DDR4-2133 (M391A2K43BB1-CPB)
I was going to start with just the one stick and probably add another later. Is there any harm in this approach?
Are there any cheaper options for RAM that are known to work with this board and FreeNas?

The cheapest place to buy this in the US is from SuperBiz. I have posted this before a few times. And I would recommend purchasing a pair at a time. Consult the user manual of the motherboard to see if it allows a single stick. I can say that the X11SSM-F will allow a single module to be installed but it MUST be installed in slot DIMMB2.

I also have (2) WD 4TB Red HDD's. I know I will need to pick up some more drives, but I don't quit grasp all the RAID nuisances yet. Is it better to have an odd or even number of drives? Which RAID solution is best suited for my situation? Any truly critical files I plan to also backup somewhere off the server.

Even/Odd, doesn't really matter for your usage. What matters are two factors, configure they system to a RAIDZ2 configuration if you would rather not reinstall all your data in case you have a 2 drive failure (1 drive fails and you replace it but while that drive is rebuilding you have another drive failure, yes it happens) and if you don't mind reloading all your data, RAIDZ1 will do. But we will all talk you into RAIDZ2. And how much capacity you need. Use one of the RAID calculators (two are in my signature) and ensure you choose the correct RAID configuration. This will give you an approximate USABLE storage capacity. So if you are currently needing to store 6TB of storage, then double it and buy enough hard drives. There is more to it than that but this should get you started.

BOOT STORAGE: Transcend MTS800 32GB M.2-2280 Solid State Drive
OR: Supermicro SSD-DM016-PHI 16GB SATA DOM
The DOM is about $15 more, not a big deal. I understand that USB as boot is not an option at this time.
Which is the better approach for the boot drive M.2 SSD or SATA DOM?
Is there a better option than either of these?
If M.2 is bootable, and the better choice, is this Transcend SSD a good choice?

You are better off using a SSD but if you can use a normal SATA SSD, then you will be not committed to a unique form factor and since you said SATA DOM, I'd just shoot for a cheap 60 to 128 GB SSD. When I say cheap, I mean cheap. Visit NewEgg and find something cheap like this (I own the 128 model and it works great, meaning no problems)

 

[John3-6teen]

Thanks joeschmuck for the info. SuperBiiz also has the best price I've found so far on the Processor. I checked the manual and this board does support odd numbers of DIMM's, though you lose the interleaving and hence some performance. I'll probably purchase a pair of the memory for 32Gb total.

I think my storage needs won't exceed 4TB, at least not for some time, so using the calculators in your signature I'll need to add two more drives. Must all the drives be the same size? I assume they do, but now that I'm thinking about it I don't recall reading anything specifically stating that fact.

On the Boot drive what is the advantage of using a 60 to 128 GB drive? I thought the FreeNas uses the entire drive and the additional space cannot be used for any other purpose? Even if it is a cheap drive there are even cheaper drives with smaller capacities. I think I'm missing something here. This would definitely be the cheaper solution though since I doubt I will populate all the SATA ports with data disks, thanks for tip.

 

[joeschmuck]

I think my storage needs won't exceed 4TB, at least not for some time, so using the calculators in your signature I'll need to add two more drives.

So if you currently have two 4TB drives and add two more drives with a RAIDZ2 configuration, you will have approximately 6TB of storage. This does not take into account any compression you might gain from, but if you are storing computer backups and movies, you are not likely to gain a lot in compression. Also, the capacity seems short but that is because you need to take into account that the ZFS filing system significantly slows down when the drives become too full. And I generally think 3 years out because most hard drive warranties are 3 years in length and then you may want to purchase new drives. In my situation I have six 2TB drives with a capacity of approximately 7TB and I store about 5TB on the drives and have for about the last 3+ years. The drives are out of warranty now and with any luck they will last a few more years but I am considering replacing them with 3TB or 4TB drives, the cost will be the factor not the capacity because I realistically will not use significantly more capacity over the next several years.

Must all the drives be the same size? I assume they do, but now that I'm thinking about it I don't recall reading anything specifically stating that fact.

When you create a pool (vdev) the size is limited to the smallest drive in the group. This is because the data is equally striped across all the hard drives with error correction information so that if a drive or two dies, your data is still in tact. So if you purchased two 6TB drives to add to your two 4TB drives, the system will treat them all as 4TB drives. Now if you replace the 4TB drives over time to a larger drive size, once that last drive has been replaced, the pool size will jump up appropriately.

On the Boot drive what is the advantage of using a 60 to 128 GB drive?

No benefit at all. FreeNAS only needs a very small piece of that huge capacity. The entire drive will be formatted but FreeNAS only uses a small section of it.

As for smaller capacity and cheaper drives, if you could buy a 16GB SSD for cheaper, that would be the way to go. I would not recommend smaller than 16GB just because, well that's my gut speaking, I have nothing to base my gut on.

I hope I answered all your questions without too many typo's.

 

[Robert Trevellyan]

I would not recommend smaller than 16GB just because, well that's my gut speaking, I have nothing to base my gut on.

When I recently did a clean install of FreeNAS 9.3 STABLE, it consumed about 500MB. That's why I'm still completely comfortable using 4GB USB sticks for my boot devices (my system dataset is on my pool). Usage will obviously rise with each update (I'm currently at 1GB after 2 updates), but a clean install/restore saved config is quick and easy. Of course, if I were buying new boot devices, it probably wouldn't make any sense to choose 4GB. And, I have no idea how much a clean install of 10.x will consume, or what other factors might indicate using larger devices for that release.

 

[joeschmuck]

Yea, I don't expect FreeNAS 10 to change the requirements much, if at all. But if I were to purchase a new SSD, the smallest I'd feel comfortable with is 16GB but I'd of course prefer 60GB or more depending on the price, price wins all arguments for a FreeNAS boot SSD. Call it future proofing if you like. I myself would like to see a partition created on a SSD boot device that could be utilized for jails or VMs, but only if the device is a SSD. That would be bad news if it were a flash drive. Now I know you can use the partitions on the boot device but that is risky since during an update that data could be destroyed. I played around with that last year and didn't feel comfortable using that space due to the risks.

 

[John3-6teen]

With cheap 32GB SSD's out there, sometimes as low as $20, it seems like a good solution. It would be nice, as you said, if you could just use a partition on a larger drive. I have some larger drives lying around I could put to use and I might just do that for now anyway, though I don't like wasting all that extra space.

Any experience/thoughts on Samsung vs. Crucial RAM? I know that the list for the X11SSH-LN4F doesn't list Crucial, but I've always had good luck with them and they have a pair of 16GB's (32 total) that they list as compatible and there is a cost savings over the Samsung. I'm thinking about giving it a shot unless folks here have had bad experience with it on SuperMicro server boards.

 

[ChriZ]

Quite the opposite, actually...

 

[Arwen]

...
I would like to re-utilize some other smaller drives I have sitting around for the Home Theater to use for those movies, tv-shows, etc. I have no desire to keep around after watching them i.e. I don't need to store them for backup or anything else. Is it possible to have these drives operate outside the drive pool for my regular RAID server?
...

The simplest way to deal with these, is to use a separate striped Pool. Basically create a
pool without any redundancy. Great for scratch data, and data that can be re-created.

If / when you get bad blocks, and if they are data blocks, ZFS will tell you which file(s) are
damaged. (Meta-data is always redundant even on non-redundant Pools). You do loose all
data on loss of an entire disk. Then the pool has to be re-created. Either without the failed
disk, or with a replacement disk. Last, re-populate your scratch pool.

 

[joeschmuck]

Any experience/thoughts on Samsung vs. Crucial RAM?

I purchased Samsung RAM for my Supermicro board. It has not given me any issues. I also own Crucial RAM too and it works perfectly as well, but that was in a completely different system. I like them both myself however when purchasing for a specific motherboard, you don't need to run with the rest of the pack, just ensure you run MemTest86 for about 1 week to ensure you don't have any issues no matter what RAM you purchase. If your RAM fails, try to drop the RAM speed a bit and test again. Don't tweak voltages unless you know what you are doing. .1 volt is a huge amount with respect to these small circuits these days.

 

[John3-6teen]

Though this board calls for 2133MHz RAM the Tested Memory List includes 2400 MHz RAM. I don't see any advantage to getting the faster RAM if the board doesn't use the extra speed. Am I missing something? Is there any reason to get the 2400MHz RAM? The pricing, at least from Crucial, is the same.

 

[joeschmuck]

You could purchase the 2400 MHz RAM if you like, especially if the price it the same. You can still run them at 2133 MHz if you like as well (something I'd do myself to reduce the heat & stress on the RAM). But in the FreeNAS environment, the speed difference will not truly get you more performance because you are limited to your connectivity (Ethernet in most cases) speed or in badly configured systems, even the hard drives themselves.

 

[Mirfster]

The pricing, at least from Crucial, is the same.

If the pricing is the same and it works, why not get the faster RAM. While you may not really see the speed increase, the only concerns would be if it used more electricity or caused more heat. If the answer is "No", then I would get the faster RAM just for "Future Proofing" and the semblance of getting more for my money. :)

/Edit: JoeSchmuck beat me to it... ;)

 

[John3-6teen]

From what I can tell the specs are the same on both 2133 and 2400 Crucial RAM except of course the speed and then the CL is 15 and 17 respectively. If I am doing the math correctly the effective speed of the RAM comes out nearly the same i.e. 15/2133 = .00703 vs 17/2400 = .00708. If I'm doing this wrong please correct me, I'm still in the learning curve on some of this. On the Crucial site they express the speed in MT/s vs MHz and I know there is a difference, but for purposes of calculating effectual speed does it really matter?

Does this mean that if I run the 2400 at 2133 it will actually be slower than the 2133 module? I'm assuming the CL does not change just because I run it slower so the speed would be 17/2133 = .00797? In the end it is such a tiny difference that I doubt it would ever be noticed, which makes it a mute point.

Given the 2133 RAM is out-of-stock in most places and the 2400 is the same price (and for the reasons given by JoeSchmuck and Mirfster) I believe I will opt for the later.

Thanks for the tips guys. If you would care to comment on my evaluation above please do so that I might know if I am doing this correctly or not. :)

 

[joeschmuck]

Yea, the speed differences with all the clock delays waiting for the data to be valid, it can drive a person crazy to know if you actually have "faster" RAM. But the value which you specified is just that, how many clock cycles must you wait before the data can be considered valid. There is another aspect you have missed, how fast that data can then be transferred across the data bus and into the CPU which would be 1200MHz (half the clock rate I believe of the 2400MHz RAM) vs. the 1066MHz for the 2133MHz RAM.

Regardless of the RAM speed calculations, I think you are doing the right thing by purchasing the faster qualified RAM for the same price.

 

[Wouter]

No benefit at all. FreeNAS only needs a very small piece of that huge capacity. The entire drive will be formatted but FreeNAS only uses a small section of it.

As for smaller capacity and cheaper drives, if you could buy a 16GB SSD for cheaper, that would be the way to go. I would not recommend smaller than 16GB just because, well that's my gut speaking, I have nothing to base my gut on.

Me, a newbee too, would like to ask: is FreeNAS able to use this SSD where one might install FreeNAS on, as the disk for L2ARC too? Or must the install disk be another disk than the disk used for L2ARC?

 

[Mirfster]

Me, a newbee too, would like to ask: is FreeNAS able to use this SSD where one might install FreeNAS on, as the disk for L2ARC too? Or must the install disk be another disk than the disk used for L2ARC?

You really don't want the disk to have multiple purposes. It will cause excessive wear and those roles should be separate. I would suggest just grabbing two older/used SSDs (like 32 GB) from eBay for the OS. Should run you ~$20.00 each. However, you may be able to get larger ones for the about same price (like this). I would opt for the 2nd choice personally.

For some, having two SSDs (mirrored) OS Drives is overkill, but I prefer it.

 

[Wouter]

You really don't want the disk to have multiple purposes. It will cause excessive wear and those roles should be separate. I would suggest just grabbing two older/used SSDs (like 32 GB) from eBay for the OS. Should run you ~$20.00 each. However, you may be able to get larger ones for the about same price (like this). I would opt for the 2nd choice personally.

Clear!

For some, having two SSDs (mirrored) OS Drives is overkill, but I prefer it.

And how to mirror: you mirror it with hardware RAID (yes, I know, not for NAS data, only for OS) or BIOS Software RAID or is FreeNAS able during install time to mirror two (small) SSD's and then install?

 

[jgreco]

And how to mirror: you mirror it with hardware RAID (yes, I know, not for NAS data, only for OS) or BIOS Software RAID or is FreeNAS able during install time to mirror two (small) SSD's and then install?

BIOS software RAID will not work.

You can mirror both boot devices during install and it will mostly be fine. There are some cases where it is possible for a boot device to fail and still be visible to the BIOS that might cause some issues with the BIOS bootstrapping the OS on the alternate drive. If you absolutely cannot tolerate such a situation, you can use a supported HBA in IR mode. That's what we used to do for FreeNAS 8 and it works fine.

 

[Mirfster]

FreeNAS able during install time to mirror two (small) SSD's and then install?

Yep, when installing FreeNAS just select the two drives and it will "automagically" create the Mirror for you. Otherwise you can just select one drive and add the 2nd later (but that is just making more work for yourself).

ou mirror it with hardware RAID (yes, I know, not for NAS data, only for OS) or BIOS Software RAID

Neither of these, FreeNAS is very capable of handling it. You really never want Hardware Raid or BIOS Software Raid anywhere in the mix. Just let FreeNAS have direct access to the drives.

*** It actually took me a while to get that through my thick skull at first as well, coming from a Hardware Raid background. :)