Looking for feedback on new build

[walvord]

Hi,

I'm hoping to get some feedback (positive or negative) on this new build.

Motherboard: SUPERMICRO X9DRL-3F-O
CPU: pair of Intel Xeon E5-2609 v2 Quad-Core 2.5GHz 80W
RAM: Kingston 64GB (4 x 16GB) 240-Pin DDR3 SDRAM ECC Registered DDR3 1600
Case: Rosewill RSV-L4500
HDD: starting with 7 x WD Red Pro 4TB in raidz3
PSU: I haven't spec'd out the power supply yet.

This will be for backups, media storage and streaming at home, and some occasional use for lab esxi vm storage.

Some open questions though (pretty sure I've read/searched enough but I've got thick skin so if I ask dumb questions, let me have it):
1) This blog which seems legit states that raidz3 is significantly better at 5, 7, or 11 drives per vdev. Is that crucial or is that more for "optimal" performance meant more for business use vs home use?
2) Will there be any significant gains or pluses for using the onboard SAS ports vs M1015's or can I think of them as equivalent?
3) Is there a 'better' (knowing that's a really open ended question) drive choice then the wd reds? Maybe something with a higher density?
4) Will this give me as much room to grow as I think it will?
5) What do people typically backup home freenas servers to?


thanks in advance,
will

 

[Bidule0hm]

1) This is true only if you disable the compression as it's enabled by default so don't worry too much about the number of drives per vdev ;)

2) No, they are equivalent AFAIK.

3) Better on which specs? "higher density" like more TB per drive?

4) What room do you need?

5) Personally I currently backup to offline drives but I'm searching some used LTO drive for a decent price. From what I've seen the most used method is to backup to another server using ZFS replication.

NB: RAID-Z3 is very good for reliability (and paranoids like me :P) but not for VMs (usually we use striped mirrors for that usage).

 

[walvord]

Thanks for the great information!

3) Not just higher density but all factors. For home usage, is there a 'better' or more appropriate alternative to the wd red/pro series?

4) By room to grow, I mean all of the unforeseen changes between what you know and expect now vs the future. I'd like to future proof this as much as I can. I've been lurking on and off for awhile now reading and trying to absorb all of this information. What I don't want to do is decide that I want to change a requirement in a few months once I have more experience and find that if only I had chosen a different motherboard or something like that... It looks like I can expand the ram via either replacing the 16GB sticks with 32 and/or adding additional sticks. It also looks like there's plenty of room for adding sas expanders and a bit more room in the case before I have to think about either a new case or an external option. I just don't want to plan for the bare minimum of what I think that I need/want at this time.

thanks,
will

 

[Bidule0hm]

3) For home? no. I'll personally choose the Reds over anything else currently ;)

4) Ah ok. Yes, no problem with that ;)

FYI: for the PSU I recommend SeaSonic, very good value for money.

 

[danb35]

Skip one of the CPUs and replace the chassis with this SuperMicro from eBay when it becomes available again. It gives 9 more drive bays, better construction, and a SAS expander backplane so you won't need to buy an HBA to use additional disks in the future. It also includes a quality power supply (two of them, in fact). I'd probably even drop the RAM to 32 GB--more RAM never hurts, but nothing in your use case says you need even the 32 GB now.

 

[walvord]

Thanks guys!

danb - So by skip 1 cpu as in just get 1 now, leaving myself open to the possibility of adding the 2nd later if needed? ...or are you suggesting a different motherboard?

 

[danb35]

I doubt you'll ever need that second CPU, so you could get a single-chip motherboard and maybe a E5-1650 instead (I recall reports here that it's a very good choice for price/performance, but I don't have any personal experience with it), but you could also stick with your current choice for motherboard and give yourself that room for expansion of desired.

 

[walvord]

The only CPU caveat I remember reading about FreeNAS is with CIFS shares, but again...for home use a single reasonably modern cpu will be fine right? If there's literally no good reason to have a multi-processor motherboard then I won't even bother.

 

[Bidule0hm]

Yes, for example Samba is single threaded, so it's better to have a 4 cores at 3 GHz than 8 cores at 2 GHz for example ;)

 

[danb35]

The most common CPU-hungry task for home users is transcoding media using Plex. If you were going to do that, and serve up several HD streams at a time, you might reach a point where you needed two CPUs.

 

[Ericloewe]

Yes, for example Samba is single threaded, so it's better to have a 4 cores at 3 GHz than 8 cores at 2 GHz for example ;)

Multiple clients will spawn multiple threads. It's not like Samba is running everything in a single thread.

 

[walvord]

Think I'd be good with samba but I was thinking of testing moving plex to the FreeNAS server once it's up. This is what I meant by future proofing. :)

Good catch! So in terms cpu utilization for that under FreeNAS... Has anyone seen a writeup running through that? Wondering if it can be calculated some way or if it's just something that I'll have to do some trial and error on. Was looking at the e5-2609 v2 quad core 2.5ghz cpu's. Seems like it'd be beefy enough but there are faster cpu's out there...

 

[walvord]

@Bidule0hm
I've spent days googling and searching this forum, but possibly don't know the right terminology to search for... I've found a ton of writeups on configuring the iSCSI shares, but for the actual storage configuration piece I can't find anything basic enough for me to understand. I feel like a dumbass. All of the other posts I've found related to this, everyone just glosses over this, so it must be really basic for everyone else.

Can you please explain to me (or point me in the right direction) how to setup what's needed under the storage section of freenas 9.3 prior to configuring the iSCSI shares? That piece I think I have a handle on. I've got 6 x 2TB drives.

 

[Bidule0hm]

I'm not competent on the iSCSI shares but before configuring the shares you need to create a volume anyway. So I'd recommend to read the manual, section 8.1 more exactly.

You first need to create a pool (also known as volume in the manual and GUI), I recommend a RAID-Z2 if you don't need a lot of IOPS, or striped mirrors otherwise (VMs for example).

Then you need to create a dataset or a zvol. IIRC you need a zvol if you want to use iSCSI, just wait for an answer from a more knowledgeable member and/or read the manual section about datasets, zvol and iSCSI.

After that you can create your iSCSI share.

Remember that with iSCSI you need to keep your pool usage below 50% if you don't want to kill the performances.

 

[walvord]

Thanks! It's the striped mirror creation part that I don't understand. I can pick either under the raid setting but not sure how to do both.

 

[Bidule0hm]

You should choose mirror, stripe is not what you want. Then you can drag and drop the round button to select the 6 drives (2 lines of 3 columns IIRC), see the screenshots in the manual for more details ;)

 

[Spearfoot]

Can you please explain to me (or point me in the right direction) how to setup what's needed under the storage section of freenas 9.3 prior to configuring the iSCSI shares? That piece I think I have a handle on. I've got 6 x 2TB drives

With 6 drives I suggest you use either a RAIDZ2 pool (for more storage) or 3 striped mirrors (for more IOPS). In either case you will want to create a ZVOL dataset for the iSCSI share (see screenshot):

If you're uncertain how to create the storage pool, don't be afraid to experiment - 'cause you can always tear it down and start over if you goof something up. And if you search the forum you should be able to find plenty of helpful instructions. But the GUI really is easy to use, once you get the hang of it. And you can always use Oracle's free VirtualBox hypervisor to experiment with FreeNAS in a virtual environment.

To create a RAIDZ2 pool, use Storage -> Volume Manager, name the pool, select all six of your disks, then choose RaidZ2 from the Volume Layout dropdown.

It's a little trickier to create a pool of striped mirrors. Using Volume Manager, specify the pool name, choose two disks by dragging the Volume layout gizmo, and make sure Mirror is selected as the type. Then run the Volume Manager again, but specify the pool you just created in the 'Volume to extend' field. Select the next two disks and then extend the volume. Repeat for the last of two disks and you're finished. This is all harder to explain than it is to do, hopefully these screenshots will help.

Creating the pool:

Extending the pool:

As for iSCSI: there's a good blog write-up about running FreeNAS on VMware by a fellow named Benjamin Bryan at the link below. Section 18 -- "Create an ISCSI share for VMware" -- is easy-to-understand and pretty explicit. It may be helpful to you:

https://b3n.org/freenas-9-3-on-vmware-esxi-6-0-guide/

Be sure to read his caveat about iSCSI vs. NFS shares. He and some of the commenters on that thread (including me - I'm 'Keith' there) have found problems with iSCSI and use NFS instead. YMMV.

 

[walvord]

Thanks guys! Thanks for the screenshots and explanation Keith. I'm reading that article now.

thanks again,
will

 

[Bidule0hm]

You're welcome ;)

 

[walvord]

Rather than link to my original post, I thought I'd extend it.

As before, the (6) 2TB disc striped mirror is still configured. Thanks again Keith. I'm wondering if I'll see any significant performance gains if I switch from 7200 rpm sata drives to ssd's. I noticed woot has a good deal on 480gb "enterprise" ssd's.

I have a fairly beefy esxi host (48 cpu, 192gb ram) in my lab along with a fairly beefy freenas server (24 cpu, 128gb ram) with 2 x 10gb direct connect nics for the iscsi links. I only mention the specs because I'm not sure where my bottleneck is. I haven't noticed any performance issues, but have been thinking about upgrading the iscsi striped mirror discs to ssd. Just not sure I'd gain much. Thoughts??