Building my first FreeNAS system, build advice

[Kiskaa]

Hi there,

I’ve got quite a complex home network system at home, and am looking to replace my rather old and slow NAS server.

I work from home, so this would be a file server, ftp server, potentially also running some VMs for testing program builds etc.

I’d also be using it very heavily as a Plex media server, transcoding up to 4x 1080p streams at a time so I need a cpu score of at least 10-12k.

I’ve been looking around on eBay and come across what looks to be a very promising system with pretty damn good specs.

My question is, would the following spec work with FreeNas and would all 4 nodes as as one huge server?

These are the specs and components.

Intel H2312JFKR 2U Rack Mount 4 node Server Chassis comprising
12 x 3.5" Hot Plus HDD caddies
2 x Hot plug 1600W Platinum Efficiency Power Supplies for redundancy
Rack MountRails

4 x Intel HNS2600JF Server Nodes each comprising
2 x Intel XEON Six-Core E5-2630 2.2GHz Processors and heatsinks
32GB Ram
Dual Port Intel I350 Gigabit Controller
1 x PCIe expansion slots
1 x IO module or dedicated management port expansion slot
System fans
VGA Port
2 x USB Ports


Full Specs

Chassis Form Factor 2U Rack

Chassis Dimensions 3.5'' x 17.2'' x 30.5''

Board Form Factor Custom 6.42'' x 17.7''

Rack Rails Included Yes

Compatible Product Series Intel® Xeon® Processor E5-2600 v1 / v2 Product Family

Socket Socket R

Heat Sink 8

Heat Sink Included Yes

System Board Intel® Server Board S2600JF

Board Chipset Intel® C602 Chipset

Target Market High Performance Computing

Rack-Friendly Board Yes

Power Supply 1600 W 120-127V / 200-240V Auto Switching for use Worldwide

Power Supply Type AC

Thanks in advance for any assistance/advice.

Max

 

[Kiskaa]

The other system I was looking at which albeit is quite a bit less power, without having nodes to deal with could potentially be a lot easier to configure and maybe more compatible If the above system isn’t possible.

Dell PowerEdge R710

Xeon CPU

2 x Intel® Xeon® Processor X5650 ( 12M Cache, 2.66 GHz, 6.40 GT/s Intel® QPI ) - 12 Cores & 24 Threads

Drive Capacity

No hard drives & NO caddies ( takes 6 x Hot Swap SAS/SATA 3.5" hard drives)

Memory Installed

32GB DDR3 RAM

Raid Controller

Perc 6i 256mb cache with Battery - RAID 0, RAID 1, RAID 5, RAID 10, RAID 50

Power Supply

2 x Power Supply Installed , 4 PCIe G2 slots + 1 storage slot : Two x8 slots, Two x4 slots, One x4 Storage slot

Network NIC

Embedded ( 4 ) Quad-port Broadcom® NetXtreme® II 5709C

Remote NIC

Dell iDRAC6 Enterprise Baseboard Management Controller

Would this be a suitable alternative if the first one isn’t compatible ?

Again, thanks in advance
Max

 

[Inxsible]

The 4 nodes, I believe are pretty independent of each other. It will serve well for you to have a separate node for VMs and a separate node for file server etc. I don't see why it wouldn't work with FreeNAS either.

You haven't mentioned the price or a budget, but here are a couple of SuperMicro chassis that you can build into as well : The first one is just the chassis and the second one has a X9 board, but you'd have to add the dual CPUs and RAM

1. https://www.ebay.co.uk/itm/Supermic...165732?hash=item213e8a3524:g:zEUAAOSw441a36fO
2. https://www.ebay.co.uk/itm/Supermic...574716?hash=item239fb945bc:g:eyIAAOSwc2FaBe7M

 

[Kiskaa]

You haven't mentioned the price or a budget

Thanks for that, I`ll take a look.
I don't specifically have a budget as such, but I`d like to be able to do it for between £2000 - £2500. (excluding HDD`s)
The 4Node NAS was £1200 and I don't recall the price of the other one off the top of my head, it was around £400ish.

 

[Kiskaa]

This one looks as though it`ll do the job nicely too. (based on your links above, same chassis with components installed.)
https://www.ebay.co.uk/itm/Supermic...202236239291?_trksid=p2349526.m4383.l4275.c10
With the upgraded raid card to run all 12 drives instead of it being split.

This has a CPU pass mark of 11000 for the E5-2660 and it has 2 of them, so well within the capabilities of my transcoding needs also, and plenty powerful enough for the other things I want to run on there.

Will this be suitable for the FreeNas system do you know?
Thanks in advance.

 

[Inxsible]

This one looks as though it`ll do the job nicely too. (based on your links above, same chassis with components installed.)
https://www.ebay.co.uk/itm/Supermic...202236239291?_trksid=p2349526.m4383.l4275.c10
With the upgraded raid card to run all 12 drives instead of it being split.

This has a CPU pass mark of 11000 for the E5-2660 and it has 2 of them, so well within the capabilities of my transcoding needs also, and plenty powerful enough for the other things I want to run on there.

Will this be suitable for the FreeNas system do you know?
Thanks in advance.

That's a great chassis. I have one too. 12 Drives are plenty for me. The board X9DRi-LNF4 is also a great board. Many users use it on the forums. 96GB of RAM seems plenty too. The included PSUs are Platinum rated which is great.

With the upgraded raid card to run all 12 drives instead of it being split.

You might want to stick with the 2 LSI cards because FreeNAS doesn't do well with hardware RAID and if I am not mistaken that Adaptec ASR 71605 card does not support JBOD. You'd be wasting that 250 Pounds (us keyboard no pound sign :( ) upcharge.

Instead, if you have a 4-post rack enclosure/open frame rack then pay up the 66 pounds for the rails.

 

[Kiskaa]

You might want to stick with the 2 LSI cards because FreeNAS doesn't do well with hardware RAID and if I am not mistaken that Adaptec ASR 71605 card does not support JBOD. You'd be wasting that 250 Pounds (us keyboard no pound sign :( ) upcharge.

Instead, if you have a 4-post rack enclosure/open frame rack then pay up the 66 pounds for the rails.

Thanks for the reply, I`m leaning towards this system, the other one seems massively overkill for my needs, although, the 4 nodes are tempting but would be a waste of extra money to be honest, and the sheer extra heat in my little server room would be quite noticeable.

I understand what you are saying about the Raid Card being a waste due to compatibility issues, however, how would that affect my raid array? I`m assuming instead of 1 large volume, i`d have 2 raid volumes, 1, based on 8 drives, the other on 4?

Or is there a way around this?

I`ll be needing rails, as it`ll be going into my rack enclosure.

Thanks in advance.

 

[Inxsible]

I understand what you are saying about the Raid Card being a waste due to compatibility issues, however, how would that affect my raid array? I`m assuming instead of 1 large volume, i`d have 2 raid volumes, 1, based on 8 drives, the other on 4?

Or is there a way around this?

A lot depends on your pool configuration. Since you want to run VMs and run a storage server, I'd advise you go with 2 separate pools. Use RaidZ2 on each and so you'd have 2 drive redundancy on each pool. Depending on the amount of space you need for each application (storage and VMs) you can select the appropriate size of the HDDs.

So if you are going to have, say 3 VMs, then 4 drives for the VMs should be plenty as long as you buy 2TB+ drives. That's obviously your call in how much space you need for each application. But that's just an example.

Let 1 card control your storage drives and the other card control your VM OS/storage drives

 

[Kiskaa]

A lot depends on your pool configuration. Since you want to run VMs and run a storage server, I'd advise you go with 2 separate pools. Use RaidZ2 on each and so you'd have 2 drive redundancy on each pool. Depending on the amount of space you need for each application (storage and VMs) you can select the appropriate size of the HDDs.

So if you are going to have, say 3 VMs, then 4 drives for the VMs should be plenty as long as you buy 2TB+ drives. That's obviously your call in how much space you need for each application. But that's just an example.

Let 1 card control your storage drives and the other card control your VM OS/storage drives

I currently have 4x 2tb drives in my current Nas Running Raid5, which im actually going to change.
I could put these as you suggested in the 4bay section, use as OS/VMs which is a great idea, and start a new storage pool with the 8 remaining. Since I dont need massive amounts of space for the VMs, I can happily use my current 2TB drives on that card and start using 4tb drives on my new array.
Thanks for the input/suggestions.

I`ll go ahead and order that system.

 

[Inxsible]

I can happily use my current 2TB drives on that card

That would work great. Although, since there are only 4 drives for the VMs and if the data is not very important -- then you might also choose RAIDZ1 to get a bit more space. Mirrors is also an option when you only have 4 drives to work with.

For the 8-drive storage pool, I would go with RAIDZ2 and 1 vdev of all 8 drives.

 

[Kiskaa]

That would work great. Although, since there are only 4 drives for the VMs and if the data is not very important -- then you might also choose RAIDZ1 to get a bit more space. Mirrors is also an option when you only have 4 drives to work with.

For the 8-drive storage pool, I would go with RAIDZ2 and 1 vdev of all 8 drives.

Thats pretty much when I intend on doing.

[edit] I thought I submitted this reply yesterday, turns out I hadn't clicked "Post Reply"

 

[Kiskaa]

That would work great. Although, since there are only 4 drives for the VMs and if the data is not very important -- then you might also choose RAIDZ1 to get a bit more space. Mirrors is also an option when you only have 4 drives to work with.

For the 8-drive storage pool, I would go with RAIDZ2 and 1 vdev of all 8 drives.

I wasn't sure how to tag you into a reply, so just quoted your previous reply, hope that's ok.

Just a quick question, that server arrived today, 2 things I have an issue with.
1) Its as loud as a jet taking off, are there any quieter fan systems that you are aware of that I can install, so its not going to deafen me. It`s going to be in a server room, so doesnt need to be whisper quiet, but my god, at the moment, its monumentally loud.
2) There are only 2 USB ports on it at the rear, which poses a little bit of an issue setting it up.
There is no DVD drive, so I cant put the image onto a DVD. I need one of the USB ports for the install USB disk, and the other for the keyboard during setup. So im not quite sure how I can get FreeNas installed. Any suggestions? As far as I`m aware, I'm not able to use a USB hub to install media from, and eventually, i`ll need to use the second usb port for the second USB install disk (I believe I needed 2 USB drives, so one is mirrored as a failsafe or something.

Thanks in advance
Max

 

[Chris Moore]

You don't use hardware RAID with FreeNAS. ZFS provides the RAID at the filesystem level.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Chris Moore]

There are only 2 USB ports on it at the rear, which poses a little bit of an issue setting it up.

There is a USB port on the system board (inside the case) where you can plug a drive internal in the chassis and there are headders where you can add additional ports if you need to.

So im not quite sure how I can get FreeNas installed. Any suggestions?

It is a Supermicro server board with IPMI, so you can remote into it and do the Keyboard, Mouse and Monitor over Ethernet as well as remotely mount the ISO file. There is no need to connect any of that locally at the server.

 

[Chris Moore]

I believe I needed 2 USB drives, so one is mirrored as a failsafe or something.

You really need to look at some of the resources on this before you go any further. Check the links under my "Useful Links" button in my signature.

 

[Inxsible]

2) There are only 2 USB ports on it at the rear, which poses a little bit of an issue setting it up.
There is no DVD drive, so I can't put the image onto a DVD. I need one of the USB ports for the install USB disk, and the other for the keyboard during setup. So im not quite sure how I can get FreeNas installed. Any suggestions? As far as I`m aware, I'm not able to use a USB hub to install media from, and eventually, i`ll need to use the second usb port for the second USB install disk (I believe I needed 2 USB drives, so one is mirrored as a failsafe or something.

It is a Supermicro server board with IPMI, so you can remote into it and do the Keyboard, Mouse and Monitor over Ethernet as well as remotely mount the ISO file. There is no need to connect any of that locally at the server.

Yup. All you need is a ethernet cable connected to the IPMI LAN port. Make sure your router assigns an IP to that MAC and you should be golden. IPMI is very useful. I don't buy server hardware without IPMI on it.

 

[Chris Moore]

Its as loud as a jet taking off

This is why I never suggest 2U rack units for home use. Sorry. You can replace all the fans with Noctua fans to make the system a bit quieter, but you will need to keep the speed relatively high to get enough airflow to cool the drives and processors.

 

[Kiskaa]

You really need to look at some of the resources on this before you go any further. Check the links under my "Useful Links" button in my signature.

I have, regarding the actual FreeNAS setup, I'm pretty well up on what I need to do, hence why I know I need the 2usb ports (im doing a mirrored install)

Regarding the Noise, I'm fine with the 2u rack unit, I was just taken back a bit by the sheer noise, I knew it would be loud, as all servers are, I`ll just replace them with whisper quiet fans which should help.
For all its at "home" I have a server room built, and it`ll be going in there, once in there, with some slightly quieter fans, all will be good, with the door closed to the room, and using my other rooms, its only just audible as it is, I just want that extra loud part taking away so its not noticeable from the outside at all.

I`ll take a look at those fans you suggested, thanks.

 

[Kiskaa]

Yup. All you need is a ethernet cable connected to the IPMI LAN port. Make sure your router assigns an IP to that MAC and you should be golden. IPMI is very useful. I don't buy server hardware without IPMI on it.

Excellent, I`ll look into that, would make things much easier. Thankyou.

 

[Chris Moore]

Regarding the Noise, I'm fine with the 2u rack unit, I was just taken back a bit by the sheer noise, I knew it would be loud, as all servers are, I`ll just replace them with whisper quiet fans which should help.

Normally, you don't get the airflow required with the quiet fans. I have seen many systems where the result of replacing the fans was overheating of some component or other. The 2U and 1U systems were designed to be cooled by high volume (fast) airflow. Without that, something will probably break / overheat, so keep close attention to that.