Design Review

[Mirfster]

Hello all,

Just seeing if my first proposed design meets your scrutiny?

Audience
Customer wants to:

1. Virtualize their existing instance of SME Server (CentOS Based)
2. Have the ability to create a very limited amount of VMs
3. Increase their SME Server Backup speeds to a NAS
   • Currently they have a Drobo connected over 1 GB
   • Backups are so big and slow that they are running into the next day and colliding with previous backups...
4. Move away from Drobo, per my suggestion and just too many reasons to list in this thread :)

FreeNas Server Design
Server - Dell PowerEdge FS12-TY C2100
CPU - Two (2) Intel Xeon E5506 SLBF8 2.13GHz/4MB/4.8GT/s Quad Core LGA1366 CPUs
Memory - 24 GB 1066 Mhz ECC
Controller # 1 - Dell H200 Mezzanine SAS Raid Controller (Flashed to LSI 92118 IT Mode; No Boot BIOS)
Controller # 2 - Dell Perc H700 512MB Cache 6GB Raid w/Battery

• This is going to change to a PERC 6/I since for some reason the H700 and H200 are not existing well together on the C2100 (NVRam Errors...)

OS - FreeNAS-9.3-STABLE-201509022158
Hard Drives (For OS) - Two (2) Samsung 850 EVO 250GB 2.5-Inch SATA III Internal SSD
Hard Drives (For Data Storage) - Twelve (12) Hitachi HUA723030ALA640 3TB 7200RPM 64MB SATAIII (6GB/S) 3.5" (Enterprise Class)
NIC # 1 - Intel 82576 GB Ethernet (On-Board)
NIC # 2 - Intel 82576 GB Ethernet (On-Board)
NIC # 3 - 100Mb Ethernet Dedicated Management Port (IPMI)
NIC # 4 - Chelsio 110-1088-30 10GB 2-Port PCI-E HBA Adapter Card

So may plan is

1. Setup the two (2) Samsung 850 EVO 250GB in a RAID 1 using the PERC 6/I as the FreeNas OS Drives
   
   • I know this may be a little over-kill; if there are any blatant reasons not to do this; I am all ears
   • Will be running FreeNAS-9.3-STABLE-201509022158
2. Connect the twelve (12) hard drives to the C2100 Backplane using the flashed H200
   1. This will be the Data Storage
3. Direct connect the Chelsio to another Server (PowerEdge T310) used only for backup purposes (fast connection)
   • PowerEdge T310 will be running ESXi 5.5 Update 3 with a VM running SME Server 9.x
   • ESXi will have an additional Virtual Switch (Call it "10 GB Switch") just configured for the QLogic QLE8152 10GB Dual Port PCI-E Fibre Channel Host Bus Adapter; only backups will be using this 10 GB connection
   • All other traffic will be ran on the default Virtual Switch
4. Use the two (2) 1 GB NICs for standard Network connections (to their Switch)
5. Leverage LDAP on the SME Server for Permissions
6. Two (s) Intel 82576 GB NIC on the C2100 will be used for all standard traffic

If there are any design flaws or suggestions; I would be happy to hear from you all. I have been perusing these forums for a bit and appreciate all the effort/knowledge that is put forth.

Thanks!

 

[pirateghost]

What is the reasoning behind 250gb ssd for OS? You realize that all the space of those will be wasted and Freenas was intended to be installed on USB?

 

[Mirfster]

Very good question.

I was actually mulling this over myself. On the ESXi Server, I currently have two (2) 32GB Samsung mSATA SSD Solid State Hard Drives in a StarTech Dual mSATA SSD to 2.5-Inch SATA RAID Adapter Converter (25SAT22MSAT). Which allows me to install ESXi and have a RAID 1 (for redundancy). Rest of the drives on the ESXi Server are in RAID 6 via a PERC H700.

However, maybe it would be best to use the StarTech and mSATA SSDs on the FreeNas Server and use the two (2) 250 GB drives (in RAID 1) on the ESXi Server? That way I would add usable space to the ESXi Server (to house ISOs or other items; just not really wanting to put VMs on it).

So, if I was just using the 32 GB mSATA SSD Drives (in RAID 1) for FreeNas I would be all set as far as FreeNas?

Thanks!

 

[pirateghost]

Well, I still don't understand why you would spend the money on a perc and ssds, when a pair of USB is all you need.

Also, I don't understand your network layout. You mention hooking up both gb NICs, but don't explain how. Lacp?

 

[Mirfster]

C2100 does not have internal USB ports and the customer wants redundancy even on the OS Drives but a "clean" appearance (external USB sticks can be bumped/knocked out). The mSATA SSDs, StarTech Dual mSATA SATA RAID Adapter Converter and PERCs were items "readily available" that they had.

As far as the the two (2) GB NICs, they will be attached to their existing switch so normal shares can be accessed on the network. Link Aggregation is undetermined at this time; may be FailOver or LACP.

The 10 GB NICs will be via direct connection (Servers will be within 1 Meter of each other) and have Static IP(s) assigned to them that are in a different range/subnet from the 1 GB NICs so to limit that connection only for backups coming from the SME Server. As well, Link Aggregation is undetermined at this time; may be FailOver or LACP.

Forgot to mention the existing Drobo will be re-provisioned to house DR data and FreeNas config file(s), so there will be an additional layer of disaster recovery. But, that is a little off topic and don't want to change focus...

So, out of curiosity, if the C2100 got an internal USB card and two USB sticks (say 32 GB or so); does FreeNas have the ability to RAID 1 (clone) to the 2nd USB without the need of anything else? Sorry if that is a dumb question, just not familiar with that ability. I suppose a USB Card with RAID ability could be purchased, if needed...

Thanks

 

[pirateghost]

Freenas can mirror the OS thumb drive. Also, the preferred thumbdrives barely stick out a couple of mm from the back of the case. Your cables are more noticeable than the USB...

 

[Mirfster]

Thanks for the info, I will look into the USB Thumb Drives and Mirroring. Back to the manual for me. :D

 

[Fuganater]

The recommended thumb drive of the forum is the SanDisk Fit.

Sent from my SM-T700 using Tapatalk

 

[joeschmuck]

The recommended thumb drive of the forum is the SanDisk Fit.

It is? Hum, I don't recall that anywhere but it does get mentioned a bit. Don't get me wrong, it's a good little drive but there are others out there as well that do just as good of a job. And the Fit is relatively new with respect to use in the FreeNAS environment so longevity has not been established. But I do like the small size of them, I have one myself that I use in my truck for MP3 music, nice low profile, and I would use it in my machine without hesitation.

I'd say the only non-recommended USB Flash drives are the really cheap ones, buy a quality name brand you may have seen on the forum here and it is highly preferable (not required) to have one with 8GB (4GB is the actual minimum for FreeNAS 9.3) of capacity or more to deal with holding excess old OS upgrades for people who do not clean up their systems regularly.

@pirateghost If a person is going to build the server listed above and then drop it off for a customer and leave it alone, wouldn't it be better to use a single SSD as the boot device vice dual USB flash drives? I know my answer but I value your opinion as you have all that networking backgound that I do not have.

 

[gork]

Dell have a mirrored dual SD option that provides the storage redundancy in hardware and presents as a single device for boot. We use these for our R710 vmware farm which are otherwise diskless.

I personally went with dual SATA disk-on-modules on my Supermicro build, but I suspect the dell would not have the sata ports to support those. The dual SD are your best bet, and IIRC were not that expensive an option.

 

[joeschmuck]

DOM's are a bit pricey and are fine for equipment which must be very small/compact, but SSD's have been making moving in the right direction with size and cost to make DOM's almost a thing of the past.

 

[gork]

It's not always about the cost. A couple of SSD for boot take up bays that might better be used for storage drives or l2arc/slog drives though. I think the DOM is still a great option especially on SuperMicro where the board supplies power on the same connector. On mine I used the rear 2x2.5" bays for SLOG/ZIL and put L2ARC is PCIe nvme. The box is a big honkin 4U with 36 bays; it's far from compact and I am very happy with the DOMs. Ideally they wouldn't be much more expensive than flash drives if they were more popular. I think I paid about $50 each which I felt was worth it compared to both 2.5" SSD and USB

I tried USB first though. A lot can go wrong with USB boot; you have to rely on the BIOS or EFI and the OS USB driver to do lots of things right. I could not make it work reliably on the supermicro board despite the settings -- including forcing the USB3 controller to 2.0, completely disabling the USB3 controller, adjusting the handoff and emulation settings, etc. Sometimes it just would not do the handoff properly after the kernel had loaded and the system would hang. You could boot the system fine, then issue a reboot and it would freeze. On top of that you could not set the boot order of multiple USB devices -- It just wasn't something I felt comfortable relying on even in a non-critical system.

 

[Mirfster]

Just an update, as far as the two OS Hard Drives go I decided not the use the 250 GB SSDs. Will use smaller drives and connect then directly to the SATA Ports. During the installation, I will select both drives so that FreeNas will mirror the Boot Device. Also, I will ensure that in the BIOS Boot Settings I have it configured for Disk 0 first then Disk 1 second.

Since I am new to FreeNas and ZFS I was thinking that the OS Drives has to be mirrored via hardware instead of software.

 

[pirateghost]

Just an update, as far as the two OS Hard Drives go I decided not the use the 250 GB SSDs. Will use smaller drives and connect then directly to the SATA Ports. During the installation, I will select both drives so that FreeNas will mirror the Boot Device. Also, I will ensure that in the BIOS Boot Settings I have it configured for Disk 0 first then Disk 1 second.

Since I am new to FreeNas and ZFS I was thinking that the OS Drives has to be mirrored via hardware instead of software.

In freenas, don't use hardware raid at all

 

[Fuganater]

When you install FreeNAS you can choose 2x drives to install the OS on. That creates your mirror.

 

[Mirfster]

Yeah, stating to get that point through my thick skull. ;)

Thanks for the input.