new build proposal for streaming/backup server

[mjt5282]

Hi. I am hoping to build my first freenas server and have read the docs and most of the hardware critiques. Here is mine. Feel free to make suggestions.

The Server will be used to stream flac audio to Sonos endpoints on my LAN using SMB, also used to stream HD MKV files to DUNE media players using SMB or NFS. It will also sure as a rsync server and possible Time Machine for my Mac clients (though this remains to be seen...)

I will probably install VMWare server virtualization (but not for freenas but I have some learnin' to do there). I might eventually be running plex so I guess I need some processing headroom there.

1) SUPERMICRO MBD-X10SL7-F-O - the Haswell SM board with the built-in LSI 2308
2) Intel Xeon E3-1230V3 Haswell 3.3GHz CPU
3) 16 Gb of ECC Samsung M391B1G73BH0-CK0 RAM , with 32 Gb later
4) 11 3Tb WD Red and Green drives (already purchased and configured to be NAS friendly)
5) 1 or 2 M1015 passthrough RAID card (already purchased, need to flash to IT firmware)
6) 4 Discrete to SFF-8087 (Reverse breakout) Norco cables (would handle 16 drives)
7) Supermicro SuperChassis CSE-836BA-R920B 3U with 16 hot swap bays or the SUPERMICRO SuperChassis CSE-846A-R1200B Black 4U with 24 hot swap bays. Both come with high quality PSU's and fans. The 3U seems more appropriate to a garage/basement environment (the 3U has the smaller more efficient PSU).

my intent is to create a 11 -wide ZFS raid3z (33 Tb raw, 21.8 usable approx) and a smaller mirrored dataset for Vmware Fusion (either a simple mirror or a smallish 5wide raid2z). I am currently running 8-wide raid2z using OpenZFS on Mac OS X (pool version 28). I would like to migrate to something with more storage headroom and better ZFS stability.

 

[mjt5282]

I've chosen to go with the 3U chassis and I noticed that I don't need reverse breakout cables but rather SFF-8087 to SFF-8087 cables (I've ordered 4). My memory has arrived today and the rest of the kit sometime next week.

 

[Ericloewe]

Do I understand correctly you already had the M1015s? If so, why bother with the X10SL7-F?

 

[mjt5282]

Do I understand correctly you already had the M1015s? If so, why bother with the X10SL7-F?

yes, probably overkill . I bought the motherboard with the integrated LSI card and already owned ebay purchased M1015's a while ago ...

 

[Ericloewe]

yes, probably overkill :) . I bought the motherboard with the integrated LSI card and already owned ebay purchased M1015's a while ago ...

Well, you won't be lacking SAS ports, at least.

 

[mjt5282]

ok, my server has been running successfully for a few days, I thought I would update the thread with my results. The newegg cables I bought didn't work, and they were not long enough anyway. I purchased SFF-8087 cables from another source that worked fine. I had to buy Y splitter power cables from newegg to control a few of the fans as I guess the motherboard is a little smaller than normal and the fan cables didn't reach for 2 out of 5.

I love the case - it is the cadillac of 3U cases though they could use some more cabling documentation. The Intel CPU is great and the motherboard, no problems. I have 16 more GB of samsung ram coming today. The biggest question I had about the case was: would it be too loud for use in my network/storage closet? The answer is , its OK but the fans really blow loud when it turns on initially or reboots. once it settles down it is acceptable, though I think I am planning on putting it into my garage in a couple of months.

OCD got the better of me and I felt I had to update the firmware of the M1015 to flash it to IT mode. I read many "how to's" on the net about how to flash the M1015 to LSI IT mode, of course the X10SL7-F-O motherboard required work not mentioned on most of the blogs I searched. The problem #1 was the UEFI (?) BIOS mode. LSI provides a utility to flash from UEFI boot shell. But mistakenly I was trying to flash the built-in LSI SAS2308, which didn't work. I figured out how to list all the LSI HBA's with the uefi utility and added the arguments to specify which card to flash. In the process I also flashed the 2308, though I am not using it right now.

I am populating my new ZFS pool using rsync and gig ethernet ... The HBA's are using V19 firmware but V16 drivers ... : mps0: Firmware: 19.00.00.00, Driver: 16.00.00.00-fbsd ... is this going to be a problem ?

 

[ser_rhaegar]

I love the case - it is the cadillac of 3U cases though they could use some more cabling documentation. The Intel CPU is great and the motherboard, no problems. I have 16 more GB of samsung ram coming today. The biggest question I had about the case was: would it be too loud for use in my network/storage closet? The answer is , its OK but the fans really blow loud when it turns on initially or reboots. once it settles down it is acceptable, though I think I am planning on putting it into my garage in a couple of months.

How much power does it pull when no drives are in the system? Just curious. I have a different SuperMicro 3U with an 800W PSU that I think pulls way to much power for just the board/proc/ram/fans. I'm considering upgrading to the 920 platinum PSU as the 800W has no rating and could have real poor efficiency.

I am populating my new ZFS pool using rsync and gig ethernet ... The HBA's are using V19 firmware but V16 drivers ... : mps0: Firmware: 19.00.00.00, Driver: 16.00.00.00-fbsd ... is this going to be a problem ?

Your safest bet is to match the firmware and driver... which means flashing v16 firmware to the card. It should work as you have it, but you could hit unforeseen problems and LSI recommends matching them. I highly recommend reflashing.

 

[cyberjock]

Your safest bet is to match the firmware and driver... which means flashing v16 firmware to the card. It will work as you have it, but you could hit unforeseen problems and LSI recommends matching them. I highly recommend reflashing.

I think it's better to say it "should" work as you have it, but you could hit unforeseen problems and LSI recommends matching them.

I will tell you that if you had a system with a problem and called LSI and your driver and firmware didn't match they'd tell you to fix the mismatch and call them back afterwards.

 

[ser_rhaegar]

I think it's better to say it "should" work as you have it, but you could hit unforeseen problems and LSI recommends matching them.

I will tell you that if you had a system with a problem and called LSI and your driver and firmware didn't match they'd tell you to fix the mismatch and call them back afterwards.

Yes, you're right. I corrected it.

 

[mjt5282]

I found the V16 firmware so perhaps when I add the other 16 Gb of RAM I will also flash the M1015's down to V16 to be consistent. I will say that with approximately 8Tb sync'd already there have been zero problems once the system stabilized (when I figured out that I was trying to flash the integrated HBA and some cards were loose). For anyone who's truly interested, the sas2flash.efi command includes the -listall argument, which then lists the various HBA's in your system, and you can add -c (0,1,2) to the argument to flash a specific HBA. Don't try to flash the integrated HBA with the M1015 firmware, it won't work, and it upsets the system