BUILD 24U FreeNAS Build - Your feedback is welcome!

[fn369]

FreeNAS has supported mirrored boot devices since the 9.3 release. If you're using USB sticks as boot devices, mirroring them is highly recommended, as they're notoriously unreliable. SSDs and DOMs are much more reliable, so there's less incentive to mirror them. Also, reinstallation is simply a matter of doing a clean install and uploading a saved copy of your config file, so the consequences of a failed boot device are minimal. Mirrored DOMs are certainly fine, but probably not necessary.

Duly noted, thank you.

I'll see, though I'm not looking as much now that I've found one (though I did get an email a couple of weeks ago from the folks I bought mine from--if I get another I'll let you know). I think my ebay search when I found mine was simply "supermicro 4u". My server came with E5-2660s, which I upgraded to E5-2670s for no other reason than that the 2670s were too cheap to pass up, and a RAID card that I sold to recoup some of my cost. I'll warn you that energy-efficiency isn't its strong suit--it idles at over 300 watts, and burns 400-450 most of the time.

Don't worry. I got the impression you were still active in eBay, and I'm happy to do some searching. Certainly the power consumption figures are much higher than for my proposed E5-1650, but if I get an incredible deal perhaps it's worth it!

 

[philhu]

Thanks for your suggestions. I only need triple backup for photos and some work files, so external HDs and encrypted cloud (SpiderOak) should be ok I hope.

I did wonder about a 2U server + expansion chassis, but it seemed that all in one solutions were favoured here?

I did have this for a while. A 2u dell 2950 with 2 MD1000 addon boxes connected by fiber.
It worked well, and you can get a jbod box using an HBA connector to a pci or pci/e card. Just make sure the jbod box fires up and disk spun up before the
1/2U tries to find them. If it doesn't, all hell can break loose with the raid trying to mount

 

[depasseg]

I did wonder about a 2U server + expansion chassis, but it seemed that all in one solutions were favoured here?

Only because most people don't need that much storage. Besides, how else are you going to get up to 24U (sorry, could resist).

 

[fn369]

I did have this for a while. A 2u dell 2950 with 2 MD1000 addon boxes connected by fiber.
It worked well, and you can get a jbod box using an HBA connector to a pci or pci/e card. Just make sure the jbod box fires up and disk spun up before the
1/2U tries to find them. If it doesn't, all hell can break loose with the raid trying to mount

Interesting, thank you! And you were using this with FreeNAS?

 

[fn369]

Only because most people don't need that much storage. Besides, how else are you going to get up to 24U (sorry, could resist). :)

Understood, and fair comment! I left myself wide open to that one! Fire away!!!

 

[danb35]

Well, when you can get Sandisk 140G SSD brandnew for $59 for *2*, why wouldn't you mirror them?

• $30 is still $30. Compared to the rest of the cost of the system, of course, it's a drop in the bucket, but why spend money you don't have to?
• The SSD takes some space inside the server, and there may not be an "official" space to mount a second one. Granted, it doesn't take much space, and it can be mounted with sticky-back Velcro.
• The second SSD takes a SATA port that may not be available.

But I want to be clear that I'm not opposed to mirroring SSDs/DOMs, just that I don't think it's generally necessary. I had mirrored DOMs in my old server, but just a single SSD in my current one.

 

[SweetAndLow]

This is very similar to my build

*Supermicro 846e16 dual 1200watt
*X10srl
*E5-1620v3
*4xSamsung ddr4-2133 16GB
*supermicro snk-p0048ap4 cooler
*Lsi 9211-8i
*8x4tb z2
*8x3tb z2
*Cyberpower 1000va ups

My system reads and writes around 900MB/s streaming. The CPU hardly has to work even if my jails are busy. I run transmission, sonarr and plex. I also run crashplan for backup of my freenas on a Linux vm that is hosted on a different box.

I choose the xeon 1620 because it was cheaper and my usage isn't very CPU intensive. Also the idle watt usage is around 200watts, I should double check though.

Do you know how loud that case is? Mine is in the crawl space under house and you can still hear it with the fans on the lowest setting.

Good luck with your build, you will want 10gig after you start playing with it and seeing the performance

 

[fn369]

This is very similar to my build

*Supermicro 846e16 dual 1200watt
*X10srl
*E5-1620v3
*4xSamsung ddr4-2133 16GB
*supermicro snk-p0048ap4 cooler
*Lsi 9211-8i
*8x4tb z2
*8x3tb z2
*Cyberpower 1000va ups

My system reads and writes around 900MB/s streaming. The CPU hardly has to work even if NY jails are busy. I run transmission, sonarr and plex. I also run crashplan for backup of my freenas on a Linux vm that is hosted on a different box.

I choose the xeon 1620 because it was cheaper and my usage isn't very CPU intensive. Also the idle watt usage is around 200watts, I should double check though.

Do you know how loud that case is? Mine is in the crawl space under house and you can still hear it with the fans on the lowest setting.

Good luck with your build, you will want 10gig after you start playing with it and seeing the performance.

Very similar build! Thank you for your real-world feedback. As a matter of interest, have you ever tested to see how much simultaneous Plex transcoding it can manage?

No, I haven't a clue how loud the case is, other than @DataKeeper sayimg he could speak on the phone when standing by it. Is that not your experience? I have ordered a UCoustic Rack Enclosure just in case!

I take your point about wanting 10GbE, thank you, and will factor that in. Thanks for the good luck wishes!

 

[SweetAndLow]

Very similar build! Thank you for your real-world feedback. As a matter of interest, have you ever tested to see how much simultaneous Plex transcoding it can manage?

No, I haven't a clue how loud the case is, other than @DataKeeper sayimg he could speak on the phone when standing by it. Is that not your experience? I have ordered a UCoustic Rack Enclosure just in case!

I take your point about wanting 10GbE, thank you, and will factor that in. Thanks for the good luck wishes! :)

I'll test out the plex transcodes tonight. When I had a e3-1230v3 I could run 7 1080p transcodes so I would assume that the e5 system could run at least that. The sound thing is personal preference some people think it's ok and others will have problems, that is why I try to explain that mine is under my house and can still be heard at night when the house is quite. It sounds like a faint high pitched buzzing sound.

I don't have 10gig but would like to start looking into it just from a fun perspective. It isn't really required but I'm also not getting the full performance of my pool.

 

[philhu]

for 10g network, remember, you need something to plug the 10g nic into.

I bought a board on ebay, about $150, with 2 ports for the NAS, came right up, 2 ethernet ix0/ix1. Then I bought a HP Procurve 10gb module and a cat7 wire

 

[fn369]

I'll test out the plex transcodes tonight. When I had a e3-1230v3 I could run 7 1080p transcodes so I would assume that the e5 system could run at least that.

That seems like a very reasonable assumption!

The sound thing is personal preference some people think it's ok and others will have problems, that is why I try to explain that mine is under my house and can still be heard at night when the house is quite. It sounds like a faint high pitched buzzing sound.

Thank you. I really appreciate getting the benefit of your real-life experience.

I don't have 10gig but would like to start looking into it just from a fun perspective. It isn't really required but I'm also not getting the full performance of my pool.

Bigger. Better. Faster. More. I hate thinking of your pool's performance going to waste, and I will definitely be looking into getting it for mine. Thanks again!

 

[fn369]

for 10g network, remember, you need something to plug the 10g nic into.

I bought a board on ebay, about $150, with 2 ports for the NAS, came right up, 2 ethernet ix0/ix1. Then I bought a HP Procurve 10gb module and a cat7 wire

The motherboard needs to expressly support 10GbE, right? (Cue eye-rolling!) I will certainly put HP Procurve on my shortlist, thanks!

 

[philhu]

No, it just needs a pci slot or pci/e x8 or better slot

Here is the one I bought:
http://www.ebay.com/itm/401119530574?_trksid=p2057872.m2749.l2649&ssPageName=STRK:MEBIDX:IT
http://www.ebay.com/itm/401119530574?_trksid=p2057872.m2749.l2649&ssPageName=STRK:MEBIDX:IT
Put in machine, plugged in , works fine

 

[danb35]

The motherboard needs to expressly support 10GbE, right?

You can always buy a 10G NIC; it doesn't have to be built in to the motherboard. My Chelsio T420 is working well.

Edit: See https://forums.freenas.org/index.php?threads/10-gig-networking-primer.25749/ for a good into to 10G networking. I have two T420s, one in my FreeNAS box and one in my Proxmox host, and a Dell PowerConnect 5524 switch.

 

[fn369]

No, it just needs a pci slot or pci/e x8 or better slot

Thank you for confirming. I didn't think it did, but started to second guess myself! I've spent the past 3 hours in eBay, and my head is spinning!

 

[fn369]

You can always buy a 10G NIC; it doesn't have to be built in to the motherboard. My Chelsio T420 is working well.

Edit: See https://forums.freenas.org/index.php?threads/10-gig-networking-primer.25749/ for a good into to 10G networking. I have two T420s, one in my FreeNAS box and one in my Proxmox host, and a Dell PowerConnect 5524 switch.

Thanks for the reminder and recommendations, @danb35 - clearly I need to keep re-reading that thread, because it's not sinking in! :-(

 

[Mirfster]

I've spent the past 3 hours in eBay, and my head is spinning!

Funny... I always end up with "visions of sugar plums dancing in my head"... :)

 

[fn369]

Funny... I always end up with "visions of sugar plums dancing in my head"... :)

Nope, I'm definitely more than :) !

 

[SweetAndLow]

Very similar build! Thank you for your real-world feedback. As a matter of interest, have you ever tested to see how much simultaneous Plex transcoding it can manage?

No, I haven't a clue how loud the case is, other than @DataKeeper sayimg he could speak on the phone when standing by it. Is that not your experience? I have ordered a UCoustic Rack Enclosure just in case!

I take your point about wanting 10GbE, thank you, and will factor that in. Thanks for the good luck wishes! :)

The most streams I can play at 1080p is limited by the number of clients that I have. So I'm only able to get 5 streams to play on original quality which is 1080p with a bit rate of 27512kbps. The cpu is currently at 40% and network usage is around 80mb/s. Plex/Freenas server seems to be doing very well and could probably play twice this much.

 

[fn369]

The most streams I can play at 1080p is limited by the number of clients that I have. So I'm only able to get 5 streams to play on original quality which is 1080p with a bit rate of 27512kbps. The cpu is currently at 40% and network usage is around 80mb/s. Plex/Freenas server seems to be doing very well and could probably play twice this much.

Thank you very much for taking the time to try and stress test your setup! And how gratifying for you that you ran out of clients before the system even reached 50% - very nice!

Given how many people here use Plex on their FreeNAS, I would have thought that reports on various systems would be really useful, especially for people building something new. Perhaps I should start a new "How much Plex transcoding can your FreeNAS setup support?" thread?