Apologies if this has been discussed to death! - NEW Hardware?

Status
Not open for further replies.

bravonoj

Cadet
Joined
Sep 8, 2013
Messages
7
Looking to build a new Freenas box in the near future and I am looking for suggestions - I will go with an HP or Dell server, probably Dell, but wanted to know from those "in the know" what is the most compatible, or if I should expect the have any issues? Any of the PERC controllers to look out for? Looking to do a rackmount server with probably 6tb of space to start with - hopefully RAID or something with redundancy. Suggestions for current mainstream hardware on the market right now?

Thanks!
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
Well, many HP/Dell servers won't run FreeBSD.

As for Perc cards you are on a small island with a few other people. Not exactly recommended hardware. May or may not have problems that will screw you over later. May or may not be supported in future builds of FreeBSD.

Personally, I go for the "do it right or not at all" when I start talking about my server and my data. So I went with stuff that is basically THE most recommended hardware. I'd rather not get it wrong and find out later when I lose my data.

Remember, there is no ZFS recovery tools. If it won't mount it's game over. You cant go buy Ontrack's data recovery software and recover zpools. It's just f*cked!
 

9C1 Newbee

Patron
Joined
Oct 9, 2012
Messages
485
What hardware, if any, do you have? How much are you willing to spend? How heavily will it be used?
 

ZFS Noob

Contributor
Joined
Nov 27, 2013
Messages
129
I'm running some tests on a FreeNAS install on an R710 right now. If it works I'll be doing another install on another Dell server I already have.

In my case I'm removing the PERC controllers and am going with SAS 6i's instead, reflashed to run the LSI BIOS that does passthrough so FreeNAS can access the drives directly. I couldn't make the controller recognize/acknowledge my SSD drive either; originally I thought it was because I had a SATA SSD mixed with SAS drives, but even with everything else unplugged it still wouldn't take it. And of course a nice clean rack server like that has a spare SATA port on the motherboard, but nowhere to pull power from. I ended up buying one of these as a work-aropund, but I don't yet know if it'll work.

If I was buying new I'd look at these guys: http://www.aspensystemsdirect.com/ As far as I know they sponsor the project, and the hardware looks like about what you'd want if you were going to piece it together yourself.

Really though: Dell makes good gear, but Dell sells servers with hardware RAID cards, and that's imperfect at best. I'd go Supermicro.
 

9C1 Newbee

Patron
Joined
Oct 9, 2012
Messages
485
Wow! You are not playing. Dell seems to get some sick demented pleasure out of handicapping their systems to perform other tasks. Let us know how your roll of the dice works out.
 

bravonoj

Cadet
Joined
Sep 8, 2013
Messages
7
OK - I'll bite and move from Dell or HP - let's say we have a budget of $5k including drives; what would your (2U) supermicro recommendation be? I can't make heads or tails out of their site.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
OK - I'll bite and move from Dell or HP - let's say we have a budget of $5k including drives; what would your (2U) supermicro recommendation be? I can't make heads or tails out of their site.

The expectation is that you'll provide your own hardware list to be critiqued. Generally people that say "show me how to spend $XXX" get no replies. ;)
 

ZFS Noob

Contributor
Joined
Nov 27, 2013
Messages
129
Well, let's look at requirements. You want to have 6TB of storage with room to grow, right? What sort of data are you hosting on this machine? VMs? Databases? Porn streaming for the houses on Frat Row at the University that employs you? (Probably not the latter - not enough space.)

Can you tell us a bit more about what you'll be doing with it?
 

bravonoj

Cadet
Joined
Sep 8, 2013
Messages
7
Agreed ;) - Here's my FreeNAS background before everyone starts flaming me. I run a 10tb home server that is purely DIY and is set as a JBOD array. Not what I am wanting to do for work. I wanted to go with a Dell or HP server initially becuase of the support I know I can get from them hardware-wise, but I know Supermicro is used all over the place and is a FreeNas favorite. There seem to be a lot of different sata cards to use, and of those, multiple modes to use them in. Browsing the forum, I see people that want to build a server with 384gb of Ram and 72 drives, and then some posts that want to know how to use a HP Gen8 Microserver. I have found very little in the middle ground here.

I did a bit more homework - and now let's say this is what I am looking at:

SuperMicro 2u dual PSU SuperChassis 826BE16-R920LPB
Supermicro X9SCA-O LGA1155/ Intel C204 PCH/ DDR3/ SATA3/ V&2GbE/ ATXIntel® Xeon® Processor E3-1230 v2
(8M Cache, 3.30 GHz)IBM ServeRAID M1015 SAS/SATA Controller
4 WD RE SAS WD2001FYYG - hard drive - 2 TB - SAS-2 drives
2 Transcend JetFlash 500 - USB flash drive - 8 GB for system boot
 

bravonoj

Cadet
Joined
Sep 8, 2013
Messages
7
lol, this box will be used primarily for storage of video archives, I do a lot of audio and video editing, and would like to offload the projects to the server once I am finished with the "working files". For example, the project I am working on now is in excess of 200 gb of video footage.

This system will also be used as an FTP server to share production files once they are completed

Basically an all around FTP/NAS server. No databases or VMs on this one (have EMC for that) but availability, speed and redundancy are important.
 

ZFS Noob

Contributor
Joined
Nov 27, 2013
Messages
129
What are you going to use this for? I ask because using 4 2TB drives looks like you're planning on using a RAIDZ1 solution, which I think is a really bad idea with modern drives. Here is an older article on it, but basically if a drive fails and you're rebuilding/resilvering the array and another failure happens, your data is toast. The drives you're looking at have a one in 100,000,000,000,000 chance of producing an unrecoverable read error for every bit read, but when you multiply that out by the TB of data you've got there's a non-negligible chance that RAID5/RAIDZ1 is gonna kill your data pool on a resilver. So don't do it.

You can go with two parity drives pretty easily by choosing RAIDZ2 instead, but there are still some trade-offs to examine:
  • RAIDZ vdevs can not be expanded. The way to add more space is to create another vdev (also RAIDZ?) and add it to the pool.
  • RAIDZ performance likely isn't what you think. Here's a blog entry from Oracle (or maybe it's rebranded Sun) that talks about this, but essentially a 6-drive mirrored stripe has random read performance of 6x the single drive performance. A RAIDZ2 set of six drives has the random read performance equal to one drive. That's a big hit for increased capacity.
  • I'd default to mirrors. So 6 2TB drives, or 4 3TB drives, or 4 4TB drives in a mirrored configuration. This also means you can reliably expand your storage later by adding 2 new drives to the machine and adding them to the pool, though there are some caveats related to performance if the existing drives are pretty close to capacity.
You also didn't mention memory. I'd probably start with 32G because why not, and I'd choose the densest memory available in order to make it simpler to add memory later.
 

bravonoj

Cadet
Joined
Sep 8, 2013
Messages
7
Sorry, been dealing with this crazy snow storm here in Philly today!

I hadn't thought of mirroring or RAID yet - I havent dealt with that since the days of the original HP DL380s...

so if I do 4 4tb drives, what configuration would give me good performance and the ability to add to the pool as needed? It sounds like if we do add drives it should be done before the existing drives fill up, correct? You say mirrored, what current configuration would that be to FreeNAS?

Agreed on the memory - it is inexpensive enough to load it up to begin with. I was thinking 8G sticks either 16g or 32g to start with.

Sorry if my questions sound inconsistent or ignorant, I've been out of the storage game for a long time!!
 

bravonoj

Cadet
Joined
Sep 8, 2013
Messages
7
I think I answered one of my questions - would I be looking to do a ZFS mirrored drive?
 

ZFS Noob

Contributor
Joined
Nov 27, 2013
Messages
129
so if I do 4 4tb drives, what configuration would give me good performance and the ability to add to the pool as needed? It sounds like if we do add drives it should be done before the existing drives fill up, correct? You say mirrored, what current configuration would that be to FreeNAS?
Think about it like RAID10. You'll have two mirrors, and ZFS will stripe across them. Since they're mirrored you'll lose half of your capacity, which means 16 TB native --> 8TB available.

The caveat about expanding zpools is this: ZFS doesn't distribute the data across the vdevs to balance things out.

I'm having trouble explaining things today, so I'll do it this way:
  • Start with a 4tb mirrored pair as a vdev for this example. You've got 2 drives, you mirror them, and you create a Zpool that just contains those drives.
  • Performance of this mirrored pair is X.
  • If you add a second mirrored pair, then you'll now have the Zpool consisting of two vdevs it can stripe across, so you've got 2 mirrored vdevs in the pool.
  • In this configuration, performance is 2X, because the ZPool is striping.
Pretty straightforward so far. More vdevs = better performance.
This breaks if you fill the first vdev up to 90% capacity then expand the pool. ZFS will put most of the new data on the new vdev, so reads won't be striped evenly - they'll be drawing from the new vdev (for newly written data) or the old one (for older data). So if you wait a long time to expand the pool, then capacity will grow but performance won't increase as you'd expect.
I hope that's clearer than it sounds. feels like it is. I probably need a nap...
 

9C1 Newbee

Patron
Joined
Oct 9, 2012
Messages
485
Status
Not open for further replies.
Top