Bear with me folks as I'm learning, this is going to be a fairly long reply because I'm still trying to soak it all up, and hopefully summarize all the research I've done so far such that if others are considering this configuration they can learn from my experience. :D
Most likely the included controller with that server will only support drives up to 2TB. I am not sure about the backplane support. If it has a SAS expander in it, that could be a problem too. I don't personally like to go with the Dell chassis because of some of the proprietary things they do. If you are already dead set on that, you should do more research on the system. Other people on the board surely will have something to say about this.
Thank you, Chris, for your assistance.
I ended up buying a Dell R510 on ebay for only $200. The price for everything included really couldn't be beaten by other competing options from what I could tell. This is my first time ever delving into FreeNAS. I'd like to try the R510 based upon the price and also I didn't see many SuperMicro servers with a lot of RAM, dual PSU, dual CPU, 12 x 3.5" HDD bays, etc. included without massively jumping the price. I have heard good things about SuperMicro, but I'd like to give the Dell a try first. Seeing that I didn't spend much on this box, I could always repurpose it for something else. That being said, I'd appreciate your input on the points below.
It has the following specs:
- 12 x front 3.5" drive bays
- Included activated license of Windows Server 2016 Standard(not sure what I might use it for, but I look at it as an extra bonus)
- 2 x 750W Gold-rated Dell PSU's (nice for redundancy)
- 96 GB of ECC RAM
- Over 1.2 TB of hardrive space spread across 10 SAS 15k enterprise drives
- Dual E5540 Xeon CPU's
- Qlogic 8GB Fiber Channel card (I'm planning on removing this as I don't have any other Fiber in my setup at this time)
- 4 extra 1gb NICS ports (2 PCI cards, each with 2 x 1gigabit ports)(I don't currently have a network switch that supports link aggregation, nor a client machine that has more than 1 gigabit NIC, so I'll probably remove these as well.)
- PERCH700i Controller w/512 cache (Has to be removed, not recommended per my reading of the forum and FAQ's.)
- Motherboard has 5 built-in SATA ports
Here's the Dell Service Tag lookup of my machine to see what's inside of it,
you can find it here.
I read the Dell R510 user manual
here (PDF link)
From what I can tell, the R510 included backplane is supposed to support up to 12 SATA/SAS drives. It is my understanding that the PERCH700i controller is a no-go for use with FreeNAS (I'm a newbie, but I did read the hardware FAQ where it clearly states to avoid the H700 due to it not having a true JBOD mode for drive pass-thru), and needs to be replaced with a card that has true I.T. mode.
So far, the list of contenders are:
I'm strongly leaning towards the IBM M1015 seeing how much support it seems to have from within the FreeNAS community. What's the old saying? If it isn't broken, don't fix it? Given that it is in so many FreeNAS machines, I'm going to make an assumption that the developers will keep this card in mind for future updates and compatibility testing.
Given that I have a Dell system, could there be any benefit to using the Dell H200 or H310 as compared to the IBM M1015? If there's no clear-cut case for going with one of the Dell cards, I'm going to just get a IBM M1015 and plug the two 2xSFF-8087 ports into my 12 bay Dell backplane. According to
cyberjock the moderator
in this thread, he said the card can support up to 32 physical drives, so I should be good, correct? I'd connect no more than 14 drives via the backplane. (12 x 3.5" HDD's bays in the front of the R510, plus up to 2 SSD's inside of the detachable caddy, see this pic for reference):
I plan on running FreeNAS off two SSD's in that detachable caddy. I haven't purchased any for that purpose yet, but I've had pretty good results with the Samsung 850 and 950 series in my desktop PC's. Any thoughts as to which SSD makes and models are recommended?
We have four systems at work (16 drive rack mounts) that were delivered with the WD Red Pro 4TB drives. They have been running 24/7 for about two years and I have only had to replace two drives. The Red Pros are alright, not great, but they will do. The best drives (in my experience) are the HGST drives. I have a unit with those in it that is at the 5 year mark with not even a slight hiccup. The ones we have are obviously not the latest model but I wouldn't hesitate to buy them in a new system. I have another system that has 60 of the Red Pro 6TB drives that is only about six months in and I have had to replace 3 of those. Could be because of all that vibration.
The thing is, you said that you already bought the drives, so why ask now? The time to ask was before you bought it.
Regarding the WD Reds, I'm not so much looking for validation of my purchase, but more so what other individuals have experienced seeing them run in larger arrays such as the 60 disk one you mentioned. I'm all about compiling information for future reference. I bought them a while back when I was thinking of building a Synology Diskstation with them, but decided I could get a higher degree of customization with FreeNAS, plus I prefer open source projects compared to Synology's quite underpowered hardware, even if it has an easy to use OS. I'm going to roll with the WD Reds in my Dell R510 for now, but I will certainly keep your HGST suggestion in mind should any of them fail.
I worry about this closet you are talking about though. You need plenty of cool air to keep the drives and the CPUs in that chassis cool. The WD reds we have run on the hot side. I have seen them hit 113F in normal use. We keep the room at 65F and the heat out the back of the server runs around 120F. You have to exhaust that heat and keep cool air going in the front all the time. Constant air flow, not like a normal home AC unit where it goes on and off. Let those drives get too hot and they will die very early. They are rated to a max of 60C but they don't like to live there all the time.
My goal is to build a closet door with a built-in fan that pulls air through a filtered mesh to keep dust intake down. Once the cool air is pulled in, I want a second fan to be ceiling mounted above the server rack pushing the rising hot air out to a roof vent. The closet is located inside of an insulated garage. The indoor air temps stay fairly low, even in the summer time. Combined with a push/pull fan combination with high-speed fans, I think it should be fine. If the temps rose higher than what the hardware is rated for, I could also mount an A/C unit inside of the closet. There's flexibility there, as this will be new construction and I'm consulting with my contractor as I build out the space.
The set of servers I had before I built the two I am using now had socket 1366 processors, which is what you are asking about. The best bang for the buck that I found was the Xeon x5670. It is a 2.93 GHz Six core processor and it doesn't run as hot as the 3.4 GHz unit or use as much power. These CPUs are a couple generations older and take more power, generate more heat and have just a bit less performance than newer CPUs. If this is what you want, If you feel like Dell is what you want or you have some vested interest in this model, go for it, but I think there are better options depending on your budget. Did you look at the recommended hardware list?
As noted above, I just went ahead and purchased the Dual E5540's that came with the R510. Yes, you're right, the older CPU's tend to have higher TDP ratings and consume more juice. This was something I've given thought to, but given I have no intention to run the server in a 24/7 environment, it didn't seem to make sense to purchase a brand-new latest gen Xeon at hundreds of dollars more per processor. I'd have to use my less-efficient ones for a decade or more before the outlay of cash caught up to the electric bill. :)
I did take a look at the Hardware List before purchasing.
I ended up choosing the Dell R510 that I went with for several reasons:
- Dell seems to have a very strong reputation for quality when it comes to reliability of their servers. I like things that generally work and have a reasonable expectation for them to last a long time.
- The Dell R510 was one of the very few 12 x 3.5" drive bay servers I could find in a 2U format, plus included 96 GB of ECC RAM for FreeNAS or if I ever decided to turn it into a VM box. I've read that you can't have too much ECC RAM with FreeNAS, especially with my multiple 8TB drives. Combined with dual CPUs, dual PSUs, included SAS drives and an activated copy of Windows Server 2016 and free shipping, I really couldn't say no to buying it at $200, even though I didn't have a chance to come back here and post this first. In an ideal world, I might be putting the cart before the horse.
That being said, I'd love to hear what you think are some better alternatives. I could always repurpose the R510 as a VM machine. What do you feel is a better configuration (be it price, performance, compatibility, reliability, etc.)? It seems SuperMicro is pretty popular. Are there any models that you'd recommend that have 12 or more 3.5" bays? I really want my FreeNAS box to have plenty of expansion space. If not SuperMicro, how about some other makes and models? While I have the R510 now, I certainly don't mind getting another machine. I consider it all a learning experience. :)
The Dell H310 controller can be cross-flashed with the IT firmware to make it into a plain HBA. I use those in my FreeNAS builds. Works great.
Thanks for the heads up. Do you have any comparison using the H310 versus say the IBM M1015 or the Dell H200? Performance? Reliability? Compatibility? How many machines have you deployed the H310 to? Any particular reason why you went with it over the M1015 seeing it is on the hardware list and seems to be all the rage around here?
I thought about getting one of those Dells when I setup my FreeNAS, but wound up with a Supermicro 8 Bay (2U) case with X8DTI-F mainboard and 2x Xeon E5620. I put a best offer on ebay for $160 shipped, didn't expect to get it but the seller accepted. The server works great, though I did have to flash the firmware to get it to recognize drives over 2TB. Definitely watch out for that, get the wrong MB and you will wind up having to buy a new controller.
sfryman,
Thanks for your advice. Congrats on getting a great deal. I know the feeling when you never expect a seller to accept and then when you wake up in the morning you get a Christmas present in your email from ebay. :D
Was there any particular reason why you leaned towards the SuperMicro over the Dell, or was it a coin toss and the seller just happened to accept your offer?
I installed six x 4TB "Red" drives and use it for backup, file storage and Plex. The server only came with 32GB RAM but I haven't run into any problems with that. Full disclosure: I never worked on a server before this one.
I've heard that for reliability and performance you can never have too much RAM with FreeNAS. Good to error on the side of caution and use the ECC memory and plenty of it. Been reading a few too many horror stories to want to buy a box without much RAM or non-ECC RAM. Not a risk I'm willing to take! :)
This is my first ever (maybe there will be a second one soon if someone convinces me to buy a SuperMicro or similar machine!) server build. I am completely new to server architecture and have spent dozens of hours reading the forums, google, youtube, etc. learning about it. Thankfully, there's never EVER been wrong information on the internet, so one never has to be too careful before you put your precious data on a new machine, right? ;)
I don't know about video editing but dual Xeon E5620 should be able to do everything else you have listed, and is basically free (~$20 on ebay). The most processor-intensive is transcoding. When I had 2x Xeon E5620 it could transcode any of my 1080p content fine, even multiple streams, but struggled with some of the 4k movies. The downstairs TV refuses to direct play 4k content with H.265, so I upgraded the processors to dual Xeon X5680 for another $125 (ebay, "best offer" auction again). So now it is able to handle real-time transcoding of everything in my library, though it cannot handle two x 4k streams simultaneously (understandably).
I don't plan on doing too much transcoding, as I'm going to rip my DVD collection and just cast it to the 1080p TVs. I'm going to need a bunch of different formats for tablets and cell phones, as I only really watch on the TV. Combined with the dual Xeons in there, I'm pretty sure that transcoding shouldn't be an issue. My curiosity is more so if I can offload some of my video editing encoding from my Windows client to this box, perhaps running a secondary OS on a separate drive? Who knows? Perhaps that is a question better asked in the Adobe forums, or I should get a separate server dedicated for that purpose altogether. I'll have to do more research to find out for sure.
People will point out that I'm using alot of electricity with this beast, but you should check the math before springing for new components. I use 150-160 watts under normal loads. A newer server would use less electricity, but cost at least a thousand more. I spent under $300 for my server. I'd have to keep it running for over a decade before the energy cost exceeds the price of new equipment.
Agree with CM on the heat/noise issues. You need a cool, isolated place for the 5600 servers.
You pretty much came to the same conclusion I did regarding up-front cost versus long-term cost to run it. I'll probably replace it with something else before the break even point of electricity ever comes into question. My garage is cool, insulated from the outside and thankfully doesn't share a wall with any room that people sleep in. I powered up my R510 for the first time today and I can confirm those fans are LOUD! :D :D :D
Thanks once again guys for the continued input. I'm really enjoying learning, and it is always encouraging when you have an active community of vets keeping a newbie on track.
Much obliged!