ZFS Server plan and questions

[LIGISTX]

I am entirely new to freenas, and linux itself for that matter. I have done a pretty solid amount of research on this forum, and I hope this isn't too stupid of questions, but I am just unsure on a few things still. Also, I won't be buying/building this system for a few months, so things easily may change if drive prices change.

This is my current plan:
8x4 TB WD greens (unless someone confirms this no longer works, in which case I will man up and get Reds)
Intel Pentium G3240 (I already have one sitting around, and it supports ECC RAM, so we good!)
SUPERMICRO MBD-X10SLL-F-O uATX Server Motherboard
Crucial 16GB (2 x 8GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1600 (PC3 12800) Server Memory Model CT2KIT102472BD160B
Fractal Design Define R5
SeaSonic 450 watt PSU

My first question is probabaly one of the more asked question, is this enough RAM? I will be running a RAIDZ2 setup, so that will grant me ~24 TB of raw space, 32 TB of total disc space. This isn't an enterprise system, it is a media server and daily backup server for myself, and myself only. It will not get hit by any more than 1 or 2 PC's at a time, and those "hits" will be accessing my media files, be it movies, tv, pictures, or music. Although, I do want to saturate my gigabit network... There will not be too many times when it is being used for reads and write at the same time, but think of this is a file server, where all of my data is stored, I have SSD's in my main RIG and HTPC to handle programs and games.

Second, is this a good PSU choice? I know the watts should be fine, but is that a quality PSU that should last in a 24/7 deployment situation? Opinions would be great!

Third, I know people have said this should be a fine low cost CPU (I don't plan on running deduping or encryption, I know this won't handle compression either, but I wonder if compression is a big deal... is it?) Additionally, can I encrypt the drives or set up compression after I set it up assuming I buy a Xeon down the road?

I am building this as a bit of an overkill system that should last me a long time. 20+ TB's is a lot of storage space, but I currently have about 13 TB of data, but to put that in perspective, about a year ago I had 11 TB of data, it doesn't grow too fast at this point. Hopefully this hardware lasts until its time to replace all the drives with new and larger drives, and potentially new hardware backbone as well, but that is for the future.

 

[Wyl]

I got 2 WD greens first time with my NAS Setup. Actually I didn't realise it but I purchased 4 Reds and 2 Greens thinking they were Reds on sales. Stupid me for not checking the HDs during install.

Anyway.... the two greens starting showing SMART errors within 4 months of usage. I replaced them with Reds very, very quickly.

You might have better luck then me?

 

[BigDave]

Just one comment about your board choice.
You mentioned in your intro post about a M1015 HBA card.
If your plan is a total of eight drives, why not take a look at this board.
Other than that, you sound like you have done your homework;)

 

[LIGISTX]

I got 2 WD greens first time with my NAS Setup. Actually I didn't realise it but I purchased 4 Reds and 2 Greens thinking they were Reds on sales. Stupid me for not checking the HDs during install.

Anyway.... the two greens starting showing SMART errors within 4 months of usage. I replaced them with Reds very, very quickly.

You might have better luck then me?

Did you do the hack with WDIDLE3.exe to your greens?

 

[LIGISTX]

Just one comment about your board choice.
You mentioned in your intro post about a M1015 HBA card.
If your plan is a total of eight drives, why not take a look at this board.
Other than that, you sound like you have done your homework;)

You know what, I looked at that board and something in my head told me that cost 200 bucks more than the board I already chose. Not sure what math I was doing... That board looks like a better options, cost about the same as buying the M1015, and it has one less point of error, less heat, and less watts. Thanks!

Now any input on pool performance with 16 gigs of RAM?

 

[BigDave]

You know what, I looked at that board and something in my head told me that cost 200 bucks more than the board I already chose. Not sure what math I was doing... That board looks like a better options, cost about the same as buying the M1015, and it has one less point of error, less heat, and less watts. Thanks!

Now any input on pool performance with 16 gigs of RAM?

With your anticipation of low usage, I'd say 16GB (8GB x 2) would be fine, you could always upgrade to Xeon and 16GB more in the future.

 

[LIGISTX]

Another question, just a forward thinking question. How hot do these server type mobo's get? I am pretty used to the heat a high end desktop board pumps out with with a decently overclocked CPU (and the answer is really not too much), but do I need to think about spot cooling the heatsinks on the mobo? Like I have said, not a high duty system, but I also could be writing MANY gigs to it at a time which could take a while, even over gigabit, don't need the motherboard controllers getting to hot and putting things in jeopardy.

Also, I have a Samsung 840 128GB literally sitting on my desk doing nothing, (long story short, amazon messed up...) Any way I could put it to good use in the server? I know the OS will be on a flash drive, and using the SSD as a cache without enough RAM can be an issue, but is there anything it would be useful for?

 

[BigDave]

Another question, just a forward thinking question. How hot do these server type mobo's get? I am pretty used to the heat a high end desktop board pumps out with with a decently overclocked CPU (and the answer is really not too much), but do I need to think about spot cooling the heatsinks on the mobo? Like I have said, not a high duty system, but I also could be writing MANY gigs to it at a time which could take a while, even over gigabit, don't need the motherboard controllers getting to hot and putting things in jeopardy.

Also, I have a Samsung 840 128GB literally sitting on my desk doing nothing, (long story short, amazon messed up...) Any way I could put it to good use in the server? I know the OS will be on a flash drive, and using the SSD as a cache without enough RAM can be an issue, but is there anything it would be useful for?

If you provide enough air flow to cool your drives in a manufactued computer case, the board will be fine. I have a large water cooled puter like yours and I
can tell you that my server doesn't even come close to needing that amount of cooling.
The 840 SSD could be the boot drive! You don't have to use a flash drive. The reason most folks use a flash drive is to save a SATA port.

 

[LIGISTX]

If you provide enough air flow to cool your drives in a manufactued computer case, the board will be fine. I have a large water cooled puter like yours and I
can tell you that my server doesn't even come close to needing that amount of cooling.
The 840 SSD could be the boot drive! You don't have to use a flash drive. The reason most folks use a flash drive is to save a SATA port.

Is there ANY reason to use it as a boot drive tho? lol. Only because it does make a very handy large and USB saturating usb "stick" (I have a usb adapter just plugged into it). Since the boot drive is pretty much read only, I feel like I would be throwing away an SSD by using it for that. I was thinking maybe I could put it as its own vdev (sorry, I am still fuzzy on the lingo), not in the Z2 array, and leave it as a super frikin fast scratch disc in case my Z2 pool(also, right lingo? I dunno) doesn't saturate my gigabit connection.

I would leave like my pictures on the SSD, and documents and such, things that I would like to get at RIGHT NOW, but also have that data backed up on the Z2 array as well. I don't know, thats sort of a spit ball idea, especially because I am assuming the Z2 array WILL saturate gigabit...

 

[BigDave]

Since 9.3 the boot drive does get written to. There're many peeps who use SSDs for boot drives, if
you consider it a waste that's ok. I suggested it only because it sounded as if it didn't cost you $.
I was "taking all the way" on that spit ball :p

I was thinking maybe I could put it as its own vdev (sorry, I am still fuzzy on the lingo), not in the Z2 array, and leave it as a super frikin fast scratch disc in case my Z2 pool(also, right lingo? I dunno) doesn't saturate my gigabit connection.

You're kinda getting away from the file server/storage with redundancy thing ;)

 

[LIGISTX]

Since 9.3 the boot drive does get written to. There're many peeps who use SSDs for boot drives, if
you consider it a waste that's ok. I suggested it only because it sounded as if it didn't cost you $.
I was "taking all the way" on that spit ball :p

You're kinda getting away from the file server/storage with redundancy thing ;)

O I didn't know it got written too. How is that then redundant...? And yes it is free, but if there is no gain to be had there is no point you know...?

What gain would I get from having a boot SSD?

And you are right about the getting away from redundancy thing on your last point...


Sent from my jailbroke iPhone using Tapatalk

 

[BigDave]

I read alot about flash sticks failing in the forums here, happens every day it seems.
When I updated to 9.3, I purchased a 16GB DOM, which is basically a small capacity
flash drive that plugs directly into a SATA port. Much more reliable, better quality.
In 9.3 you can mirror your boot drives for redundancy and many folks have done this.
If you use the SSD as your boot drive, you get reliable and faster boot device. Is it needed?
No, not really, but then a water cooled video card at stock speeds is kinda overkill too, right?

 

[LIGISTX]

I read alot about flash sticks failing in the forums here, happens every day it seems.
When I updated to 9.3, I purchased a 16GB DOM, which is basically a small capacity
flash drive that plugs directly into a SATA port. Much more reliable, better quality.
In 9.3 you can mirror your boot drives for redundancy and many folks have done this.
If you use the SSD as your boot drive, you get reliable and faster boot device. Is it needed?
No, not really, but then a water cooled video card at stock speeds is kinda overkill too, right?

To be fair, my GPU is OCed to the max ;). Haha. But I understand your point.


Sent from my jailbroke iPhone using Tapatalk

 

[Wyl]

Did you do the hack with WDIDLE3.exe to your greens?

No, It was a simple mistake, which is why I replaced them with reds.

 

[LIGISTX]

No, It was a simple mistake, which is why I replaced them with reds.

Gotcha.


Sent from my jailbroke iPhone using Tapatalk

 

[BigDave]

To be fair, my GPU is OCed to the max ;). Haha. But I understand your point.

Well I kinda figured that, given your proc is 4.4GHz

 

[LIGISTX]

Well I kinda figured that, given your proc is 4.4GHz

Meh, 4.4 is easy. It's just the Intel TIM and sealer around the ihs is crap. I am at ol volts, but temps are not high, but warmer than I would love... Thus I don't push farther. If I delid it I would, but I don't care enough. 4.4 is fine... Lol


Sent from my jailbroke iPhone using Tapatalk

 

[BigDave]

Well my 4770 was not a chip lotto winner either
But it was fun to see how far you could go though. My rigs looks were far more important
than how fast it would fly, I just don't game any more. Some times I look over at this
Behemoth and just think how many servers I could have built with that coin.
Maybe one day I'll part it out on ebay and build a kick butt server with the proceeds :)

 

[LIGISTX]

So knowing that I shouldn't fill it up past 80%, im thinking I may need more space...

I know I read it somewhere, but I am still a bit confused and unsure of the actual negative implications.

I think I want to run a 9x4TB system, but is having the odd number of drives bad for performance? Remember, I am trying to saturate gigabit, but with one or two users at a time max, not high workload environment.


Sent from my jailbroke iPhone using Tapatalk

 

[Bidule0hm]

Don't worry, you'll easily saturate a gigabit link, if you don't then there's something really wrong ;)

If compression is enabled (and it is by default) then the number of drives doesn't matter for performance.