Sanity check.

Status
Not open for further replies.

Huib

Explorer
Joined
Oct 11, 2016
Messages
96
Hi guys,

I'm thinking to deploy a FreeNAS to replace a windows file server.
My first idea was to keep working on the windows machine and then "back up" using rsync or something similar to the FreeNAS box.
From there I would have the ability to make snapshots etc. However, scanning the windows server seems expensive so now I'm actually thinking to replace the windows server outright.
I might make a weekly backup of a windows virtual machine also (our domain pc)

I'm using the word backup, but I just want a better rollback solution than we have now.

anyway.
In my company we need to server about 10 workstations. Most is cad/cam with reasonable file sizes, but we have 3 persons that are working in a design package that uses many many small files (hundreds of thousands small files)

That's the intended use part out of the way.

The intended system is:
case Fractal design Define R5. (due to noise... It's not a NAS case so I know replacing drives will be a bit of a pain. I can live with that)
Motherboard: SuperMicro X11SSL-F.
power supply: EVGA SuperNOVA 650 P2, 650 watt, 80 plus platinum.
4x 8 GB Samsung DDR4, 2133 Mhz unbuffered ECC M391A1G43DBo-CPB (32 GB in total).
Intel core I3-6300, 3.8 Ghz.
6 x Western Digital Red 3TB drives to be configured as raidZ2 for 12TB usable space.

I might replace the 4x8 GB to 2x16 GB ram sticks if I can find them for future upgrade to 64 GB if that's needed.

Any remarks would be great!

Thank you in advance!
 
Last edited by a moderator:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Motherboard: SuperMicro X11SSL-F.
Why not the X11SSM-F? The two extra SATA ports are extremely useful.
4x 8 GB Samsung DDR4,
That makes no sense. Get two 16GB DIMMs instead.
x Western Digital Red 3TB drives to be configured as raidZ2 for 12TB usable space.
You should realize that you'll be closer to 10TB than 12TB. Those "3TB" drives are actually 3*0.909TB.
Look around for @Bidule0hm 's calculator.
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
My first idea was to keep working on the windows machine and then "back up" using rsync or something similar to the freenas box.
we have 3 persons that are working in a design package that uses many many small files (hundreds of thousands small files)

The performance requirement difference for working on the NAS directly and using it as a backup offloading station is pretty significant, having impact on the drive setup.
It might be good to keep in mind. Working on files directly on the FreeNAS box will likely benefit from using mirrors, while the backup target is fine with RaidZ2.
 

Huib

Explorer
Joined
Oct 11, 2016
Messages
96
Why not the X11SSM-F? The two extra SATA ports are extremely useful.

That makes no sense. Get two 16GB DIMMs instead.

You should realize that you'll be closer to 10TB than 12TB. Those "3TB" drives are actually 3*0.909TB.
Look around for @Bidule0hm 's calculator.

Thanks for the tip regarding the SSM-F. For the price difference that seems extremely useful indeed!

I was thinking that 4 dims might get more throughput than 2. I'm not sure if that still works that way though and if it will even help in FreeNAS.

I know that it will be 12 "commercial" gigabytes :)

Thanks for the reply!
 
Last edited by a moderator:

Huib

Explorer
Joined
Oct 11, 2016
Messages
96
The performance requirement difference for working on the NAS directly and using it as a backup offloading station is pretty significant, having impact on the drive setup.
It might be good to keep in mind. Working on files directly on the FreeNAS box will likely benefit from using mirrors, while the backup target is fine with RaidZ2.

Thank you for the warning!

I still don't get the mirroring speed improvement though.
Assuming 6 drives
In raidz2 you are writing to 4 drives that help in the speed aspect and you drive to 2 drives for parity (not exactly since parity is divided over multiple drives also, but I'm trying to understand where the speed gets lost) is the parity so costly that you don't get the speed boost of this additional devices to drive to?

With mirroring, do I understand that you are writing to two sets of 3 striped disk that are mirrored?
That's like a stripe of 3 disks performance wise. But if you lose 2 disks in one set you lose all data. That sounds less safe than raidz2.....
Is it faster because there is no parity information?

Where is my mistake?

Thank you!
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
I've heard of people encountering performance problems with samba and CAD / CAM applications. (As in Samba was more or less unusable for the purpose). You'll want to do some small-scale testing with samba and your applications prior to committing to purchasing hardware. You can probably just use a VM with some sort of *nix and samba 4.3 for testing.

You don't want to be in a position where you have over-promised and under-delivered. It makes for awkward conversations with management.
 
Last edited:

Huib

Explorer
Joined
Oct 11, 2016
Messages
96
I've heard of people encountering performance problems with samba and CAD / CAM applications. (As in Samba was more or less unusable for the purpose). You'll want to do some small-scale testing with samba and your applications prior to committing to purchasing hardware. You can probably just use a VM with some sort of *nix and samba 4.3 for testing.

You don't want to be in a position where you have over-promised and under-delivered. It makes for awkward conversations with management.

Thank you!
However I'm my own client so I will be lenient against myself... I promise ;)

Windows server is actually also extremely bad at handling small files. So I'm not sure how much worse it could be...

I can do a test (I already am actually) but not with the amount of drives I'm intending to use. And I don't believe that a VM will give the same results performance wise as a barebone implementation (in the end you write to disks of the hardware the VM is running on. That's not the same as 6 bare drives I suppose).

on a VM it works ok (host has a SSD that saturates the 1 GB nic for larger files).
On a actual barebone server with 3 drives in raidZ it works euh.... with different results. between 25 and 80 MB per second transfer speeds. I was hoping the additional drives and 32 GB ram instead of 8 would push this up a little... But I can't know for sure until I actually build the server. If it stays at this speeds it would be workable. Not optimal, but workable

If i want to do a true test in the end I think I will have to bite the bullet. I don't have 6 drives lying around to do the test without investing.

Do I have any stupid assumptions in the above?

Thanks!
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
I was thinking that 4 dims might get more throughput than 2.
No. LGA115x processors have two RAM channels, each supporting up to two DIMMs. Any performance difference will happen in pairs.
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
Windows server is actually also extremely bad at handling small files. So I'm not sure how much worse it could be...

Is that a challenge? ;)

I can do a test (I already am actually) but not with the amount of drives I'm intending to use. And I don't believe that a VM will give the same results performance wise as a barebone implementation (in the end you write to disks of the hardware the VM is running on. That's not the same as 6 bare drives I suppose).

on a VM it works ok (host has a SSD that saturates the 1 GB nic for larger files).
On a actual barebone server with 3 drives in raidZ it works euh.... with different results. between 25 and 80 MB per second transfer speeds. I was hoping the additional drives and 32 GB ram instead of 8 would push this up a little... But I can't know for sure until I actually build the server. If it stays at this speeds it would be workable. Not optimal, but workable

RAIDZ may not be a good choice... especially if you have 10 simultaneous users. Once you get hardware, you'll want to test with striped mirrors and RAIDZ. You'll also probably want to get 2* 16GB DIMMs instead of 4* 8GB DIMMS. This will give you the option of increasing the amount of RAM in your system if you discover that 32GB is insufficient. Read and write performance increases as RAM increases. Another motherboard option to look into is the X10SRL with an E5-1620 (10 SATA, lots of memory). If it were me, I'd also look into getting at least a 12 bay rackmount chassis because you never know when you'll want to add more drives:D

You'll also want to look into the optimizations for small files that are at the bottom of the SMB tips resource - https://forums.freenas.org/index.php?resources/smb-tips-and-tricks.15/
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
Where is my mistake?
Have you read through all the links in my signature? In particular the newbie related.
 

Huib

Explorer
Joined
Oct 11, 2016
Messages
96
Have you read through all the links in my signature? In particular the newbie related.
Not all of them look familiar.
I guess I have more reading to do :)
I AM trying to do my homework though....
 

Huib

Explorer
Joined
Oct 11, 2016
Messages
96
Is that a challenge? ;)

I do not doubt your skills whatsoever :)
As i understand from "experts" is that the problem is that we have a mix of very small files (several bytes) and files between 2 mega and 2 gigabyte.
Apparently it's hard to optimize for everything at once. If you have tips though... But that is off topic.


RAIDZ may not be a good choice... especially if you have 10 simultaneous users. Once you get hardware, you'll want to test with striped mirrors and RAIDZ. You'll also probably want to get 2* 16GB DIMMs instead of 4* 8GB DIMMS. This will give you the option of increasing the amount of RAM in your system if you discover that 32GB is insufficient. Read and write performance increases as RAM increases. Another motherboard option to look into is the X10SRL with an E5-1620 (10 SATA, lots of memory). If it were me, I'd also look into getting at least a 12 bay rackmount chassis because you never know when you'll want to add more drives:D

You'll also want to look into the optimizations for small files that are at the bottom of the SMB tips resource - https://forums.freenas.org/index.php?resources/smb-tips-and-tricks.15/

Yeah I will try to find more reading regarding mirror sets and raid.

Ram is clear already 2x16 is the way to go.

I'll also check the X10SRL.

Thank for all the tips to all of you.

Time to do some more homework!!
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
As i understand from "experts" is that the problem is that we have a mix of very small files (several bytes) and files between 2 mega and 2 gigabyte
Don't rule out (yet) the option of two pools with different vdev layouts.
 
Status
Not open for further replies.
Top