New supermicro server

[Thomas102]

I plan A NAS for

• VM Host (I am on the way of NFS share but it should support iSCSI as well)
• SMB sharing (small load, mainly for backup/storage purposes)

Disk layout : 6x8TB in stripped mirror + one hot spare
L2ARC and SLOG.

The NAS should be expandable at "low" cost:

• I uses 8TB drives to leave empty space in the bay.
  
• RAM may be expanded to 128GB later based on real usage. L2ARC SDD has been sized for 128GB memory.

This is what i got from my seller :

• Motherboard: MBD-X10DRH-iT x10 motherboard
• CPU 2xE5-2620v4 : P4X-DPE52620V4-SR2R6 BDW-EP 8C/16T E5-2620V4 2.1G 20M 8GT 85W R3 2011 R0
• 8x8GB RAM: MEM-DR480L-SL02-ER24 8GB DDR4-2400 1Rx4 LP ECC REG DIMM
• 4xGB network : AOC-SGP-I4 Std LP 4-port GbE RJ45, Intel i350 (Retail Pack)
• 2x10GB optic network : AOC-STG-I2 Std LP 2-port 10G CX4, Intel 82598EB (Retail Pack)
• 7xHDD 8TB : HDD-T8TB-HUH728080ALE600 HGST 3.5" 8TB SATA 6Gb/s 7.2K RPM 128M 0F23267 ISE
• SSD (L2ARC) : HDS-2TV-MZ7LM960HMJP005 Samsung PM863a, 960GB,SATA 6Gb/s,VNAND,2.5",7.0mm,24nm
• SSD (SLOG, 2X) : HDS-2TV-MZ7LM240HMHQ005 Samsung PM863a, 240GB,SATA 6Gb/s,VNAND,2.5",7.0mm,24nm

Some thought:
I have to check with the seller because I did not find any lsi 3008 controler in the system

Server is overpowered with 2xE5-2620. But we might make use of docker or VM later.

About SLOG:

• I don't know if the PM863a are ideal for SLOG. If you have a replacement priced the same i'll ask for it. I know they are oversized with 240GB.
  
• This is not a critical system. I mean loosing a few hours of data is not that bad. I may go to disable synchronous writes if it helps resolving performances issues.
  
• Backup is much more important. Being able to quickly restart a broken service is the priority. I'll buy two server and must be able to switch between servers in a few hours

One provider proposed me WD RED pro 8TB, this one Hitashi. What do you think about thoose drives ?

Thanks.

 

[Arwen]

...
About SLOG:

• I don't know if the PM863a are ideal for SLOG. If you have a replacement priced the same i'll ask for it. I know they are oversized with 240GB.
  
• This is not a critical system. I mean loosing a few hours of data is not that bad. I may go to disable synchronous writes if it helps resolving performances issues.
  
• Backup is much more important. Being able to quickly restart a broken service is the priority. I'll buy two server and must be able to switch between servers in a few hours

...

Please note that a power loss prevention, (some abrivate as PLP), for an SLOG is to maintain the VM disk structure. NOT the ZFS structure. ZFS was purposefully designed such that is can survive power loss just fine. You may loss any data in flight, (aka not yet fully written). But that was the price ZFS paid to avoid FSCK, (file system checks).

However, when using another file system inside a ZFS volume, in order to maintain THAT file system's integrity, you have use sync=always. Then if performance is not suitable, you add a SLOG device WITH PLP. (And if worried about failure of a SLOG device during recovery, use 2 SLOG devices in a Mirror).

So using sync=none is a prime way to loose your Virtual Machine's volume.

 

[Thomas102]

Hi,
As per Samsung documentation PM863a includes PLP and have great endurance and speed. But I don't see it in freenas reference lists. This is why I wonder if it is ideal.

I agree with the technical points but sync writes degrades performance to improve reliability.
The tradeoff is not about sync writes being good or bad. It is about requirements.

 

[Arwen]

Hi,
As per Samsung documentation PM863a includes PLP and have great endurance and speed. But I don't see it in freenas reference lists. This is why I wonder if it is ideal.
...

Sounds like a reasonable device for a SLOG. Just pay attention to the write speed. A SLOG is basically a write only device unless recovery is required.

...
I agree with the technical points but sync writes degrades performance to improve reliability.
The tradeoff is not about sync writes being good or bad. It is about requirements.

Well, my point is that without sync=always, you run the risk of a crash of the FreeNAS server causing the VM disk structure to be messed up. Even messed up to the point of needing a full restore. Thus, if you run without syncs, backups are vitially important. Preferably daily backups.

 

[Chris Moore]

I plan A NAS for
This is what i got from my seller :

Thanks.

Sorry, you already bought everything? I don't see a question here.

If you have a question, you should ask it before you buy because often people make bad choices for wrong reasons when they don't ask questions first.

 

[nightshade00013]

One thing I am going to chime in on is the L2ARC. You only have 64GB of ram so it's not going to be that useful. It would probably be better to run 8 X 16GB sticks of ram and nix the L2ARC until you see what things are doing and even then doubling the ram to 16 X 16GB of ram before adding the L2ARC would be a better way to do it. It is always preferable to have more ram than to add L2ARC if you can manage it since the L2ARC's metadata will eat the ram leaving less space available for ARC. Also for 64GB of ram a 960GB L2ARC probably is not correctly sized either https://forums.freenas.org/index.php?threads/at-what-point-does-l2arc-make-sense.17373/

The other is the added network controllers. The board has two 10Gbit ports already available and with a proper switch they will easily outperform the four 1Gbit ports and you are also going to add a 10Gbit network card so you will have 8 network connections.... Maybe need to rethink all of that and save the cash for the RAM. The only way I can think of even being able to use that much throughput would require a major SSD based rig, the hard drives you are planning to use will be the bottleneck even with one 10Gbit port.

The board does not have a SAS controller built in so you will need to pick one up if it is desired. The board does have 10 sata ports though so as long as they are used in AHCI mode they should work fine.

 

[Thomas102]

Thanks all for you answers.

Well, my point is that without sync=always, you run the risk of a crash of the FreeNAS server causing the VM disk structure to be messed up. Even messed up to the point of needing a full restore. Thus, if you run without syncs, backups are vitially important. Preferably daily backups.

Backups are vitals anyway. I feel like assuming less backup is required because sync writes are enabled is bad. Also some VM are very resilient to power interruption.
So I may prefer a specific VM to be restored from a backup once in a while than the same VM useless because of performance issue. FreeNAS allows this because sync writes is a per dataset setting.

Sorry, you already bought everything? I don't see a question here.

If you have a question, you should ask it before you buy because often people make bad choices for wrong reasons when they don't ask questions first.

I bought nothing yet. I'm on this project for about 2 month and still trying to figure what will work or not :)

But yes I have this kind of (hard) question :)
The 6x8TB stripped mirror is a tradeoff.
For best performances I should go with 16x3TB and NearLine SAS instead of SATA.

I assume the new NAS will fullfill the job whatever the disk layering is (Actual "kind of SAN" is RAID6)
I assume (?) SATA entreprise HDD and Nearline SAS are reliable the same.

It is very hard for me to predict future performance requirements.
I then conclude the 6 disks will make it easier for me to extend the NAS with what will be required then (might be from nothing to SSD arrays).

Does it make sense ?

 

[Thomas102]

Nightshade00013,

I turned my mind a lot. and it's quite long to get a quote during holidays, I agree the current cotation is not perfect.

ARC/L2ARC,
I think 64GB is enough for now. I may upgrade to 128GB if it is not the case. Actually I plan to partition the SSD with 300GB L2ARC (rest of the SSD unused).

About network I need 4 RJ45, connexions. not all being 10Gbps.

I did'nt get it a first but the motherboard should be the CLN4 version with a 3008 onboard.