Write performance ZIL Or Memory

[dennisjoh]

I have a FreeNAS 9.3 ZFS server with 2x 4 Tb disk and 8 Gb memory using as backup server for big snapshot images from Proxmox (10G - 500G files) mounted with NFS.

We need a high write speed to the FreeNAS server. At this moment the write speed is to low.

- The amount of memory is applicable for the write speed? or only for the read speed? How more memory how faster the write performance? Also for big files?
- Is it usefull to add a SSD for SLOG/ZIL?
- If i add a SSD for SLOG/ZIL we need NFS change from async to sync? Also for big files?

Read speed is not necessary.

Thanks!

 

[Spearfoot]

I have a FreeNAS 9.3 ZFS server with 2x 4 Tb disk and 8 Gb memory using as backup server for big snapshot images from Proxmox (10G - 500G files) mounted with NFS.

Please tell us more details about your server, per the forum rules.

We need a high write speed to the FreeNAS server. At this moment the write speed is to low.

Can you quantify what you mean by 'low'? 10MBps? 100MBps? If you're using gigabit ethernet, it isn't possible to get much more than ~115MBps. If you're already saturating your network connection, there's nothing you can do to speed up the transfers except move to a faster networking topology (e.g., 10Gigabit).

- The amount of memory is applicable for the write speed? or only for the read speed? How more memory how faster the write performance? Also for big files? - Is it usefull to add a SSD for SLOG/ZIL? - If i add a SSD for SLOG/ZIL we need NFS change from async to sync? Also for big files? Read speed is not necessary. Thanks!

ZFS loves RAM, and 8GB is the minimum recommended amount of RAM for a basic file server; using more would probably help -- unless, as I mentioned above, you're already saturating your network connection.

Are you running your NFS dataset with synchronous writes disabled? If so, your system is already writing as fast as it possibly can (albeit unsafely). With synchronous writes enabled it will run slower, even with a SLOG device.

 

[dennisjoh]

I have now 75 MBps write speed.

Server specs:
Supermicro server
Intel(R) Xeon(R) CPU L5410 @ 2.33GHz
4x 2 Gb DDR-2 ECC reg memory
1 Gbit network speed.
SATA on-board
FreeNAS-9.3-STABLE-201503270027
2x 4 Tb disk TOSHIBA MD04ACA400

ZFS loves ram also for write speed or only for read performance?


zfs set sync=disabled tank/dataset for disabled sync writes?

 

[Spearfoot]

I have now 75 MBps write speed.
Server specs:
Supermicro server
Intel(R) Xeon(R) CPU L5410 @ 2.33GHz
4x 2 Gb DDR-2 ECC reg memory
1 Gbit network speed.
SATA on-board
FreeNAS-9.3-STABLE-201503270027
2x 4 Tb disk TOSHIBA MD04ACA400

What motherboard model? In particular, how many memory slots does it have? It will be more expensive to upgrade your memory if all of the system's RAM slots are already filled -- you would have to pull the 2GB sticks and replace them with higher-capacity units.

Also, how do you have the 2 x 4TB disks configured? In a mirrored vdev? That's the only 2-disk topology that makes sense for your use-case.

ZFS loves ram also for write speed or only for read performance?

Both...

zfs set sync=disabled tank/dataset for disabled sync writes?

To see the current sync setting for a dataset (in these examples I will use dataset 'vmware_nfs' in pool 'tank'):

Code:

zfs get sync tank/vmware_nfs

To set the sync setting, the choices are standard, always, or disabled. To disable synchronous writes:

Code:

zfs set sync=disabled tank/vmware_nfs

Run a quick test: disable synchronous writes on your NFS dataset and set if your transfer rate increases.

 

[dennisjoh]

Supermicro X7DCU (6 ram slots)
16 Gb ram costs 40 dollar (4x 4gb)

2x 4 Tb (not mirrored) (8 Tb diskspace)


I test the speed with sync disabled tonight.

Thank you!

 

[Spearfoot]

2x 4 Tb (not mirrored) (8 Tb diskspace)

Not mirrored? Then how are they configured? The whole point of using FreeNAS & ZFS is to gain performance with redundancy and safety. Running single-disk vdevs defeats the whole purpose...

 

[dennisjoh]

Configged in FreeNAS as stripe.


pool: nfsbackup
state: ONLINE
scan: scrub repaired 0 in 6h26m with 0 errors on Sun Jun 26 06:26:46 2016
config:

NAME STATE READ WRITE CKSUM
nfsbackup ONLINE 0 0 0
gptid/7958ae7e-393a-11e5-b934-003048bb1d68 ONLINE 0 0 0
gptid/79cc3a1f-393a-11e5-b934-003048bb1d68 ONLINE 0 0 0

errors: No known data errors


Redundancy is not needed. It is a backup server.

 

[Spearfoot]

Redundancy is not needed. It is a backup server.

This doesn't make any sense to me. I would think you'd want redundancy on your backup more than anyplace else!

But that's another issue... right now we're trying to improve your data transfer rate.

 

[Ericloewe]

If you're using gigabit ethernet, it isn't possible to get much more than ~115MBps.

Well, decent GbE gear will squeeze out 930-940Mb/s for TCP. 920Mb/s sounds like a reasonable target, though.

 

[jgreco]

4x 2 Gb DDR-2 ECC reg memory
1 Gbit network speed.
SATA on-board
FreeNAS-9.3-STABLE-201503270027
2x 4 Tb disk TOSHIBA MD04ACA400

ZFS loves ram also for write speed or only for read performance?

How full is the pool? Sync write issues aside, ZFS will tend to be faster at writing if there's more free space on the pool, if the pool fragmentation is lower, and if it has more memory.

 

[SweetAndLow]

we still don't know what speeds they are getting or how they are testing speeds. Please include this data, you might be getting the max speed you can get.

 

[dennisjoh]

The pool is 82% full now.

I test the speed with a snapshot backup over NFS with Proxmox (real test).
I have now enable LACP and sync disabled i get max. 1,5 Gbps (different clients at the same time).