Poached_Eggs
Dabbler
- Joined
- Sep 17, 2018
- Messages
- 31
I've been messing with Freenas on and off for almost a year. After going down the Inverted triangle of doom - I shy'ed away from a glusterfs 2 node replica +arbiter - (The cost just wasn't justified with doing massive duplication efforts, which was just plain stupid for home lab).
I was 95% settled into a Primary & Backup Freenas - 2x HP DL380 G8 - Optane 900 Slog, 12x 4TB Ironwolf. - I could get 500MB/S to 600MB/S up/down - and with the slog for my VM's - the numbers were even better with NFS. (
Of course - What falls in my lap a week or 2 before I finalize my plans - 2x JBOD trays of 24x 400GB Intel s3700 SSD. - so of course back to drawing board.
----------------------------------------
My current config: Each tray is 4x Vdev of Raidz2 (6 drives). So I was of course hoping that with 48x SSD drives my read/write over 10gb should max out...
My write are no better then with my 12x 4tb ironwolf's. My reads via SMB from freenas to desktop are 950MB/S +, but writes are still 550ish. And NFS read/write (without slog) are 550MB/s read - 650MB/s write. - With slog it goes to 850MB/s write, but still 550ish read.
Are my expectations unreasonable with this setup? Is my hardware not capable? A configuration error on my side? Tried with/without autotune - tried with jump packets all the way through)
Iperf numbers are good all which ways. Desktop has X550-T and using stripped nvme and a OCZ Z Drive for copy to/from. (Benc mark 2.86GB/s Read & Write)
(Would have LOVED to use these OCZ Raid-Z 3.2 PCIE cards for NFS storage but it seems freebsd doesn't recognize them)
Any suggestions are appreciated - Even telling me to sell the SSD arrays and just keep the spinners..
Server - HP DL380e G8 12+2
1x E5-2407 v2 @ 2.40GHz
96GB ECC Ram
Internal: LSI 9211-8i
External: LSI 9201-16e
Intel Optane SSD 900P
Nic: 560SFP+
2x JBOD:
Super Micro: BPN-SAS2-216EB w/ SAS2-216EL1
I was 95% settled into a Primary & Backup Freenas - 2x HP DL380 G8 - Optane 900 Slog, 12x 4TB Ironwolf. - I could get 500MB/S to 600MB/S up/down - and with the slog for my VM's - the numbers were even better with NFS. (
Of course - What falls in my lap a week or 2 before I finalize my plans - 2x JBOD trays of 24x 400GB Intel s3700 SSD. - so of course back to drawing board.
----------------------------------------
My current config: Each tray is 4x Vdev of Raidz2 (6 drives). So I was of course hoping that with 48x SSD drives my read/write over 10gb should max out...
My write are no better then with my 12x 4tb ironwolf's. My reads via SMB from freenas to desktop are 950MB/S +, but writes are still 550ish. And NFS read/write (without slog) are 550MB/s read - 650MB/s write. - With slog it goes to 850MB/s write, but still 550ish read.
Are my expectations unreasonable with this setup? Is my hardware not capable? A configuration error on my side? Tried with/without autotune - tried with jump packets all the way through)
Iperf numbers are good all which ways. Desktop has X550-T and using stripped nvme and a OCZ Z Drive for copy to/from. (Benc mark 2.86GB/s Read & Write)
(Would have LOVED to use these OCZ Raid-Z 3.2 PCIE cards for NFS storage but it seems freebsd doesn't recognize them)
Any suggestions are appreciated - Even telling me to sell the SSD arrays and just keep the spinners..
Server - HP DL380e G8 12+2
1x E5-2407 v2 @ 2.40GHz
96GB ECC Ram
Internal: LSI 9211-8i
External: LSI 9201-16e
Intel Optane SSD 900P
Nic: 560SFP+
2x JBOD:
Super Micro: BPN-SAS2-216EB w/ SAS2-216EL1