ndboost
Explorer
- Joined
- Mar 17, 2013
- Messages
- 78
Alright so here is my layout..
freenas 9.2.1.6-RC
Intel G2030
32GB DDR3-1600 ECC Kingston
Super Micro X9SCM-F
Intel Pro 1000/pt quad gig nic
dual onboard gig nics
single esxi host with a dedicated vDS switch for ISCSI traffic, two seperate vmKernels on different subnets, and in freenas a single portal with two IPs again seperate matching subnets.
I am getting horrendous write performance on larger files and I cant figure out why.
stg01 has a dual gig LACP bonded NICs on VLAN 20 192.168.20.12 for NFS
also has two gig NICs on VLAN 40 (192.168.40.98 & 41.99) specific for iscsi traffic
All VMs are on VLAN 20 192.168.20.x, esxi is configured to use 40.100, 41.100 for vmkernel iscsi traffic..
local stg01 write to /mnt/vmfs 10gb file ~ 846 Mbit/s (3x WD Raptor 3000HLHS sata2 Raidz1)
local stg01 write to /mnt/vault 10gb file ~1663 Mbit/s (5x WD Red 2TBs sata3 Raid Z1)
nfs write from an ubuntu VM on VLAN 20 to /exports/downloads (/mnt/vault/downloads) 10gb file ~ 568 Mbit/s
iscsi write inside vm to /tmp (vmdk mounted via iscsi in esxi with two paths) 10gb file ~ 56 Mbit/s
After configuring iscsi to be on seperate subnets and its own VLAN 40.. Slight
iscsi write inside vm to /tmp (vmdk mounted via iscsi) 50mb file ~ 424 Mbit/s
iscsi write inside vm to /tmp (vmdk mounted via iscsi) 500mb file ~ 76.8 Mbit/s
I don' think i'm CPU bound but that very well may be it, at this point i'm about ready to start throwing parts at it and hoping something sticks lol.. I have a 30GB SSD coming for ZIL as a few people have recommended such a thing for NFS writes and ISCSI writes.. I realize I should ideally have at least 64GB of mem but this board caps at 32GB.. We're talking about a 6TB pool here, nothing too major.
edit: here are a bunch of images of my configs, and the esxi 5.5 reporting of performance for that iscsi storage device..
http://i.imgur.com/neqQkkT.png
http://i.imgur.com/TjkCaLm.png
http://i.imgur.com/uAf0eDu.png
http://i.imgur.com/xLtpZbU.png
http://i.imgur.com/zI3LvZB.png
http://i.imgur.com/LUS7gWA.png
http://i.imgur.com/Hj15YtD.png
freenas 9.2.1.6-RC
Intel G2030
32GB DDR3-1600 ECC Kingston
Super Micro X9SCM-F
Intel Pro 1000/pt quad gig nic
dual onboard gig nics
single esxi host with a dedicated vDS switch for ISCSI traffic, two seperate vmKernels on different subnets, and in freenas a single portal with two IPs again seperate matching subnets.
I am getting horrendous write performance on larger files and I cant figure out why.
stg01 has a dual gig LACP bonded NICs on VLAN 20 192.168.20.12 for NFS
also has two gig NICs on VLAN 40 (192.168.40.98 & 41.99) specific for iscsi traffic
All VMs are on VLAN 20 192.168.20.x, esxi is configured to use 40.100, 41.100 for vmkernel iscsi traffic..
local stg01 write to /mnt/vmfs 10gb file ~ 846 Mbit/s (3x WD Raptor 3000HLHS sata2 Raidz1)
local stg01 write to /mnt/vault 10gb file ~1663 Mbit/s (5x WD Red 2TBs sata3 Raid Z1)
nfs write from an ubuntu VM on VLAN 20 to /exports/downloads (/mnt/vault/downloads) 10gb file ~ 568 Mbit/s
iscsi write inside vm to /tmp (vmdk mounted via iscsi in esxi with two paths) 10gb file ~ 56 Mbit/s
After configuring iscsi to be on seperate subnets and its own VLAN 40.. Slight
iscsi write inside vm to /tmp (vmdk mounted via iscsi) 50mb file ~ 424 Mbit/s
iscsi write inside vm to /tmp (vmdk mounted via iscsi) 500mb file ~ 76.8 Mbit/s
I don' think i'm CPU bound but that very well may be it, at this point i'm about ready to start throwing parts at it and hoping something sticks lol.. I have a 30GB SSD coming for ZIL as a few people have recommended such a thing for NFS writes and ISCSI writes.. I realize I should ideally have at least 64GB of mem but this board caps at 32GB.. We're talking about a 6TB pool here, nothing too major.
edit: here are a bunch of images of my configs, and the esxi 5.5 reporting of performance for that iscsi storage device..
http://i.imgur.com/neqQkkT.png
http://i.imgur.com/TjkCaLm.png
http://i.imgur.com/uAf0eDu.png
http://i.imgur.com/xLtpZbU.png
http://i.imgur.com/zI3LvZB.png
http://i.imgur.com/LUS7gWA.png
http://i.imgur.com/Hj15YtD.png
Last edited: