mstang1988
Contributor
- Joined
- Aug 20, 2012
- Messages
- 102
I'm running into issues with performance when utilizing truenas with jails. The bridged adapter lowers the throughput of my network significantly.
Running my config with ESX and passthrough (on a server board). On the datastore I've booted a W10 VM. On that VM I'm getting ~1400MB Read (1M blocks, 16GB total transfer, 4 thread) from disk mark and about 1300 MB/s write. Random 4KQ1 is down to about 40MB/s (which isn't surprising). 4KQ1 with 4thread is closer to 100MB/s which is pretty good for 4K random. I'm pulling/pushing about 1GB/s from a Windows VM on another ESX host to this freenas instance with SMB.
When configuring I first enabled LRO, TSO, and TXSUM offloads. I ran into the freebsd TSO bug in bsd 12.0 so has to disable TSO.
The next thing I've run into (and will be starting a dedicated thread on) is as soon as I enable a jail (and get a bridge) my read speed declines to around 300MB/s. This appears related to the bridge. Disabling the plugin and rebooting remediates the problem.
I tried turning off the hardware offloads, same results. I looked at adding a separate unaddressed NIC on the same network and then assigning that to the jails with promiscuous mode but truenas/BSD didn't like that.
I have a desire to have plugins and performance. Any thoughts on the workaround without setting up a separate NIC and separate subnet for plugins?
Running my config with ESX and passthrough (on a server board). On the datastore I've booted a W10 VM. On that VM I'm getting ~1400MB Read (1M blocks, 16GB total transfer, 4 thread) from disk mark and about 1300 MB/s write. Random 4KQ1 is down to about 40MB/s (which isn't surprising). 4KQ1 with 4thread is closer to 100MB/s which is pretty good for 4K random. I'm pulling/pushing about 1GB/s from a Windows VM on another ESX host to this freenas instance with SMB.
When configuring I first enabled LRO, TSO, and TXSUM offloads. I ran into the freebsd TSO bug in bsd 12.0 so has to disable TSO.
The next thing I've run into (and will be starting a dedicated thread on) is as soon as I enable a jail (and get a bridge) my read speed declines to around 300MB/s. This appears related to the bridge. Disabling the plugin and rebooting remediates the problem.
I tried turning off the hardware offloads, same results. I looked at adding a separate unaddressed NIC on the same network and then assigning that to the jails with promiscuous mode but truenas/BSD didn't like that.
I have a desire to have plugins and performance. Any thoughts on the workaround without setting up a separate NIC and separate subnet for plugins?
