SOLVED Severe limitation of network throughput in jail

[-fun-]

I have a downstream of effectively 100Mb/s (Megabits per seconds) from the internet which I can actually observe during downloads directly on my FreeNAS-Box. In a jail as well as a VM I only get approximately 10 Mb/s of this value.

I compared download speeds as displayed by wget when downloading a large ISO file from a sufficiently fast internet server.

• On my FreeNAS-Box I nearly hit 100Mb/s repeatedly.
• In the jail (setup still from 9.3) I get a download speed of not greater than 10 Mb/s most of the time, often significantly lower.
• In a VM run by bhyve (setup a few days ago on the current 9.10 FreeNAS version, running arch linux) I also only get approximately 10 Mb/s, often much lower.

CPU load is not noticeably increased during downloads.

This may or may not be the most reliable way to assess download speeds but this is not about percentages. What I observe is a difference by one order of magnitude. Unfortunately this impacts HD IPTV channels which suffer a lot and show dropouts. HD channels should require no more than about 6 to 8 Mb/s with my provider, but the download speed drops to lower than this value regularly. Image quality is severely degraded.

• Is a limitation of network throughput in this magnitude something regular or is this due to a configuration problem?
• Is anyone here able to answer this also for the VM? (I did not find a specific sub forum.)
• Is this question a FreeNAS question after all (in case not, where should I ask)?

Thank you

-fun-

 

[joeschmuck]

That is very interesting, I have no idea why this would happen with a jail.

Could you please describe the entire test case so someone else would be able to reproduce it? Also, I would suggest you submit a bug report, I can't imagine that this would be normal.

EDIT: And please state which version of FreeNAS you are running even though I know your signature line states 9.10.2-U2, I just want to make sure.

 

[joeschmuck]

I just did a quick test and I am using FreeNAS 9.10.2-U2 and a FreeBSD jail.

In the FreeNAS shell I ran a two different speed tests via CLI and both results were about 50Mbits per second (my max download speed).

Inside the FreeBSD jail I ran the same test and recieved the same result.

Here are the tests I ran but I could not run the second one in the jail because I did not install wget.

Code:

curl -o /dev/null http://speedtest.sea01.softlayer.com/downloads/test100.zip

Code:

wget -O /dev/null http://speedtest.sea01.softlayer.com/downloads/test100.zip

 

[IceBoosteR]

Hello -fun-,
could you reproduce that first in you LAN, so moving some files around to the jail to test if you also hit the limitation there?

This behaviour is not normal - and as joeschmuck already replied - is maybe only to your jails.
Did you have upgraded you jail to the latest version, including all software?
If not, maybe you will get your solution there.

IceBoosteR

 

[Jurgen Segaert]

Do you have VIMAGE checked?

https://forums.freenas.org/index.ph...-loss-on-one-of-my-5-jails.53552/#post-369856

 

[-fun-]

Thank you all for the replies! I looked up the VIMAGE flag this morning. This was checked. I unchecked and rebooted. No download at all.

I then checked VIMAGE again, rebooted, tested again. Full network speed, awesome! Now that's strange but anyway, it works. (I did not test this morning before the change, this is not necessarily related.)

@IceBoosteR: I cross checked with a download from WAN because there is actually no host in my LAN which would provide this speed (no other wired hosts with reliable network speed, all WLAN or raspberries).

@joeschmuck: Thank you for testing. I have indeed FreeNAS-9.10.2-U2 (e1497f2) running.

These are my latest results (traffic graph from my pfSense firewall, note the scale). The leftmost and rightmost peaks reflect the downloads with about 100 Mb/s download speed while the one in the middle reflects the download from a VM:

I did test this repeatedly before posting yesterday so there definitely was a problem. Now I can not even guess the reasons. I will observe this in the next time and if it reoccurs I will post again.

One last question: Can anyone here advise on where I could get help regarding the iohyve / bhyve stuff? I understand this is experimental but in my opinion this is a great feature ...

 

[-fun-]

After several times of additional testing now I have around 90 to 100 Mb/s from the VM too. I have no clue why this works now ...

 

[IceBoosteR]

Hi -fun-,

good to hear that everythign is fine now. Maybe with checking, reboot, and unchecking the VIMAGE box you solved the problem. Maybe it was the internet on this day.
Nevertheless you can find some subthreads in this forum about iohyve. This thing is awesome but more experimental. When you need to go with more VMs and stuff like that I would recommend ESXi. OR you wait some long time when the next updates for 9.10 will come with docker support, as Corral was cancelled.

IceBoosteR

 

[-fun-]

I'm looking forward to docker indeed.

Is ESXi an option on FreeNAS? I wasn't aware of this.

 

[IceBoosteR]

I'm looking forward to docker indeed.

Is ESXi an option on FreeNAS? I wasn't aware of this.

You CAN run FreeNAS on ESXi, but there are some things you should be aware about. I belive in this forum is a longer post for do's and don'ts for ESXi.
I would also go for ESXi if I had the option, but my system is up and running so I will not change that.

So from my point of view, the most critical aspect of ESXi and FreeNAS is the point, that ZFS should have direct access to the disks. So don't make da datastore on ESXi and pass those things to FreeNAS-VM. You will have a lot of problems. I think its better when you have a dedicated PCIe Card and pass this thru to the FreeNAS VM. On the PCIe Card you will put your drives, so only FreeNAS see them, and not the Hypervisor.
But I never have done this - it's in german "Gefährliches Halbwissen" :D

IceBoosteR

Edit: Found this for ESXi:
https://forums.freenas.org/index.php?threads/my-dream-system-i-think.41072/
https://b3n.org/freenas-9-3-on-vmware-esxi-6-0-guide/
https://forums.freenas.org/index.php?threads/esxi-server.48740/

 

[-fun-]

Ah ok. I _was_ aware of ESXi as a basis for FreeNAS and the recommendations against doing this in production installations. Therefore before doings this sort of thing I would rather get a separate machine.