Problems with transmission/jail-Freenas 9.2 on ESXi 5.5

[Kostas]

Hi all,

I am having a very dificult (to me) problem which I cannot resolve. I have tried several things with no luck...

I am installing Freenas 9.2 x64 on e virtual machine on ESXi 5.5.

Freenas installs ok. Installation of transmission pbi goes through ok and everything is created ok. Jail named transmission_1 is also created with IP 192.168.1.2

Freenas gets 192.168.1.107 from my DHCP

My problem is that I cannot connect on transmission. I cannot even ping 192.168.1.2

When manually creating a "pluginjail" with a specific IP (192.168.1.3) I cannot ping it either

When manually creating a "pluginjail.x86" I can ping it ok.

Default gateway (192.168.1.1) has been set correctly in network configuration of freenas as a fedault ipv4 gateway.

Any idea what is wrong? If I install freenas directly on my system it works ok. The problem occurs when installing over ESXi 5.5



I have an HP microserver H54L (AMD 2.2GHz) with 16GB ram.

Please help!

 

[Kostas]

Apologies for the bump post but I am getting crazy here.

Any ideas? Anyone with the same setup?

 

[cyberjock]

Well, since you said...

Any idea what is wrong? If I install freenas directly on my system it works ok. The problem occurs when installing over ESXi 5.5

That sounds like a problem with virtualization settings and not a FreeNAS problem. That being said, you're not likely to get support here since we do FreeNAS support and not ESXi.

On an unrelated note, FreeNAS shouldn't be virtualized under normal circumstances. See the stickies on the topic for more info.

 

[Kostas]

Thanks for the reply. I was just hoping someone had the same setup and could give a hand of help.

I think it is a pitty to leave the hardware doing just the freenas thing and hoping to be able to run a couple more machines.

 

[jgreco]

Fix your virtual switch settings. Guessing: you failed to allow forged transmits, since I believe FreeNAS uses some form of overly complicated bridging in the ix jail stuff.

 

[Kostas]

I really can't thank you enough!!! I slept 5am yesterday because of this

Thank you!!!!!

Additional post with settings to come for future reference

 

[rm-r]

I really can't thank you enough!!! I slept 5am yesterday because of this

if i had $1 for everytime i have done that......

 

[Krazypoloc]

Yeah unchecking VIMAGE in the jail will save you a lot of issues. I'm not sure why they added this complexity without explaining what it does.

 

[Stefano]

what is the procedure to solve this problem?
Sorry for my english:)

 

[jgreco]

See post 5.

 

[Stefano]

I have read it but i don t figure out how to do it...

 

[jgreco]

This is a FreeNAS support forum, not an ESXi support forum. Since there are multiple versions of ESXi, and different ways to do it depending on whether you're using the Web Client, the legacy client, or the CLI, I'm not really interested in trying to explain half a dozen different possibilities.

 

[amires]

For the VIMAGE to work properly promicious mode should be enabled on the vSwitch freenas is connected to. By default it is disabled.

 

[jgreco]

If that's the case be aware that you may have some issues if there's a busy network, as the FreeBSD kernel will end up having to manually filter out all the noise.

 

[cyberjock]

If that's the case be aware that you may have some issues if there's a busy network, as the FreeBSD kernel will end up having to manually filter out all the noise.

QFT. While its great to post a "fix", if you can't provide serious info on what that actually does in the background and how it could cause serious problems(in this case performance problems) later you might not be doing the OP a favor. ;)

 

[c32767a]

The physical interface underneath FreeBSD's bridge interface needs to be in promiscuous mode for the bridge to properly function:

root@nas3] ~# ifconfig ix1
ix1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
options=400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO>
ether 00:1b:21:
inet 192.168.1.52 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
status: active
[root@nas3] ~# ifconfig bridge0
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
ether 02:0b:04
nd6 options=1<PERFORMNUD>
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 16 priority 128 path cost 2000
member: ix1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 20000

[root@nas3] ~#
( This is on physical hardware, no ESX. Note ix1 is flagged PROMSIC)



In VMWare, virtualized physical adapters are subject to ESX security policy, which includes a flag for allowing the adapter to enter promiscuous mode.

http://kb.vmware.com/selfservice/mi...nguage=en_US&cmd=displayKC&externalId=1002934

Changing that setting only affects whether the adaptor is allowed to enter promiscuous mode or not. It does not change operation, unless the OS actually places the card into PROMISC.

Since FreeBSD runs the physical interface in promiscuous mode when using if_Bridge, this setting needs to be enabled in ESX for things to work properly.

 

[depasseg]

It would be EXTREMELY helpful if this little gem (enable promiscuous mode on the vswitch) was included in FreeNAS guide (beginning of section 2.2, for instance). I am testing this on ESX prior to moving to a production box, and it took hours to figure out why my jails were inaccessible.

 

[jgreco]

Setting up your networking infrastructure correctly is generally beyond the scope of FreeNAS documentation. Virtually all modern managed switches have a variety of features to prevent bad actors plugged into a network from capturing traffic or doing various types of spoofing. There is nothing special about ESXi in this regard other than it happens to be an all-software implementation of very basic switching - and ESXi tends to default some of its security paranoia to "on", whereas in a switch, it isn't.

In both cases, I'd say it is reasonable to expect that an admin has more familiarity with the local environment than we do; I do not have any plans, for example, to go around documenting how to correctly configure a port for FreeNAS on Cisco, Juniper, HP, Dell, SMC, and VMware kit. It will vary with specifics, anyways ... for example, here, we don't use jails and the default configuration for an ESXi vlan allows forged transmits by default, but on a customer- or public-Internet-facing network segment that would be different.

It may sound harsh but really we can't document every possibility without creating a TL;DR scenario. At some point you have to take responsibility for the local environment. I've had plenty of "Boy, am I the biggest idiot or what" moments in many years in this business and the thing I try to take away from every one of them is that you really need to learn about these products in detail and how they might interact, and then generalize that going forward.

 

[depasseg]

I wasn't asking for a howto setup networking infrastructure. There are 5 pages of step by step on how to install in ESX. All I'm asking for is one line that says - The use of certain plugins/Jails require the use of Promiscuous mode on the vSwitch, which is usually prohibited by default.

Or perhaps put it in the plugin section. This same issue would apply to any switch that was set up properly (i.e. limit promiscuous mode).

It's not that huge of a request.

Agree about spending time learning, but to take hours learning how FreeBSD jails work, how the plugins are configured and what is required just to do an evaluation of FreeNAS is ludicrous. The reason the ESX instructions are there in the first place is so that people can do a quick and easy install to test it out. It clearly says it isn't for production use. So why make the trial a "let's learn the inner workings of FreeNAS on top of FreeBSD"?

 

[jgreco]

So why make the trial a "let's learn the inner workings of FreeNAS on top of FreeBSD"?

Again, because it has nothing to do with FreeNAS and everything to do with configuring environment external to FreeNAS.

We don't tell you how to turn on your computer. We don't tell you not to run gigE over category 3 patch. It is difficult enough to identify all the possible issues that are in-FreeNAS-scope and help people with those. As someone who's written a nontrivial amount of help material for this project, and as the guy who actually hit the OP's issue up above in this thread, I just don't think there's a good solution. If you get sufficiently specific to document all these externalities, then you get "TL;DR". If you get vague and just talk about making sure the switching environment is set up "correctly" (bearing in mind that this term defines differently depending on the environment), then you get people saying "but I didn't know ESXi did that." There's no winning. That's part of why the forum exists and part of why guys like Cyberjock and I hang around here.

But I guess the bigger question here is why someone with a competent virtualization platform would be running jails on a virtualized FreeNAS. This is not a common use model and is adding unnecessary complexity for no obvious return.