AoE (ATA over Ethernet) kernel module

[Andreas Neiser]

Hi all,

I worked on getting the FreeBSD ports net/aoe to compile a kernel module for FreeNAS. See this bug report,
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194663
and the corresponding patches there in order to get a kernel module which works with an VIMAGE enabled kernel.

I then loaded the /boot/modules/aoe.ko into FreeNAS using a tunable, and set the sysctl net.aoe.iflist="em1" (assuming em1 has vblade's attached) and finally get devices ready to use in ZFS pools. To make it fully work, I needed to patch three FreeNAS files:

• "/conf/base/etc/ix.rc.d/ix-zfs" Add the following lines at the top of the import_zpools() function:
  Code:

  /sbin/sysctl -in net.aoe.iflist | /usr/bin/xargs -n 1 -J % /sbin/ifconfig % mtu 9000 up
  sleep 5
  /sbin/sysctl -i `/sbin/sysctl -ie net.aoe.iflist` 2>/dev/null >/dev/null
  sleep 5
  echo "========> sysctl net.aoe"
  /sbin/sysctl net.aoe
  sleep 1

• "/conf/base/etc/rc.shutdown" Comment out the complementary lines where the network interfaces are shutdown first. This, by the way, also makes ssh connections shutdown properly (and not, after it's up again, just report a broken pipe). Is there any reason why interfaces are brought down so early in FreeNAS? FreeBSD doesn't do it.
• Tell dhclient to ignore the AoE interface. Patch the file /conf/base/etc/network.subr in the routines dhcpif() and syncdhcpif(). Add the following check in both routines:
  Code:

  if echo `sysctl -in net.aoe.iflist` | grep -q $1; then
  echo "Ignoring interface $1 for DHCP, it's an AOE device!"
  return 1
  fi

The changes simply ensure that the network interfaces don't vanish although AoE needs them to communicate with the devices.

If you're interested in more details or have similar experiences, please share! I can also extend the description here if you want to try it yourself and can't get it working from this short description.

There are still many things do to (get the console "cec" running, test the performance/stability), but just to ask: Do you think there's any chance to get this aoe module into the FreeNAS release? Then I would write-up a more detailed feature request. And maybe you've also a better idea to enable network interfaces...

Best regards,
Andreas

Edit: Improved AoE discovery (also uploaded new patches to net/aoe bugreport), ignore dhclient...

 

[2607:fca8:2636:8001::4]

There was a request for this more than a year ago here: https://bugs.freenas.org/issues/2082
More details are there. You could probably just merge your changes to FreeNAS on github and wait for someone to review them. Nice work.

 

[Andreas Neiser]

Hi IPv6 address :)
Thanks for the link to the request. I was actually aware of that, but I thought about discussing here my general approach. I know that it's rather hacky, so I assumed that the FreeNAS might have some hints how to implement it into FreeNAS better and more future-proof. Or is this request the better place for discussing this? For now, I'm happy since I can use my old Coraid device (most work was patching the kernel module though).