NIC negotiating at 10/100

[Middge]

I should start off by saying that this month is my first month using FreeNAS. That being said, I am a systems administrator with a dozen years experience with the entire Microsoft spectrum, VMWare and some storage experience.

Now, I have a freenas system (FreeNAS-9.10-STABLE) running 12x4TB drives in z2. This system has 3x1GB ethernet interfaces (2 PCIE cards that are working perfectly, and the problematic integrated interface x99 chipset on an Asus x99-A). Everything is working fantastic with the exception of this one damned interface.

I know the issue is not the cable. For starters, I've tried multiple brand new cat6 cables right out of the package and several I crimped myself. Additionally I have tried using the cables that are working swimmingly at a full gig for my other 2 interfaces. This is the same reason why I know the issue is not my managed 3com switch.

So now I ask you, who can assist me troubleshooting this issue? Tell me what logs you need and I will provide. Any time you can give me is incredibly welcome! Thanks.

 

[Spearfoot]

Just wondering: why did you select the ASUS x99-a board? It's more of a gamer board; it doesn't support ECC RAM; and it has integrated graphics and audio that aren't needed in a storage server.

But I'm really intrigued by your network setup. I'm not a network engineer, but it looks as though your server is querying a DHCP server (at 128.1.1.227) and the re0 and re1 interfaces are being dynamically assigned addresses (at 128.1.1.62 and 128.1.1.82 in your log). So you have two IPs in the same subnet, which can be a problem.

And aren't these all publicly routable internet addresses? Most (all?) FreeNAS users don't attach their server directly to the internet, but instead use a private A, B, or C network (10.x.x.x, 172.16.x.x-172.31.x.x, 192.168.x.x) to share files on their LAN.

Would you please tell us about your network setup? Details about your router/switch and how you've configured LACP on it, if relevant, how you're attempting to use the 3 network ports on this server, that sort of thing?

Looking over your message log, it appears that you're using a LAGG group. It's been a while since I set one of these up, but I recall that you need to delete any configured network interfaces before configuring them. More details here in the documentation:

https://doc.freenas.org/9.10/freenas_network.html#link-aggregations

 

[Middge]

The Asus board is something I had laying around. I knew going in I might have problems with it, but I've overcome most of them. Considering I am only serving media on this particular NAS, I'm not overly concerned about the DDR RAM corrupting some data every now and then. Also, the X99-A does not have integrated graphics. I actually wish it did, so I wouldn't have to waste a PCIE slot.

My addressing schema is weird, I know. A loooooooong time ago before I knew much about subnetting, I wanted to pick and addressing schema that was entirely unique so it would be easier for me to remember IP addresses. This has since kind of stuck in my household. I never encountered a good enough reason to change things to a more traditional range. Also, needless to say, my local network is NOT directly routed to the internet. Despite what it may look like, it still operates 100% like any other LAN.

As of right this second my network setup is the following;

FreeNAS host with 3 NICS. Integrated NIC set to dhcp and doing nothing. No interface assigned in FreeNAS at all. 2 PCIE GB nics are aggregated with LACP (before I did that I made sure to delete any interfaces in FreeNAS assigned to those NICS) . Static address assigned to LAGG (128.1.1.62). Managed 3c0m switch configured for LACP on the two relevant ports, and seems to have created a link properly. See screenshot. Ideally, I would love to add my third integrated NIC to the aggregation, but as it's only negotiating at 10/100 it would actually hurt rather than help.

Also, even though the link aggregation *appears* to be working, it still acts funny. I can't pin it down or define what is odd, but it is. It is slower than what I thought a paired set of GB nics would be first of all. I will admit I am at best a novice in regards to enterprise networking, but I would have assumed that I would be able to achieve greater than 1.2gbps (peak) with 3 simultaneous data transfers from 3 clients. Also, the network connection will drop completely randomly and the console will complain of flapping. Odd.

 

[Middge]

BTW, here is my latest and greatest message log after a fresh reboot. Maybe this will give you some more insight?

 

[fn369]

@Middge -

You might find this interesting

LACP - Friend or Foe

 

[Middge]

Hmm that was an interesting read. This thread implies there isn't really a reliable way to upgrade the network trunk on FreeNAS without implementing 10g. :-(

 

[Spearfoot]

Also, even though the link aggregation *appears* to be working, it still acts funny. I can't pin it down or define what is odd, but it is. It is slower than what I thought a paired set of GB nics would be first of all.

If by 'acting funny' you mean that your ethernet speeds haven't doubled, that's because LACP/LAGG seldom (if ever) delivers a doubling of speeds, contrary to what we would all love to see! Details at thread posted by @fn369.

Are you able to browse to the FreeNAS server's GUI interface? If so, are you able to connect to shares created on your FreeNAS server?

You are free, of course, to use whatever networking scheme you like. But my advice is to ditch it and switch to one of the well-known private network schemes, e.g., 192.168.x.x. They were set aside specifically for this purpose and continuing with the approach you're using now will probably cause problems for you in the future, if not now.

 

[fn369]

Hmm that was an interesting read. This thread implies there isn't really a reliable way to upgrade the network trunk on FreeNAS without implementing 10g. :-(

That's the conclusion I've come as well. Also my first month here, so not of much help I'm afraid, other than remembering that thread from my research to date.

It doesn't have to cost the earth, however, so it's not all bad! Try this, also by @jgreco :

https://forums.freenas.org/index.php?threads/10-gig-networking-primer.25749/

 

[Middge]

If by 'acting funny' you mean that your ethernet speeds haven't doubled, that's because LACP/LAGG seldom (if ever) delivers a doubling of speeds, contrary to what we would all love to see! Details at thread posted by @fn369.

Are you able to browse to the FreeNAS server's GUI interface? If so, are you able to connect to shares created on your FreeNAS server?

You are free, of course, to use whatever networking scheme you like. But my advice is to ditch it and switch to one of the well-known private network schemes, e.g., 192.168.x.x. They were set aside specifically for this purpose and continuing with the approach you're using now will probably cause problems for you in the future, if not now.

I'm positive that any issues I am having have nothing to do with the IP schema.I am perfectly able to browse the GUI and see the shares and transfer data at a full 1GB under normal conditions.