Intel X520-DA2 only used for upload of files - Truenas uses 1Gbit for the rest

[Johev]

New Issue, please read the 2nd post.

Dear Truenas Community,

I am a long-time supporter of FreeNAS and now Truenas Scale, and I have been extremely happy with it.
After seeing all the YouTube videos on 10Gbit networking, I decided to use the PCIE port on my X10SL7-F (a popular motherboard many years back in this forum), however, I am extremely frustrated with not being able to make the card work properly.

The good news:
The card is detected under network interfaces. I can see both my 1gbit Eth cards which are on the motherboard and the 2 SFP+ cards. I am running Truenas Scale 23.10.1.1.

When I try to enable DHCP or create a static IPv4 address for the SFP+ card, I get the following errors which repeat each other every second. I am only able to make it stop, once I revert to the old configuration.

These are the errors / see print screen attached:

• "NIC Link is up 10Gbps, Flow control: RX/TX"
• "Received ECC Err, Initiating reset"
• "Reset adapter"
• "detected SFP+: 3"

Even without an IPv4, it still looks like it is up in the overview of TrueNAS, however, the only traffic that it is using are some bits or at maximum Kbits.

I am using the following SFP+ Direct Attach Copper (DAC) cable (ipolex Câble SFP+ 1m, SFP+ 10Gb Direct Attach Copper (DAC) Twinax Cable pour Cisco SFP-H10GB-CU1M, Ubiquiti, Netgear, D-Link, Supermicro, Mikrotik, Compatible avec Freebox Delta, Passif ), to connect it to my Netgear M4300-52-POE switch, with 4 10Gbit ports (2 RJ45 and 2 SFP+). The switch shows that the 10Gbit link is up (lights the 10Gbit led).

Can anyone suggest how I could get it to work? I also bought an X540-DA2 card, however it has the same issues.

Many thanks in advance for all your support.

Best regards to all.

J

Server specs:
Supermicro X10SL7-F
Xeon E3-1265L v3
32GB ECC RAM
RAIDZ2 - 8 - 4TB Ironwolf 5900RPM drives
2 SSDs for Dockers.

 

[Johev]

Originally I had the issue above, but now I have a new one. I was finally able to make the Network card work, but now it only is used when I save files to the NAS. When I try to retrieve them, it only uses the 1Gbit connection. I tried to search for the solution and checked the networking part of the web interface, but with no results.

When I unplug the 1Gbit connection from the switch, both IP (the 1Gbit and the 10Gbit one) addresses stop working.

Does anyone know how I can solve the situation? Many thanks in advance for your assistance.

 

[Redcoat]

You have both NICs in your TrueNAS box on the same (172) subnet. At the very least you need to assign one of them to another subnet - the rest of the fix depends on your network setup.

 

[Etorix]

You have multiple interfactes on the same network. Don't do that! Plug only one port.

And do not trust YouTube videos.

 

[Johev]

Thank you both for your input. I have tried to change the IP so that the subnet is not the same, however, I get the following error.
I don't know how to change the applications to use the other interface or IP address. If I disconnect the cable from the 192.168.172.57, I lose access to the server. No ping, no web guy, until I reinstate the 1Gbit link.

 

[Redcoat]

I don't know how to change the applications to use the other interface or IP address.

Regret I can't help you with any detail on Scale, Core only here.

 

[Johev]

@Redcoat, thank you anyway. I still have a Freenas FreeBSD working next to this one, rock stable :). Take care and thank you anyway.

 

[chuck32]

Did you check Apps -> Settings -> Advanced Settings and then node IP?

 

[Johev]

Did you check Apps -> Settings -> Advanced Settings and then node IP?

Thanks for the pointer. I have checked and this is what I get:

I have even tried to change it to this (see other print screen), however, it does not allow me to change the sub-net of the first network card.

 

[chuck32]

Thanks for the pointer. I have checked and this is what I get

I'm not too experienced with apps but I use scale. Under node IP, where it says 0.0.0.0, can you change it to the IP you want to keep in the subnet? Maybe you need to shut down your apps before making changes to the networking.

 

[Johev]

@chuck32

Good idea, I have stopped all my applications (I only have 4) and tried to change the settings. Still not possible. Here is a print screen.

 

[Johev]

@chuck32 just wanted to let you know that deleting all applications and rebooting Truenas finally worked. I was able to select the 10 Gbit connection. Thanks for all your help :)

 

[chuck32]

Glad it worked out for you!