SOLVED 10 Gbe recommendation / feasibility

[chuck32]

Just make sure there are no IPv4 settings on any physical adapter. Then go to the Network tab and create a bridge adapter (call it "br0" for example). Add whatever physical adapters you want to the bridge adapter. Then when you're done, add IPV4 to the bridge adapter.

Now, you can connect one of the physical adapters in the bridge to your router and connect the other physical adapters in the bridge to whatever other devices on the network you want to network at 10GB, and you're done.

That's exactly what I did, but I got no connection from the windows machine to the internet or truenas.

I already have a bridge running for my VMs. I added the 10gig interface to it and made sure the 10gig interface had no IP assigned. In windows I set everything to auto (DHCP etc.).

I think (probably I missed a detail somewhere) every I described 2 posts before, is what you said.

The only question is what network card to use in SCALE for 10GB connections. I would consider getting a quad port card rather than a dual port card.

Right now I don't even the second port. For the foreseeable future there will be 2 devices at most who would need a 10g link to the server. So for me a dual port is enough.

10Gb switches aren't that expensive anymore, but they take up shelf space and are an extra power cable to plug in. It's just a matter of your priorities.

This is the point. If the second machine does not really need a 10g link I think there are better options for me. For the price of a 10g switch I can also get an appliance to run pfsense, something I always wanted to try. So I'd rather hold back on the switch for now.

 

[chuck32]

I would create a bridge interface with the two 10G ports and the 1G port. Plug 1G into switch with local LAN and Internet Uplink, plug Windows PCs into 10G ports. Fixed IP address for NAS, DHCP for Windows, easy peasy.

I don't know why it did not work the first time I tried but I tried again tonight. It's working, thank you!

With NAS as switch:

With my switch as the switch:

In the meantime I daisy chained a 100Mbit switch for devices that do not require SMB access (work laptop, ipmi etc.) so I don't really need the free port on my main switch currently, but it's nice that it's working now. If I don't see any downsides I'll leave it at this settings.

I waited on my new compatible transceivers before testing because I wanted a connection between my PC and NAS that did not get interrupted no matter what testing I did. For some reason I needed to reboot the server in order for the transceiver / network interface to show up. IIrc it was detected right away last time and SFP+ modules should be hot pluggable.

I have only put 10g link towards the bridge though, I can't even saturate that connection currently.

In the future I may add the mikrotik switch and another X520 for the second windows machine.

 

[koli]

but bottom line is apparently the HP Original 10Gb SR SFP+ 455885-001 are both either broken or not compatible, I will contact the reseller and see if I can get them swapped for the finisar ones.

I have the same experience with 455885-001. Two modules came with Intel X520 card I bought on Ebay but I couldn't make the card work with them, on Linux or Windows 10. Luckily I order an Intel coded DAC cable at the same time as a backup and as soon as I used it the card connected to my mikrotik CCR2004+ router immediately on Linux - haven't tried with Truenas yet.

I now ordered a new pair of transceiver for Intel from China, only £7.60 each but reviews from buyers are positive so I am happy to give them a go. I'll try those transceivers on my Windows machine next, if they work I'll order another X520 for my PC. My Truenas Core will then be connected with DAC cable since it sits next to the router.

 

[ChrisRJ]

When the card does not "accept" the transceiver, there is a corresponding entry in the sylog file.

For Linux and FreeBSD at least (no idea about Windows) it is possible to override that check. I don't have it at hand, but Google will certainly help you here.