Help getting 10 gig Peer to Peer working

N

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
I finally bit the bullet and purchased 2 Solarflare SFN6122F cards and 10G SFP+ Passive Twinax DAC 3M cable. I've installed the cards in my 2 Freenas servers. After rebooting I noticed in the network/interface that the link is down on both machines and the sfxge0 and sfxge1 are both present in that menu. The cable is plugged into the bottom port which I think is sfxge1 port. I've assigned an IP for both cards on a different subnet from my main system.
My Main FreeNAS SuperMicro x9srl-f
ipconfig
Code:
root@freenasSuper:~ # ifconfig
sfxge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=ec07ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:0f:53:0d:f5:14
    hwaddr 00:0f:53:0d:f5:14
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-SR <full-duplex,rxpause,txpause> (autoselect)
    status: no carrier
sfxge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: 10GIG
    options=ec07ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:0f:53:0d:f5:15
    hwaddr 00:0f:53:0d:f5:15
    inet 192.168.8.48 netmask 0xffffff00 broadcast 192.168.8.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet 10Gbase-SR <full-duplex,rxpause,txpause> (autoselect)
    status: no carrier
em0: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
    ether 0c:c4:7a:08:68:82
    hwaddr 0c:c4:7a:08:68:82
    nd6 options=1<PERFORMNUD>
    media: Ethernet autoselect
    status: no carrier
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: Nic
    options=2098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
    ether 0c:c4:7a:08:68:83
    hwaddr 0c:c4:7a:08:68:83
    inet 192.168.5.48 netmask 0xffffff00 broadcast 192.168.5.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:a1:2f:de:fa:00
    nd6 options=1<PERFORMNUD>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 24 priority 128 path cost 2000000
    member: vnet0.17 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 23 priority 128 path cost 2000
    member: vnet0.16 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 22 priority 128 path cost 2000
    member: vnet0.15 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 21 priority 128 path cost 2000
    member: vnet0.14 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 20 priority 128 path cost 2000
    member: vnet0.13 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 19 priority 128 path cost 2000
    member: vnet0.12 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 18 priority 128 path cost 2000
    member: vnet0.11 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 17 priority 128 path cost 2000
    member: vnet0.10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 16 priority 128 path cost 2000
    member: vnet0.9 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 15 priority 128 path cost 2000
    member: vnet0.8 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 14 priority 128 path cost 2000
    member: vnet0.7 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 13 priority 128 path cost 2000
    member: vnet0.6 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: vnet0.5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 11 priority 128 path cost 2000
    member: vnet0.4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 10 priority 128 path cost 2000
    member: vnet0.3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 9 priority 128 path cost 2000
    member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 8 priority 128 path cost 2000
    member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 7 priority 128 path cost 2000
    member: em1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 4 priority 128 path cost 20000
vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: caddy as nic: epair0b
    options=8<VLAN_MTU>
    ether 0e:c4:7a:9a:61:30
    hwaddr 02:d4:d0:00:07:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: cloud as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:07:4c:d6
    hwaddr 02:d4:d0:00:08:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: duplicati as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:e2:92:b5
    hwaddr 02:d4:d0:00:09:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: emby as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:79:aa:5f
    hwaddr 02:d4:d0:00:0a:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.5: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: jackett as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:b1:b8:71
    hwaddr 02:d4:d0:00:0b:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: lazylibrarian as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:fc:6c:97
    hwaddr 02:d4:d0:00:0c:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: lidarr as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:bf:cd:2e
    hwaddr 02:d4:d0:00:0d:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.8: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: mono_patch as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:af:8f:a1
    hwaddr 02:d4:d0:00:0e:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.9: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: plexpass as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:e4:20:fc
    hwaddr 02:d4:d0:00:0f:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.10: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: radarr as nic: epair0b
    options=8<VLAN_MTU>
    ether 0e:c4:7a:c1:df:ca
    hwaddr 02:d4:d0:00:10:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.11: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: sabnzbd as nic: epair0b
    options=8<VLAN_MTU>
    ether 0e:c4:7a:ef:4d:38
    hwaddr 02:d4:d0:00:11:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.12: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: sonarr as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:a5:a4:7e
    hwaddr 02:d4:d0:00:12:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.13: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: tautulli as nic: epair0b
    options=8<VLAN_MTU>
    ether 0e:c4:7a:46:b0:16
    hwaddr 02:d4:d0:00:13:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.14: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: transmission as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:03:aa:46
    hwaddr 02:d4:d0:00:14:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.15: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: unifi as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:87:00:b4
    hwaddr 02:d4:d0:00:15:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.16: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: urbackup as nic: epair0b
    options=8<VLAN_MTU>
    ether 0c:c4:7a:4e:7d:7c
    hwaddr 02:d4:d0:00:16:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.17: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: wordpress as nic: epair0b
    options=8<VLAN_MTU>
    ether 0e:c4:7a:ed:53:08
    hwaddr 02:d4:d0:00:17:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: Attached to Collabora
    options=80000<LINKSTATE>
    ether 00:bd:5a:9f:fb:00
    hwaddr 00:bd:5a:9f:fb:00
    nd6 options=1<PERFORMNUD>
    media: Ethernet autoselect
    status: active
    groups: tap
    Opened by PID 11162


My Backup FreeNAS ASROCK E3C226D2I
ipconfig
Code:
sfxge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=ec07ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:0f:53:0e:68:b4
    hwaddr 00:0f:53:0e:68:b4
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-SR <full-duplex,rxpause,txpause> (autoselect)
    status: no carrier
sfxge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: 10GIG
    options=ec07ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:0f:53:0e:68:b5
    hwaddr 00:0f:53:0e:68:b5
    inet 192.168.8.50 netmask 0xffffff00 broadcast 192.168.8.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet 10Gbase-SR <full-duplex,rxpause,txpause> (autoselect)
    status: no carrier
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: FreeNAS
    options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>
    ether d0:50:99:5b:da:e4
    hwaddr d0:50:99:5b:da:e4
    inet 192.168.5.50 netmask 0xffffff00 broadcast 192.168.5.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether d0:50:99:5b:da:e5
    hwaddr d0:50:99:5b:da:e5
    nd6 options=1<PERFORMNUD>
    media: Ethernet autoselect
    status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:77:87:c7:01:00
    nd6 options=1<PERFORMNUD>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 8 priority 128 path cost 2000
    member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 7 priority 128 path cost 2000
    member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 3 priority 128 path cost 20000
vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: duplicati as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:e2:92:b5
    hwaddr 02:3c:d0:00:07:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0.2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: plex as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:14:fa:09
    hwaddr 02:3c:d0:00:08:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair


dmesg | grep -i '10 G' on both machines returns nothing.

I'm not sure how to preceed. My goal is to have my replication go over the 10 GIG network.
 
Elliot Dierksen

Elliot Dierksen

Guru
Joined
Dec 29, 2014
Messages
1,135
Do you have link lights on the cards? I am betting that you do not based on the 'no carrier' output from ifconfig. I suspect you have a hardware problem with one of the cards, or with the DAC cable. :-(
 
N

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
The green link lights are on on both cards. Not sure how to determine if it's bad cards or cable?
 
Elliot Dierksen

Elliot Dierksen

Guru
Joined
Dec 29, 2014
Messages
1,135
Maybe they aren't the ports you think. You could try swapping ports or from the CLI on both systems do ifconfig sfxge0 up and see if you get a different status.
 
Mlovelace

Mlovelace

Guru
Joined
Aug 19, 2014
Messages
1,111
Add 'up' in the options for the interface you believe to be connected
NasKar said:
The green link lights are on on both cards. Not sure how to determine if it's bad cards or cable?
Click to expand...
You need to make the interfaces active by adding 'up' in the options section of the interface configuration.
1601470174908.png
 
N

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
I setup the sfxge0 on 192.168.9.x on both computers (sfxge1 is on 192.168.8.x) and typed ifconfig sfxge0 up and it now says link is up. Thanks so much. I did check the manual for the card online to determine which port was which but must have gotten it wrong.

Additionally I changed the MTU to 9000. Do I still need to put the Options as up in case of a reboot?
 
Elliot Dierksen

Elliot Dierksen

Guru
Joined
Dec 29, 2014
Messages
1,135
NasKar said:
Additionally I changed the MTU to 9000. Do I still need to put the Options as up in case of a reboot?
Click to expand...
I don't think so. It appears that FreeNAS will bring the interface up if you assign an IP address to it. It is certainly easy to test.
 
N

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
Now that the link is up on both servers I want to create an ssh connection to use for replication. After creating an ssh key pair, I tried to create an ssh connection for 192.168.9.50 (backup server) and get an error. Unable to connect to remote system: [Errno 61] Connection refused . My main server is 192.168.9.48.
 
SweetAndLow

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
NasKar said:
Now that the link is up on both servers I want to create an ssh connection to use for replication. After creating an ssh key pair, I tried to create an ssh connection for 192.168.9.50 (backup server) and get an error. Unable to connect to remote system: [Errno 61] Connection refused . My main server is 192.168.9.48.
Click to expand...
probably using the keys wrong. Try just doing an ssh without key's and see if that works.
 
N

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
Destroyed the keys and tried from the ssh menu and get the same error
 
N

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
I ran iperf3 -P 20 -c 192.168.5.50 on the server and iperf3 -s on the backup server.

Code:
[SUM]   0.00-10.00  sec  11.4 GBytes  9.79 Gbits/sec    0             sender
[SUM]   0.00-10.02  sec  11.4 GBytes  9.78 Gbits/sec                  receiver


So I presume that it's working at just about 10 GIG.

Does it mean that my hardware is not an issue?
Why can't I setup an SSH connection?
SweetAndLow said:
Try just doing an ssh without key's and see if that works.
Click to expand...
I don't see an option to setup ssh without keys
 
SweetAndLow

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
NasKar said:
I ran iperf3 -P 20 -c 192.168.5.50 on the server and iperf3 -s on the backup server.

Code:
[SUM]   0.00-10.00  sec  11.4 GBytes  9.79 Gbits/sec    0             sender
[SUM]   0.00-10.02  sec  11.4 GBytes  9.78 Gbits/sec                  receiver


So I presume that it's working at just about 10 GIG.

Does it mean that my hardware is not an issue?
Why can't I setup an SSH connection?

I don't see an option to setup ssh without keys
Click to expand...
Just do ssh from the command line to see if it works.
 
N

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
Can do ssh from command line and snapshot / replication from cli. How do I fix ssh in the GUI.
 
Redcoat

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,925
NasKar said:
How do I fix ssh in the GUI.
Click to expand...
I have the same setup. Link is up fine.
I have a working PULL replication task over the 1Gb network that I am trying, so far unsuccessfully, to modify to run across the 10Gb peer-to-peer.
Any ideas? I assumed that using an SSH keypair based on the 10Gb link IP would do it but I haven't got it yet...
 
Last edited:
Elliot Dierksen

Elliot Dierksen

Guru
Joined
Dec 29, 2014
Messages
1,135
Redcoat said:
I am trying, so far unsuccessfully, to modify to run across the 10Gb peer-to-peer.
Click to expand...
Is it possible the known host entries for the 10G IP network haven't been imported?
 
N

NasKar

Guru
Joined
Jan 8, 2016
Messages
739
Any idea on how I can get an SSH connection working. I'm on FN 11.3U5 (upgrading from U4.1 didn't help). When I got to system/ssh connection/add and create a new connection I get the error "Unable to connect to remote system: [Errno 61] Connection refused " Not sure what to do at this point.

VPNclient-12.jpg
VPNclient-13.jpg
VPNclient-14.jpg
 
Redcoat

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,925
Elliot Dierksen said:
Is it possible the known host entries for the 10G IP network haven't been imported?
Click to expand...
Thank you for that - it certainly was possible in my case and they are now in place.

I do now have a new PUSH replication copying snapshots over the 10G connection, but I haven't yet managed to get a PULL replication to do so. However, the PUSH seems to be running very slowly. I have never attempted to investigate network speeds before and have no idea how to do so, further than often seeing references to iperf (including earlier in this thread...) - I'm just starting to bone up on it now.

Setting up the push replication by the semi-automatic method "just-worked" and gave me no clue to what might be wrong with the pull method. I suspect a key issue and I need to understand the background there better.
 
You must log in or register to reply here.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "Help getting 10 gig Peer to Peer working"

Similar threads

phynias
upgrade to 11.2-U3 and now jails have no network access
Replies
2
Views
1K
phynias
phynias
C
  • Locked
Jail has no NIC after recent update
2
Replies
28
Views
5K
Crispin
C
rodofrn
  • Locked
Very slow transfer speeds.
Replies
17
Views
4K
Elliot Dierksen
Elliot Dierksen
P
  • Locked
slow transfer rates
Replies
13
Views
4K
Psychoteur
P
jspcto
  • Locked
SOLVED Problem assigning Global static IP address (and Jail IPs) in new subnet
Replies
3
Views
2K
jspcto
jspcto
Top