Cannot Connect to Internet

[poraschaudhary]

First of all thanks for this wonderful operating system, I'm really glad that I made myself a server based on FreeNAS. I could only do it because of this fantastic community :)
As a photographer it adds a lot to the efficiency when 4 people can access the files at the same time and do their job simultaneously.

I built a FreeNAS server recently with this configuration:
Motherboard:__-Supermicro X9SCM-F-O
Processor:_____Intel Xeon E3-1230 v2
RAM:________Kingston 32GB Kit (4 x 8GB) 1600MHz DDR3 ECC
NIC:_________Intel Quad Port Ethernet Server Adapter (I350T4)
____________(4 Ethernet cables plugged in as lagg0, going into Netgear Switch)
RAID Card:___--IBM ServeRaid M1015 corssflashed as LSI9211-8iIT
Hard Drives:--_--6 x 4TB WD Red (WD40EFRX) in RAIDZ2
Smart Switch:__-Netgear GS108Tv2 (with link aggregation which doesn’t work)

The problem is that I'm not able to connect to internet.

Here are the results of ifconfig -a

Code:

em0: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                             
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>                                 
        ether 00:25:90:d7:e0:67                                                                                                   
        nd6 options=9<PERFORMNUD,IFDISABLED>                                                                                       
        media: Ethernet autoselect                                                                                                 
        status: no carrier                                                                                                         
em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                             
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>                                 
        ether 00:25:90:d7:e0:66                                                                                                   
        nd6 options=9<PERFORMNUD,IFDISABLED>                                                                                       
        media: Ethernet autoselect (1000baseT <full-duplex>)                                                                       
        status: active                                                                                                             
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536                                                                         
        nd6 options=9<PERFORMNUD,IFDISABLED>                                                                                       
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384                                                                 
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>                                                                     
        inet6 ::1 prefixlen 128                                                                                                   
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa                                                                                 
        inet 127.0.0.1 netmask 0xff000000                                                                                         
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>                                                                                 
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500                                               
        options=400b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO>                                                   
        ether a0:36:9f:35:43:f4                                                                                                   
        inet 192.168.1.102 netmask 0xffffff00 broadcast 192.168.1.255                                                             
        nd6 options=9<PERFORMNUD,IFDISABLED>                                                                                       
        media: Ethernet autoselect                                                                                                 
        status: active                                                                                                             
        laggproto lacp lagghash l2,l3,l4                                                                                           
        laggport: igb3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>                                                                   
        laggport: igb2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>                                                                   
        laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>                                                                   
        laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>                                                                   
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                     
        ether 02:45:29:4d:80:00                                                                                                   
        nd6 options=1<PERFORMNUD>                                                                                                 
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15                                                               
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200                                                                   
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0                                                                   
        member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>                                                             
                ifmaxaddr 0 port 13 priority 128 path cost 2000                                                                   
        member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>                                                               
                ifmaxaddr 0 port 11 priority 128 path cost 20000                                                                   
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500                                             
        options=8<VLAN_MTU>                                                                                                       
        ether 02:86:a1:00:0d:0a                                                                                                   
        nd6 options=1<PERFORMNUD>                                                                                                 
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)                                                                       
        status: active                                                                                                              

and here's what netstat -nr gave me:

Code:

[root@freenas ~][root@freenas ~]# netstat -nr                                                   
Routing tables                                                                  
                                                                                
Internet:                                                                       
Destination        Gateway            Flags    Refs      Use  Netif Expire      
default            192.168.1.1        UGS         0      800  lagg0             
127.0.0.1          link#10            UH          0    81538    lo0             
192.168.1.0/24     link#11            U           0 35451012  lagg0             
192.168.1.102      link#11            UHS         0    90579    lo0             
                                                                                
Internet6:                                                                      
Destination                       Gateway                       Flags      Netif
 Expire                                                                         
::/96                             ::1                           UGRS        lo0 
::1                               link#10                       UH          lo0 
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0 
fe80::/10                         ::1                           UGRS        lo0 
fe80::%lo0/64                     link#10                       U           lo0 
fe80::1%lo0                       link#10                       UHS         lo0 
ff01::%lo0/32                     ::1                           U           lo0 
ff02::/16                         ::1                           UGRS        lo0 
ff02::%lo0/32                     ::1                           U           lo0 

and here's what the network summary look like

Code:

Name IPv4 Address
lagg0 192.168.1.102/24
Nameserver
192.168.1.102
Default route
192.168.1.1

Thanks!
I'll highly appreciate the inputs..

 

[cyberjock]

Why is your nameserver the same IP as your FreeNAS' network connection? That seems to be major fubar IMO.

Also do realize that just because you have 4 client machines and 4 connections on LAG that 1 or more clients may share the same physical port and therefore still be "sharing" a 1Gb pipe with other clients. If you are truely looking for 1Gb for each client you are going to need to look at going with a 10Gb LAN connection from your FreeNAS box to your network switch OR setup each physical network port on a different subnet and have each client machine access the server on its own dedicated subnet.

 

[poraschaudhary]

Why is your nameserver the same IP as your FreeNAS' network connection? That seems to be major fubar IMO.

Wow!! Thanks for catching that.. What we newbies would've done without this forum..
I changed the nameserver to 8.8.8.8 and it connected to internet instantly.

Also do realize that just because you have 4 client machines and 4 connections on LAG that 1 or more clients may share the same physical port and therefore still be "sharing" a 1Gb pipe with other clients. If you are truely looking for 1Gb for each client you are going to need to look at going with a 10Gb LAN connection from your FreeNAS box to your network switch OR setup each physical network port on a different subnet and have each client machine access the server on its own dedicated subnet.

Now only if you could shed some light on this topic, I'll be very grateful to you.. :)
I've been reading a lot on this forum lately but never came across this point about link aggregation.

Can you please give me a little idea of how can it be done..
Does setting up each physical network port on a different subnet work with ZFS?
Do I need to add any new interfaces?
Do I need to setup Static Routes?

Thanks!!

 

[c32767a]

Most people assume Link aggregation is the same thing as using 4 water hoses to fill your pool 4x faster than 1 hose. It's not the same thing.

The reality is that nearly all common link aggregation methods will distribute "flows" across the aggregated links, not packets across.
In this case a flow can be defined as all traffic between 2 MAC addresses. (though the exact criteria for determining a flow can be changed, depending on the particular hardware in use.)
This means that any particular "flow" or communication between hosts will always traverse the same link in the link aggregate. This was done to avoid some nasty packet ordering problems that result when you randomly distribute traffic on interfaces. It might be less of an issue these days than it was in 1990, but this behavior is still the standard.

So, the maximum throughput any single flow can experience is the bandwidth of a single link. As you add more simultaneous flows, they will be distributed across the other links based on the balancing algorithms on the switch and server. Typically, these algorithms use the source and destination IP and/or MAC address, which means all traffic between 2 hosts will end up on the same link in a link aggregate.
Also, link aggregation algorithms are coarse and fixed, depending on the method used, you may see 3 flows on one link and no flows on another link. The switch does not take into account the actual traffic level on the links when it makes the hashing calculation.

So, if you have 1 host and 1 server, Link aggregation will not help you in any way.
If you have 20 hosts and 1 server, Link aggregation will distribute traffic among several links to your server in a coarse way and you will benefit from it to some degree.

 

[poraschaudhary]

Most people assume Link aggregation is the same thing as using 4 water hoses to fill your pool 4x faster than 1 hose. It's not the same thing.

Thanks for the reply!!
I know that it doesn't work like this but Can I fill 4 buckets with 4 hoses?
All I want is 80MB/s on all the computers at the same time.
Is this Possible?

Here's what my network looks like:

Thanks!!

 

[cyberjock]

Theoretically you can fill 4 buckets with 4 hoses. The problem is that there is no guarantee you will end up with that configuration. Your networking devices will determine the flowpath for your data and unless each machine happens to end up on independent channels then at least one desktop will share it's lagg port with another desktop(s). This is why the simplest way to explain LACP/lagg to people is to tell them that unless you have more than 10 workstations connecting to the server you aren't likely to see a benefit. Statistically its more likely you will share a port with another machine than not.

Your situation is perfect for one of those network switches that has a 10Gb uplink port and ten 1Gb ports. Then you *know* your workstations won't compete for bandwidth.

 

[c32767a]

Thanks for the reply!!
I know that it doesn't work like this but Can I fill 4 buckets with 4 hoses?
All I want is 80MB/s on all the computers at the same time.
Is this Possible?

It's possible, but as CJ alluded, it will depend on the hashing algorithm in use on your network switch.

For traffic flowing from the switch to freenas, the switch gets to decide how it places traffic on the aggregated links. The algorithms for that are burned into the switch chipset and are generally pretty dumb. Unless you know the algorithm and do the math ahead of time, the only way to know is to set everything up and see what happens.

For traffic flowing from freenas to the switch, the algorithm is included in the FreeBSD lagg driver, which you can specify in the "protocol type" when you set up the lagg.. Again, though, you have limited options for controlling how traffic is spread across the links.