RancherOS Connectivity Issues

Status
Not open for further replies.

blazerguns

Dabbler
Joined
Mar 15, 2017
Messages
13
Hi All,

I am trying to use the docker subsystem to learn more about docker. I followed the documentation http://doc.freenas.org/11/vms.html#docker-rancher-vm all the way. The only deviation I took is based on https://forums.freenas.org/index.ph...s-11-1-rancheros-docker-rancherui-plex.59963/ to set static IP. By default it looks like dhcp address is assigned which is not convenient.

Just before I ran the command
sudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server

I did the above steps.
Code:
[rancher@rancher ~]$ 
sudo ros config set rancher.network.interfaces.eth0.address 192.168.1.6/24
sudo ros config set rancher.network.interfaces.eth0.gateway 192.168.1.254
sudo ros config set rancher.network.interfaces.eth0.mtu 1500 
sudo ros config set rancher.network.interfaces.eth0.dhcp false 

Finally ran
sudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server
Code:
Unable to find image 'rancher/server:latest' locally
latest: Pulling from rancher/server
bae382666908: Pull complete 
29ede3c02ff2: Pull complete 
da4e69f33106: Pull complete 
8d43e5f5d27f: Pull complete 
b0de1abb17d6: Pull complete 
422f47db4517: Pull complete 
79d37de643ce: Pull complete 
69d13e08a4fe: Pull complete 
2ddfd3c6a2b7: Pull complete 
bc433fed3823: Pull complete 
b82e188df556: Pull complete 
dae2802428a4: Pull complete 
a6247572ea3c: Pull complete 
884c916ebae4: Pull complete 
85517c9c5365: Pull complete 
02dded9fe690: Pull complete 
fd9f433c3bc6: Pull complete 
44d91b3fea45: Pull complete 
0d463387dfeb: Pull complete 
60753c4d26f0: Pull complete 
a003892966fe: Pull complete 

Digest: sha256:42441f0128fae4d72d51f92de2049392427d462356282a46f28434332967c7e4

Status: Downloaded newer image for rancher/server:latest

e9958cbc517e92688e08bfc07b48b1396d00bbeb24c8e339a725baa1bac50105


This looks like success to me.
I rebooted the VM so that I get the static ip 192.168.1.6 assigned. After reboot,
#ifconfig
Code:
[rancher@rancher log]$ ifconfig 

docker-sys Link encap:Ethernet  HWaddr 00:00:00:00:00:00  

		  inet addr:172.18.42.2  Bcast:172.18.255.255  Mask:255.255.0.0

		  inet6 addr: fe80::8830:41ff:fe86:ec63/64 Scope:Link

		  UP BROADCAST MULTICAST  MTU:1500  Metric:1

		  RX packets:3 errors:0 dropped:0 overruns:0 frame:0

		  TX packets:8 errors:0 dropped:0 overruns:0 carrier:0

		  collisions:0 txqueuelen:1000 

		  RX bytes:216 (216.0 B)  TX bytes:688 (688.0 B)


docker0   Link encap:Ethernet  HWaddr 02:42:FA:ED:C8:76  

		  inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0

		  inet6 addr: fe80::42:faff:feed:c876/64 Scope:Link

		  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

		  RX packets:54 errors:0 dropped:0 overruns:0 frame:0

		  TX packets:85 errors:0 dropped:0 overruns:0 carrier:0

		  collisions:0 txqueuelen:0 

		  RX bytes:1968 (1.9 KiB)  TX bytes:6214 (6.0 KiB)


eth0	  Link encap:Ethernet  HWaddr 00:A0:98:24:D7:11  

		  inet addr:192.168.1.6  Bcast:192.168.1.255  Mask:255.255.255.0

		  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

		  RX packets:8237 errors:0 dropped:0 overruns:0 frame:0

		  TX packets:372 errors:0 dropped:0 overruns:0 carrier:0

		  collisions:0 txqueuelen:1000 

		  RX bytes:1285207 (1.2 MiB)  TX bytes:29376 (28.6 KiB)


lo		Link encap:Local Loopback  

		  inet addr:127.0.0.1  Mask:255.0.0.0

		  inet6 addr: ::1/128 Scope:Host

		  UP LOOPBACK RUNNING  MTU:65536  Metric:1

		  RX packets:0 errors:0 dropped:0 overruns:0 frame:0

		  TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

		  collisions:0 txqueuelen:1 

		  RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)


vethcfabf39 Link encap:Ethernet  HWaddr 42:90:15:A2:3F:8E  

		  inet6 addr: fe80::4090:15ff:fea2:3f8e/64 Scope:Link

		  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

		  RX packets:0 errors:0 dropped:0 overruns:0 frame:0

		  TX packets:13 errors:0 dropped:0 overruns:0 carrier:0

		  collisions:0 txqueuelen:0 

		  RX bytes:0 (0.0 B)  TX bytes:1038 (1.0 KiB)


I waited for 20 minutes and then tried 192.168.1.6:8080 but the browser would fail with "192.168.1.6 didn’t send any data."
When I did netstat -atn
Code:
Active Internet connections (servers and established)

Proto Recv-Q Send-Q Local Address		   Foreign Address		 State	   

tcp		0	  0 0.0.0.0:22			  0.0.0.0:*			   LISTEN	  

tcp		0	  0 :::8080				 :::*					LISTEN	  

tcp		0	  0 :::22				   :::*					LISTEN	  


Clearly 8080 is not bound to ipv4 0.0.0.0. Can anyone help? What am I doing wrong here?
I am running FreenNAS 11.1U1
 

blazerguns

Dabbler
Joined
Mar 15, 2017
Messages
13
Something more interesting is everytime I try to connect, I think it is crashing.

Code:
[rancher@rancher log]$ docker ps

CONTAINER ID		IMAGE			   COMMAND				  CREATED			 STATUS			  PORTS							  NAMES

e9958cbc517e		rancher/server	  "/usr/bin/entry /u..."   About an hour ago   Up 5 minutes		3306/tcp, 0.0.0.0:8080->8080/tcp   romantic_pasteur

[rancher@rancher log]$ docker ps

CONTAINER ID		IMAGE			   COMMAND				  CREATED			 STATUS			  PORTS							  NAMES

e9958cbc517e		rancher/server	  "/usr/bin/entry /u..."   About an hour ago   Up 5 seconds		3306/tcp, 0.0.0.0:8080->8080/tcp   romantic_pasteur

[rancher@rancher log]$ docker ps

CONTAINER ID		IMAGE			   COMMAND				  CREATED			 STATUS			  PORTS							  NAMES

e9958cbc517e		rancher/server	  "/usr/bin/entry /u..."   About an hour ago   Up 15 seconds	   3306/tcp, 0.0.0.0:8080->8080/tcp   romantic_pasteur

[rancher@rancher log]$ 


Up time seem to suddenly drop from 5 minutes to 5 seconds which means it restarted?
 

blazerguns

Dabbler
Joined
Mar 15, 2017
Messages
13
Further confirming that this is surely a crash and it has nothing to with the http request as I earlier thought. Here are the crsh logs from the container.

Code:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibaseCore' defined in io.cattle.platform.app.SystemConfig: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by e9958cbc517e (172.17.0.2) since 2/13/18 11:44 PM

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1578)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)

at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)

at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)

at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)

at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:776)

at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:861)

at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:541)

at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:444)

at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:326)

at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)

at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)

at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)

at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)

at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)

at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)

at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)

at org.eclipse.jetty.server.Server.start(Server.java:387)

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)

at org.eclipse.jetty.server.Server.doStart(Server.java:354)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at io.cattle.platform.launcher.jetty.Main.main(Main.java:146)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at io.cattle.platform.launcher.Main.run(Main.java:188)

at io.cattle.platform.launcher.Main.main(Main.java:251)

Caused by: liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by e9958cbc517e (172.17.0.2) since 2/13/18 11:44 PM

at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:168)

at liquibase.Liquibase.update(Liquibase.java:189)

at liquibase.Liquibase.update(Liquibase.java:181)

at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:342)

at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:299)

at io.cattle.platform.liquibase.Loader.afterPropertiesSet(Loader.java:66)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)

... 34 more

2018-02-14 01:06:35,830 WARN  [:] [] [] [] [main		   ] [o.e.j.u.component.AbstractLifeCycle ] FAILED org.eclipse.jetty.server.Server@6df97b55: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibaseCore' defined in io.cattle.platform.app.SystemConfig: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by e9958cbc517e (172.17.0.2) since 2/13/18 11:44 PM org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibaseCore' defined in io.cattle.platform.app.SystemConfig: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by e9958cbc517e (172.17.0.2) since 2/13/18 11:44 PM

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1578)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)

at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)

at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)

at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)

at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:776)

at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:861)

at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:541)

at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:444)

at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:326)

at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)

at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)

at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)

at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)

at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)

at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)

at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)

at org.eclipse.jetty.server.Server.start(Server.java:387)

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)

at org.eclipse.jetty.server.Server.doStart(Server.java:354)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at io.cattle.platform.launcher.jetty.Main.main(Main.java:146)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at io.cattle.platform.launcher.Main.run(Main.java:188)

at io.cattle.platform.launcher.Main.main(Main.java:251)

Caused by: liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by e9958cbc517e (172.17.0.2) since 2/13/18 11:44 PM

at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:168)

at liquibase.Liquibase.update(Liquibase.java:189)

at liquibase.Liquibase.update(Liquibase.java:181)

at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:342)

at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:299)

at io.cattle.platform.liquibase.Loader.afterPropertiesSet(Loader.java:66)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)

... 34 common frames omitted


2018-02-14 01:06:35,850 ERROR   [main] [ConsoleStatus] Startup Failed [344080ms] org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibaseCore' defined in io.cattle.platform.app.SystemConfig: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by e9958cbc517e (172.17.0.2) since 2/13/18 11:44 PM

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1578)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)

at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)

at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)

at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)

at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)

at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:776)

at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:861)

at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:541)

at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:444)

at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:326)

at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)

at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)

at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)

at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)

at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)

at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)

at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)

at org.eclipse.jetty.server.Server.start(Server.java:387)

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)

at org.eclipse.jetty.server.Server.doStart(Server.java:354)

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)

at io.cattle.platform.launcher.jetty.Main.main(Main.java:146)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at io.cattle.platform.launcher.Main.run(Main.java:188)

at io.cattle.platform.launcher.Main.main(Main.java:251)

Caused by: liquibase.exception.LockException: Could not acquire change log lock.  Currently locked by e9958cbc517e (172.17.0.2) since 2/13/18 11:44 PM

at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:168)

at liquibase.Liquibase.update(Liquibase.java:189)

at liquibase.Liquibase.update(Liquibase.java:181)

at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:342)

at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:299)

at io.cattle.platform.liquibase.Loader.afterPropertiesSet(Loader.java:66)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)

... 34 common frames omitted


STARTUP FAILED [344093] ms

180214  1:06:37 [Note] /usr/sbin/mysqld (mysqld 5.5.58-0ubuntu0.14.04.1) starting as process 24 ...

Uptime: 2  Threads: 1  Questions: 2  Slow queries: 0  Opens: 34  Flush tables: 1  Open tables: 26  Queries per second avg: 1.000

Setting up database

CATTLE_AGENT_PACKAGE_HOST_API_URL=/usr/share/cattle/artifacts/host-api.tar.gz

CATTLE_AGENT_PACKAGE_PER_HOST_SUBNET_URL=/usr/share/cattle/artifacts/rancher-per-host-subnet.zip
 

blazerguns

Dabbler
Joined
Mar 15, 2017
Messages
13
Attaching full error logs as I cant seem to paste it here.
 

Attachments

  • RancherOS.txt
    31.9 KB · Views: 641
Status
Not open for further replies.
Top