问题现场环境:

1、本地虚拟机CentOS7 下的docker环境。

2、docker下的redis、mysql已正常启动,且虚拟机外可正常访问。

3、启动容器(SpringCloud 网关服务)需要使用 docker下的redis、MySql。

 


问题现象:

在启动容器时,出现错误异常c.n.d.s.t.d.RedirectingEurekaHttpClient:Request execution error和nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool,如下所示:

2018-09-13 12:53:25.742 ERROR 1 --- [           main] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error

com.sun.jersey.api.client.ClientHandlerException: java.net.NoRouteToHostException: No route to host (Host unreachable)
	at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187) ~[jersey-apache-client4-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27) ~[eureka-client-1.4.12.jar!/:1.4.12]
	at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.getApplicationsInternal(AbstractJerseyEurekaHttpClient.java:194) ~[eureka-client-1.4.12.jar!/:1.4.12]

……

Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
2018-09-13 12:53:29.150 ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application startup failed

org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.cloud.netflix.eureka.EurekaDiscoveryClientConfiguration'; nested exception is org.springframework.data
.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:50) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:348) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:151) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:114) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:879) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.finishRefresh(EmbeddedWebApplicationContext.java:144) ~[spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:545) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) ~[spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1162) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1151) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at com.wondersoft.gateway.GatewayApplication.main(GatewayApplication.java:31) [classes!/:1.5.2.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_111]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_111]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_111]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_111]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]

 


问题分析:

1、异常c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error

com.sun.jersey.api.client.ClientHandlerException: java.net.NoRouteToHostException: No route to host (Host unreachable),是处于网关在连接另外一个服务(注册中心)时,抛出的异常,属于无法正常路由连接到注册中心服务。

2、异常nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool,是处于正在连接redis时,抛出的异常,属于无法连接redis。

基于上述两点,问题属于容器间网络通信问题

 


解决方法:

1、关闭防火墙。

使用root用户,执行  systemctl stop firewalld.service 命令,关闭防火墙。

 

2、重新启动 网关服务容器,正常启动,则问题解决。

3、如果启动时,出现错误

iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8111 -j DNAT,如下图:

 

[root@cen7 data]# docker start gateway
Error response from daemon: Cannot start container gateway: iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8111 -j DNAT --to-destination 172.17.0.5:8111 ! -i docker0: iptables: No chain/target/match by that name.
 (exit status 1)
Error: failed to start containers: [gateway]

则需要重建docker0网络恢复,按顺序执行如下命令,即可。

pkill docker 
iptables -t nat -F 
ifconfig docker0 down 
brctl delbr docker0 
docker -d 
service docker restart

执行过程如下:

[root@cen7 data]# pkill docker 
[root@cen7 data]# iptables -t nat -F 
[root@cen7 data]# ifconfig docker0 down 
[root@cen7 data]# brctl delbr docker0 
[root@cen7 data]# docker -d 
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock) 
INFO[0000] [graphdriver] using prior storage driver "devicemapper" 
WARN[0000] Running modprobe bridge nf_nat failed with message: , error: exit status 1 
INFO[0000] Loading containers: start.                   
..............
INFO[0000] Loading containers: done.                    
INFO[0000] Daemon has completed initialization          
INFO[0000] Docker daemon                                 commit=786b29d execdriver=native-0.2 graphdriver=devicemapper version=1.7.1
^CINFO[0061] Processing signal 'interrupt'                
[root@cen7 data]# systemctl restart docker

4、启动所需容器即可。

 

Logo

华为开发者空间,是为全球开发者打造的专属开发空间,汇聚了华为优质开发资源及工具,致力于让每一位开发者拥有一台云主机,基于华为根生态开发、创新。

更多推荐