[2022-02-22T00:09:22,656][WARN ][logstash.outputs.elasticsearch][ods_host][dc07d40532915e5f6e1679ca41a6bc431b236880d34d44f80fc563f5e4aa2692] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://elastic:xxxxxx@1.1.1.1:9222/][Manticore::SocketTimeout] Read timed out {:url=>http://elastic:xxxxxx@1.1.1.1:9222/, :error_message=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@1.1.1.1:9222/][Manticore::SocketTimeout] Read timed out", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2022-02-22T00:09:22,770][ERROR][logstash.outputs.elasticsearch][ods_host][dc07d40532915e5f6e1679ca41a6bc431b236880d34d44f80fc563f5e4aa2692] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@1.1.1.1:9222/][Manticore::SocketTimeout] Read timed out", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError", :will_retry_in_seconds=>2}
[2022-02-22T00:09:24,783][WARN ][logstash.outputs.elasticsearch][ods_host] Restored connection to ES instance {:url=>"http://elastic:xxxxxx@1.1.1.1:9222/"}

通过错误,我们大致可以判断是,这个报错的根源是ES集群导致的。
ES POOL 满了,导致,其他客户端连接挤不进去。

于是查看下ES连接池使用情况:
这一块参考:https://www.cnblogs.com/bonelee/p/7840629.html
curl -X GET ‘http://1.11.91.1:9200/_nodes/stats?pretty’
在这里插入图片描述
可以看出,线程数是17个,活跃的线程是17个,说明,17个连接池线程,都被占用了

那么,问题的原因已经定位到了,解决办法就是调高ES线程数,或者,增加ES集群节点。

参考:
https://www.cnblogs.com/xiaochina/p/8820738.html

Logo

为开发者提供学习成长、分享交流、生态实践、资源工具等服务,帮助开发者快速成长。

更多推荐