prestosql / trino 连接 Elasticsearch 遇到的问题
文章目录1. 认证问题2. 证书问题3. Elasticsearch 拒绝连接最后一开始使用presto 连接elasticsearch时,发现无法进行用户名和密码进行身份的认证,在网上找了好久才知道prestosql 更名为 trino 了。最后通过安装trino 可以设置用户名和密码成功。不过在设置过程中又出现了一系列的问题,通过去github、stackoverflow、trino社区进行提
一开始使用presto 连接elasticsearch时,发现无法进行用户名和密码进行身份的认证,在网上找了好久才知道prestosql 更名为 trino 了。最后通过安装trino 可以设置用户名和密码成功。不过在设置过程中又出现了一系列的问题,通过去github、stackoverflow、trino社区进行提问,经过网友的回答,终于解决了这个问题,在这里记录一下。
1. 认证问题
2021-07-21T10:28:24.618+0800 ERROR NodeRefresher io.trino.plugin.elasticsearch.client.ElasticsearchClient Error refreshing nodes
io.trino.spi.TrinoException: Connection closed
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.doRequest(ElasticsearchClient.java:803)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.fetchNodes(ElasticsearchClient.java:381)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.refreshNodes(ElasticsearchClient.java:185)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.http.ConnectionClosedException: Connection closed
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyncRequestExecutor.java:344)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:261)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
... 1 more
由于没有开启 tls 导致es断开了连接, 需要在
elasticsearch.properties
文件中添加elasticsearch.tls.enabled=true
2. 证书问题
2021-07-21T10:36:49.799+0800 INFO main io.trino.metadata.StaticCatalogStore -- Loading catalog etc/catalog/elasticsearch.properties --
2021-07-21T10:36:50.445+0800 INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
2021-07-21T10:36:50.446+0800 INFO main Bootstrap jmx.base-name ---- ----
2021-07-21T10:36:50.447+0800 INFO main Bootstrap elasticsearch.backoff-init-delay 500.00ms 500.00ms Initial delay to wait between backpressure retries
2021-07-21T10:36:50.447+0800 INFO main Bootstrap elasticsearch.backoff-max-delay 20.00s 20.00s Maximum delay to wait between backpressure retries
2021-07-21T10:36:50.447+0800 INFO main Bootstrap elasticsearch.connect-timeout 1.00s 60.00s Elasticsearch connect timeout
2021-07-21T10:36:50.448+0800 INFO main Bootstrap elasticsearch.default-schema-name default default Default schema name to use
2021-07-21T10:36:50.448+0800 INFO main Bootstrap elasticsearch.host ---- 192.168.90.186
2021-07-21T10:36:50.449+0800 INFO main Bootstrap elasticsearch.http-thread-count 4 4 Number of threads handling HTTP connections to Elasticsearch
2021-07-21T10:36:50.449+0800 INFO main Bootstrap elasticsearch.ignore-publish-address false false
2021-07-21T10:36:50.450+0800 INFO main Bootstrap elasticsearch.tls.keystore-password [REDACTED] [REDACTED]
2021-07-21T10:36:50.450+0800 INFO main Bootstrap elasticsearch.tls.keystore-path ---- ----
2021-07-21T10:36:50.450+0800 INFO main Bootstrap elasticsearch.max-http-connections 25 50 Maximum number of persistent HTTP connections to Elasticsearch
2021-07-21T10:36:50.451+0800 INFO main Bootstrap elasticsearch.max-retry-time 30.00s 30.00s Maximum timeout in case of multiple retries
2021-07-21T10:36:50.451+0800 INFO main Bootstrap elasticsearch.node-refresh-interval 1.00m 1.00s How often to refresh the list of available Elasticsearch nodes
2021-07-21T10:36:50.452+0800 INFO main Bootstrap elasticsearch.port 9200 9200
2021-07-21T10:36:50.452+0800 INFO main Bootstrap elasticsearch.request-timeout 10.00s 60.00s Elasticsearch request timeout
2021-07-21T10:36:50.453+0800 INFO main Bootstrap elasticsearch.scroll-size 1000 5000 Scroll batch size
2021-07-21T10:36:50.453+0800 INFO main Bootstrap elasticsearch.scroll-timeout 1.00m 30.00s Scroll timeout
2021-07-21T10:36:50.453+0800 INFO main Bootstrap elasticsearch.security ---- PASSWORD
2021-07-21T10:36:50.454+0800 INFO main Bootstrap elasticsearch.tls.enabled false true
2021-07-21T10:36:50.454+0800 INFO main Bootstrap elasticsearch.tls.truststore-path ---- ----
2021-07-21T10:36:50.454+0800 INFO main Bootstrap elasticsearch.tls.truststore-password [REDACTED] [REDACTED]
2021-07-21T10:36:50.455+0800 INFO main Bootstrap elasticsearch.tls.verify-hostnames true true
2021-07-21T10:36:50.455+0800 INFO main Bootstrap elasticsearch.auth.password [REDACTED] [REDACTED]
2021-07-21T10:36:50.456+0800 INFO main Bootstrap elasticsearch.auth.user ---- cloudfall
2021-07-21T10:36:51.847+0800 ERROR main io.trino.plugin.elasticsearch.client.ElasticsearchClient Error refreshing nodes
io.trino.spi.TrinoException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.doRequest(ElasticsearchClient.java:803)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.fetchNodes(ElasticsearchClient.java:381)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.refreshNodes(ElasticsearchClient.java:185)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.initialize(ElasticsearchClient.java:167)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at io.airlift.bootstrap.LifeCycleManager.startInstance(LifeCycleManager.java:241)
at io.airlift.bootstrap.LifeCycleManager.addInstance(LifeCycleManager.java:212)
at io.airlift.bootstrap.LifeCycleModule.provision(LifeCycleModule.java:62)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:117)
at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:93)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:296)
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:169)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:45)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1100)
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1133)
at org.weakref.jmx.guice.GuiceMBeanExporter.export(GuiceMBeanExporter.java:92)
at org.weakref.jmx.guice.GuiceMBeanExporter.<init>(GuiceMBeanExporter.java:42)
at org.weakref.jmx.guice.GuiceMBeanExporter$$FastClassByGuice$$2808288.GUICE$TRAMPOLINE(<generated>)
at org.weakref.jmx.guice.GuiceMBeanExporter$$FastClassByGuice$$2808288.apply(<generated>)
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82)
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
at com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:33)
at com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:98)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:109)
at io.airlift.bootstrap.LifeCycleModule.provision(LifeCycleModule.java:54)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:117)
at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:66)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:93)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:296)
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:169)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:45)
at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:213)
at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:186)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:113)
at com.google.inject.Guice.createInjector(Guice.java:87)
at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:270)
at io.trino.plugin.elasticsearch.ElasticsearchConnectorFactory.create(ElasticsearchConnectorFactory.java:70)
at io.trino.connector.ConnectorManager.createConnector(ConnectorManager.java:360)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:215)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:207)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:193)
at io.trino.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:88)
at io.trino.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68)
at io.trino.server.Server.doStart(Server.java:124)
at io.trino.server.Server.lambda$start$0(Server.java:77)
at io.trino.$gen.Trino_359____20210721_023634_1.run(Unknown Source)
at io.trino.server.Server.start(Server.java:77)
at io.trino.server.TrinoServer.main(TrinoServer.java:38)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:349)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:287)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1356)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1231)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1174)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1074)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:283)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:353)
at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:509)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:276)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1334)
... 19 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
... 25 more
2021-07-21T10:36:51.883+0800 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle starting...
2021-07-21T10:36:51.883+0800 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle startup complete
2021-07-21T10:36:51.885+0800 INFO main io.trino.metadata.StaticCatalogStore -- Added catalog elasticsearch using connector elasticsearch --
2021-07-21T10:36:51.888+0800 INFO main io.trino.security.AccessControlManager Using system access control default
2021-07-21T10:36:51.938+0800 INFO main io.trino.server.Server ======== SERVER STARTED ========
2021-07-21T10:36:52.890+0800 ERROR NodeRefresher io.trino.plugin.elasticsearch.client.ElasticsearchClient Error refreshing nodes
io.trino.spi.TrinoException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.doRequest(ElasticsearchClient.java:803)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.fetchNodes(ElasticsearchClient.java:381)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.refreshNodes(ElasticsearchClient.java:185)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:349)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:287)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1356)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1231)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1174)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1074)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:283)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:353)
at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:509)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
... 1 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:276)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1334)
... 19 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
... 25 more
以上的问题是es 证书配置问题,或者证书配置,需要添加以下配置
elasticsearch.tls.keystore-path=/opt/pem/http.pem
elasticsearch.tls.truststore-path=/opt/pem/elastic-certificates.pem
以上的证书对应 es 中配置,主要需要进行格式转换,可以通过命令的方式和一些在线工具进行转换, 博主本人是通过在线工具来进行转换的,在线工具链接
https://www.sslshopper.com/ssl-converter.html
xpack.security.http.ssl.keystore.path: "/usr/share/elasticsearch/certs/http.p12"
xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/certs/elastic-certificates.p12
本来以为问题到这就全部解决了, 结果发现我还是太年轻了…
3. Elasticsearch 拒绝连接
2021-07-21T10:48:44.669+0800 INFO main io.trino.metadata.StaticCatalogStore -- Loading catalog etc/catalog/elasticsearch.properties --
2021-07-21T10:48:45.311+0800 INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
2021-07-21T10:48:45.312+0800 INFO main Bootstrap jmx.base-name ---- ----
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.backoff-init-delay 500.00ms 500.00ms Initial delay to wait between backpressure retries
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.backoff-max-delay 20.00s 20.00s Maximum delay to wait between backpressure retries
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.connect-timeout 1.00s 60.00s Elasticsearch connect timeout
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.default-schema-name default default Default schema name to use
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.host ---- 192.168.90.186
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.http-thread-count 4 4 Number of threads handling HTTP connections to Elasticsearch
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.ignore-publish-address false false
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.tls.keystore-password [REDACTED] [REDACTED]
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.tls.keystore-path ---- /opt/pem/http.pem
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.max-http-connections 25 50 Maximum number of persistent HTTP connections to Elasticsearch
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.max-retry-time 30.00s 30.00s Maximum timeout in case of multiple retries
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.node-refresh-interval 1.00m 1.00s How often to refresh the list of available Elasticsearch nodes
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.port 9200 9200
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.request-timeout 10.00s 60.00s Elasticsearch request timeout
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.scroll-size 1000 5000 Scroll batch size
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.scroll-timeout 1.00m 30.00s Scroll timeout
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.security ---- PASSWORD
2021-07-21T10:48:45.312+0800 INFO main Bootstrap elasticsearch.tls.enabled false true
2021-07-21T10:48:45.313+0800 INFO main Bootstrap elasticsearch.tls.truststore-path ---- /opt/pem/elastic-certificates.pem
2021-07-21T10:48:45.313+0800 INFO main Bootstrap elasticsearch.tls.truststore-password [REDACTED] [REDACTED]
2021-07-21T10:48:45.313+0800 INFO main Bootstrap elasticsearch.tls.verify-hostnames true true
2021-07-21T10:48:45.313+0800 INFO main Bootstrap elasticsearch.auth.password [REDACTED] [REDACTED]
2021-07-21T10:48:45.313+0800 INFO main Bootstrap elasticsearch.auth.user ---- cloudfall
2021-07-21T10:48:46.912+0800 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle starting...
2021-07-21T10:48:46.913+0800 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle startup complete
2021-07-21T10:48:46.915+0800 INFO main io.trino.metadata.StaticCatalogStore -- Added catalog elasticsearch using connector elasticsearch --
2021-07-21T10:48:46.918+0800 INFO main io.trino.security.AccessControlManager Using system access control default
2021-07-21T10:48:46.961+0800 INFO main io.trino.server.Server ======== SERVER STARTED ========
2021-07-21T10:48:47.889+0800 ERROR NodeRefresher io.trino.plugin.elasticsearch.client.ElasticsearchClient Error refreshing nodes
io.trino.spi.TrinoException: Connection refused
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.doRequest(ElasticsearchClient.java:803)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.fetchNodes(ElasticsearchClient.java:381)
at io.trino.plugin.elasticsearch.client.ElasticsearchClient.refreshNodes(ElasticsearchClient.java:185)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
... 1 more
遇到这个问题的时候,一开始以为只是很简单的问题,想着拒绝连接无非就是那几种情况:
1. 网络问题
2. 防火墙端口没开
3. Elasticsearch 出问题了 (es 崩了,或者没启动等等)
最后排查下来发现都不是… 感觉心态有点小崩。 于是去各大平台求助网友,最后在花了三天时间在 trino社区解决了,问题是出在trino 中, 连接ES 时少了
elasticsearch.ignore-publish-address=true
这个配置。
最后
问题到这就解决了,后续如果再遇到问题再进一步记录。在博客末尾给出我的elasticsearch.properties配置文件
connector.name=elasticsearch
elasticsearch.host=xxx
elasticsearch.port=9200
elasticsearch.default-schema-name=default
elasticsearch.scroll-size=5000
elasticsearch.scroll-timeout=30s
elasticsearch.request-timeout=60s
elasticsearch.connect-timeout=60s
elasticsearch.max-retry-time=30s
elasticsearch.node-refresh-interval =1s
elasticsearch.max-http-connections=50
elasticsearch.http-thread-count=4
elasticsearch.security=PASSWORD
elasticsearch.auth.user=xxx
elasticsearch.auth.password=xxx
elasticsearch.tls.enabled=true
elasticsearch.tls.keystore-path=/opt/pem/http.pem
elasticsearch.tls.truststore-path=/opt/pem/elastic-certificates.pem
elasticsearch.ignore-publish-address=true
更多推荐
所有评论(0)