现象描述

beeline访问zookeeper报错

beeline报错
(base) -bash-4.2$ ./beeline -u 'jdbc:hive2://10.169.49.131:2181/default?spark.yarn.queue=szopam-test;username=leon'
Connecting to jdbc:hive2://10.169.49.131:2181/default?spark.yarn.queue=szopam-test;username=leon
21/11/12 16:51:44 INFO jdbc.Utils: Supplied authorities: 10.169.49.131:2181
21/11/12 16:51:44 INFO jdbc.Utils: Resolved authority: 10.169.49.131:2181
21/11/12 16:51:44 WARN jdbc.HiveConnection: Failed to connect to 10.169.49.131:2181
Unexpected end of file when reading from HS2 server. The root cause might be too many concurrent connections. Please ask the administrator to check the number of active connections, and adjust hive.server2.thrift.max.worker.threads if applicable.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://10.169.49.131:2181/default?spark.yarn.queue=szopam-test;username=leon: null (state=08S01,code=0)
Beeline version 2.3.7 by Apache Hive
beeline>

zookeeper日志

java.io.IOException: Len error. A message from /10.128.151.5:36498 with advertised length of 16777216 is either a malformed message or too large to process (length is greater than jute.maxbuffer=1048575)
    at org.apache.zookeeper.server.NIOServerCnxn.readLength(NIOServerCnxn.java:549)
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:339)
    at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:508)
    at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

 

原因分析 

客户端发送的包太大,超过jute.maxbuffer的设置,默认大小为1048575。

解决方法

修改jute.maxbuffer的配置,设置成50M

bin/zkServer.sh

JVMFLAGS="$JVMFLAGS -Djute.maxbuffer=50000000"

Logo

为开发者提供学习成长、分享交流、生态实践、资源工具等服务,帮助开发者快速成长。

更多推荐