执行一个插入操作,结果任务在这里卡住不动了:Tez session hasn't been created yet. Opening 今天执行一个hive表的插入任务,结果在这里卡住不动了:

Tez session hasn‘t been created yet. Opening session

网上一堆的解决办法,比如:

1、将tez-site.xml里的tez.am.resource.memory.mb的参数进行适量的增加;我将改参数增加到了2048M,依然不行;

2、设置队列名称

# 设置引擎

set hive.execution.engine=mr;

set hive.execution.engine=spark;

set hive.execution.engine=tez;

# 如果使用的是mr(原生mapreduce)

SET mapreduce.job.queuename=etl;

# 如果使用的引擎是tez

set tez.queue.name=etl

# 设置队列(etl为队列名称,默认为default)

设置了 tez,报如下错:

Application application_1622620346132_0005 submitted by user hive to unknown queue: etl

设置引擎为mr,报如下错:

Error: Error while processing statement: hive execution engine mr is not supported. (state=42000,code=1) 

最终解决办法:

[yarn@hadoop01 ~]$ yarn application -list
WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of YARN_OPTS.
21/06/02 17:19:55 INFO client.AHSProxy: Connecting to Application History server at hadoop01.com/192.168.100.151:10200
21/06/02 17:19:55 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):1
                Application-Id        Application-Name        Application-Type          User         Queue                 State           Final-State           Progress                           Tracking-URL
application_1620261296802_0013    Thrift JDBC/ODBC Server                   SPARK         spark       default               RUNNING             UNDEFINED                10%               http://hadoop04.com:4040

[yarn@hadoop01 ~]$ yarn application -kill application_1620261296802_0013

不知道为什么spark的Thrift Server会影响这里,于是先用yarn杀掉,然后再重启Thrift Server;再执行hive任务就OK了!

---------------------------华丽的分割线---------------------------------------

今天这个问题又出现了,说明上次杀掉Thrift Server的方法治标不治本;

这次将tez的配置改小:tez.am.resource.memory.mb =256

然后就可以了,网上说的调小tez.am.resource.memory.mb的值;同时,增大yarn中 Memory allocated for all YARN containers on a node的值;我只把tez.am.resource.memory.mb改小就可以了。。。

 

 

 

 

Logo

为开发者提供学习成长、分享交流、生态实践、资源工具等服务,帮助开发者快速成长。

更多推荐