虚拟机HDFS警告解决办法:WARN hdfs.DataStreamer: Excluding datanode DatanodeInfoWithStorage
查看防火墙状态,输入命令:systemctl status firewalld。设置“SELINUX=disabled” ,再次执行上传文件就不会出现警告了。禁用防火墙:systemctl disable firewalld。停止防火墙:systemctl stop firewalld。
问题
当输入命令,上传本地文件到HDFS时,出现以下提示:
2022-11-25 20:45:46,943 WARN hdfs.DataStreamer: Exception in createBlockOutputStream blk_1073741837_1013
java.io.IOException: Got error, status=ERROR, status message , ack with firstBadLink as 192.168.1.102:9866
at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:128)
at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:104)
at org.apache.hadoop.hdfs.DataStreamer.createBlockOutputStream(DataStreamer.java:1827)
at org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1728)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:713)
2022-11-25 20:45:46,945 WARN hdfs.DataStreamer: Abandoning BP-1560191366-192.168.1.101-1669379449147:blk_1073741837_1013
2022-11-25 20:45:46,951 WARN hdfs.DataStreamer: Excluding datanode DatanodeInfoWithStorage[192.168.1.102:9866,DS-514188e6-373b-49dc-9274-a42184a4316b,DISK]
2022-11-25 20:45:46,965 WARN hdfs.DataStreamer: Exception in createBlockOutputStream blk_1073741838_1014
java.io.IOException: Got error, status=ERROR, status message , ack with firstBadLink as 192.168.1.103:9866
at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:128)
at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:104)
at org.apache.hadoop.hdfs.DataStreamer.createBlockOutputStream(DataStreamer.java:1827)
at org.apache.hadoop.hdfs.DataStreamer.nextBlockOutputStream(DataStreamer.java:1728)
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:713)
2022-11-25 20:45:46,966 WARN hdfs.DataStreamer: Abandoning BP-1560191366-192.168.1.101-1669379449147:blk_1073741838_1014
2022-11-25 20:45:46,969 WARN hdfs.DataStreamer: Excluding datanode DatanodeInfoWithStorage[192.168.1.103:9866,DS-24330123-4141-4ff3-9301-7740ac23b941,DISK]
此时在hadoop webui界面查看,其实文件已成功上传
这里是因为没有关闭所有节点的防火墙,一定要关闭所有节点的防火墙
依次输入下列命令:
停止防火墙:systemctl stop firewalld
禁用防火墙:systemctl disable firewalld
查看防火墙状态,输入命令:systemctl status firewalld
永久关闭selinux
使用 vim /etc/selinux/config
命令修改 /etc/selinux/config
文件
设置“SELINUX=disabled” ,再次执行上传文件就不会出现警告了。
注:每个节点都需要设置
更多推荐
所有评论(0)