CDH6.3.1部署

CDH部署1.准备工作资源有限，使用虚拟机进行如下配置机器配置IP主机名配置192.168.130.221CDH14G 2处理器 60G磁盘192.168.130.222CDH22G 1处理器 20G磁盘192.168.130.223CDH32G 1处理器 20G磁盘建议配置IP主机名配置192.168.130.221CDH18G 4处理器 80G磁盘192.168.130.222CDH24G 2

DATA数据猿

2195人浏览 · 2022-03-18 22:26:49

DATA数据猿 · 2022-03-18 22:26:49 发布

CDH部署

1.准备工作

资源有限，使用虚拟机进行如下配置

机器配置

IP	主机名	配置
192.168.130.221	CDH1	4G 2处理器 60G磁盘
192.168.130.222	CDH2	2G 1处理器 20G磁盘
192.168.130.223	CDH3	2G 1处理器 20G磁盘

建议配置

IP	主机名	配置
192.168.130.221	CDH1	8G 4处理器 80G磁盘
192.168.130.222	CDH2	4G 2处理器 50G磁盘
192.168.130.223	CDH3	4G 2处理器 50G磁盘

1.1 免密操作

cd /root/.ssh
[root@CDH1 .ssh]# ssh-keygen -t rsa
[root@CDH1 .ssh]# ssh-copy-id CDH1
[root@CDH1 .ssh]# ssh-copy-id CDH2
[root@CDH1 .ssh]# ssh-copy-id CDH3

[root@CDH2 .ssh]# ssh-keygen -t rsa
[root@CDH2 .ssh]# ssh-copy-id CDH1
[root@CDH2 .ssh]# ssh-copy-id CDH2
[root@CDH2 .ssh]# ssh-copy-id CDH3

[root@CDH3 .ssh]# ssh-keygen -t rsa
[root@CDH3 .ssh]# ssh-copy-id CDH1
[root@CDH3 .ssh]# ssh-copy-id CDH2
[root@CDH3 .ssh]# ssh-copy-id CDH3

1.2 安装JDK

使用以往安装的JDK(这样做在启动CM server的时候会遇到问题，如果想跳过问题，请看2.7目录下的重装jdk)

1.3 安装MySQL

安装后创建如下数据库

## 查看是否有MySQL服务
[root@CDH1 software]# rpm -qa|grep -i mysql
mysql-community-release-el6-5.noarch
## 如果有，先卸载
[root@CDH1 software]# rpm -e --nodeps mysql-community-release-el6-5.noarch
## 删除原有MySQL依赖
[root@CDH1 software]# yum remove mysql-libs
## 下载MySQL依赖
[root@CDH1 software]# yum install libaio
[root@CDH1 software]# yum -y install autoconf
[root@CDH1 software]# wget https://downloads.mysql.com/archives/get/p/23/file/MySQL-shared-compat-5.6.24-1.el6.x86_64.rpm
[root@CDH1 software]# wget https://downloads.mysql.com/archives/get/p/23/file/MySQL-shared-5.6.24-1.el6.x86_64.rpm
[root@CDH1 software]# rpm -ivh MySQL-shared-5.6.24-1.el6.x86_64.rpm
[root@CDH1 software]# rpm -ivh MySQL-shared-compat-5.6.24-1.el6.x86_64.rpm

安装服务端

## 安装MySQL服务端
[root@CDH1 software]# rpm -ivh MySQL-server-5.6.24-1.el6.x86_64.rpm
## 查看随机产生的密码
[root@CDH1 software]# cat /root/.mysql_secret
# The random password set for the root user at Sun Mar 13 16:16:04 2022 (local time): 0bVp5O9LI0UmHPo3
## 查看MySQL状态并启动
[root@CDH1 software]# service mysql status
 ERROR! MySQL is not running
[root@CDH1 software]# service mysql start
Starting MySQL.. SUCCESS!
[root@CDH1 software]#

MySQL客户端

## 安装MySQL客户端
[root@CDH1 software]# rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm
## 连接MySQL
[root@CDH1 software]# mysql -uroot -p0bVp5O9LI0UmHPo3
## 修改密码
mysql> set password=password('111213');
## 退出
mysql> exit

MySQL权限

mysql> use mysql;
mysql> select user,host from user;
mysql> update user set host='%' where host='localhost';
mysql> delete from user where host!='%';
mysql> flush privileges;
mysql> quit;

MySQL建库

1）创建各组件需要的数据库
mysql> GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm';

mysql> CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
mysql> CREATE DATABASE hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
mysql> CREATE DATABASE oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
mysql> CREATE DATABASE hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

1.4 MySQL连接jar

上传mysql-connector-java-5.1.27-bin.jar到/usr/share/java

[root@CDH1 lib]# mkdir /usr/share/java
[root@CDH1 lib]# cp mysql-connector-java.jar  /usr/share/java
[root@CDH1 lib]# cd /usr/share/java
[root@CDH1 java]# ls
mysql-connector-java.jar
[root@CDH1 java]# scp -r mysql-connector-java.jar CDH2:/usr/share/
root@cdh2's password:
mysql-connector-java.jar                                                                        
[root@CDH1 java]# scp -r mysql-connector-java.jar CDH3:/usr/share/
root@cdh3's password:
mysql-connector-java.jar

1.5 CM上传jar

[root@CDH1 software]# tar -zxvf cm6.3.1-redhat7.tar.gz -C /opt/apps/
cm6.3.1/
cm6.3.1/RPM-GPG-KEY-cloudera
cm6.3.1/RPMS/
cm6.3.1/RPMS/x86_64/
cm6.3.1/RPMS/x86_64/cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
cm6.3.1/RPMS/x86_64/enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
cm6.3.1/RPMS/x86_64/cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
cm6.3.1/RPMS/x86_64/cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
cm6.3.1/RPMS/x86_64/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
cm6.3.1/RPMS/x86_64/cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
cm6.3.1/RPMS/noarch/
cm6.3.1/repodata/
cm6.3.1/repodata/3662f97de72fd44c017bb0e25cee3bc9398108c8efb745def12130a69df2ecb2-filelists.sqlite.bz2
cm6.3.1/repodata/repomd.xml
cm6.3.1/repodata/49e4d60647407a36819f1d8ed901258a13361749b742e3be9065025ad31feb8e-filelists.xml.gz
cm6.3.1/repodata/8afda99b921fd1538dd06355952719652654fc06b6cd14515437bda28376c03d-other.sqlite.bz2
cm6.3.1/repodata/43f3725f730ee7522712039982aa4befadae4db968c8d780c8eb15ae9872cd4d-primary.xml.gz
cm6.3.1/repodata/repomd.xml.asc
cm6.3.1/repodata/b9300879675bdbc300436c1131a910a535b8b5a5dc6f38e956d51769b6771a96-primary.sqlite.bz2
cm6.3.1/repodata/e28836e19e07f71480c4dad0f7a87a804dc93970ec5277ad95614e8ffcff0d58-other.xml.gz
cm6.3.1/repodata/repomd.xml.key
cm6.3.1/SRPMS/
[root@CDH1 software]#

1.6 将指定文件复制到创建的cloudera-manager

[root@CDH1 /]# mkdir /opt/cloudera-manager

[root@CDH1 x86_64]# pwd
/opt/apps/cm6.3.1/RPMS/x86_64
[root@CDH1 x86_64]# cp cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/
[root@CDH1 x86_64]# cp cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/
[root@CDH1 x86_64]# cp cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/

2.安装CM

CDH1	CDH2	CDH3
cloudera-scm-server cloudera-scm-agent	cloudera-scm-agent	cloudera-scm-agent

2.1 安装cloudera-manager-daemons

CDH1

## 安装
[root@CDH1 cloudera-manager]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
警告：cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm: 头V3 RSA/SHA256 Signature, 密钥 ID b0b19c9f: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:cloudera-manager-daemons-6.3.1-14################################# [100%]

## 安装后在opt目录下生成cloudera文件
[root@CDH1 cloudera-manager]# cd ..
[root@CDH1 opt]# ls
apps  cloudera  cloudera-manager  lib  software
[root@CDH1 opt]# cd cloudera
[root@CDH1 cloudera]# ll
总用量 12
drwxr-xr-x. 27 cloudera-scm cloudera-scm 8192 3月  12 15:51 cm
[root@CDH1 cloudera]#

## 将cloudera-manager下目录复制到CDH2和CDH3
[root@CDH1 cloudera-manager]# scp -r /opt/cloudera-manager/ CDH2:/opt/                           
[root@CDH1 cloudera-manager]# scp -r /opt/cloudera-manager/ CDH3:/opt/

CDH2

[root@CDH2 cloudera-manager]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
[root@CDH2 opt]# ls
apps  cloudera  cloudera-manager  software

CDH3

[root@CDH3 cloudera-manager]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
[root@CDH3 opt]# ls
apps  cloudera  cloudera-manager  software

2.2 安装cloudera-manager-agent

CDH1

## 安装预备环境

[root@CDH1 cloudera]# yum install bind-utils psmisc cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs /lib/lsb/init-functions httpd mod_ssl openssl-devel python-psycopg2 MySQL-python libxslt
已加载插件：fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.163.com
 * extras: mirrors.nju.edu.cn
 * updates: mirrors.ustc.edu.cn

更新完毕:
  httpd.x86_64 0:2.4.6-97.el7.centos.4

作为依赖被升级:
  cyrus-sasl-lib.x86_64 0:2.1.26-24.el7_9        e2fsprogs.x86_64 0:1.42.9-19.el7         e2fsprogs-libs.x86_64 0:1.42.9-19.el7        httpd-tools.x86_64 0:2.4.6-97.el7.centos.4
  krb5-libs.x86_64 0:1.15.1-51.el7_9             libcom_err.x86_64 0:1.42.9-19.el7        libselinux.x86_64 0:2.5-15.el7               libselinux-python.x86_64 0:2.5-15.el7
  libselinux-utils.x86_64 0:2.5-15.el7           libss.x86_64 0:1.42.9-19.el7             openssl.x86_64 1:1.0.2k-24.el7_9             openssl-libs.x86_64 1:1.0.2k-24.el7_9
  zlib.x86_64 0:1.2.7-19.el7_9

完毕！

## 安装cloudera-manager-agent
[root@CDH1 cloudera-manager]# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
警告：cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm: 头V3 RSA/SHA256 Signature, 密钥 ID b0b19c9f: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:cloudera-manager-agent-6.3.1-1466################################# [100%]
Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-agent.service to /usr/lib/systemd/system/cloudera-scm-agent.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/supervisord.service to /usr/lib/systemd/system/supervisord.service.
[root@CDH1 cloudera-manager]#

CDH2

[root@CDH2 opt]# yum install bind-utils psmisc cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs /lib/lsb/init-functions httpd mod_ssl openssl-devel python-psycopg2 MySQL-python libxslt


[root@CDH2 cloudera-manager]# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm

CDH3

[root@CDH3 cloudera]# yum install bind-utils psmisc cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs /lib/lsb/init-functions httpd mod_ssl openssl-devel python-psycopg2 MySQL-python libxslt

[root@CDH3 cloudera-manager]# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm

2.3 安装agent的server节点

[root@CDH1 cloudera-manager]# vim /etc/cloudera-scm-agent/config.ini
server_host=CDH1
[root@CDH2 cloudera-manager]# vim /etc/cloudera-scm-agent/config.ini
server_host=CDH1
[root@CDH3 cloudera-manager]# vim /etc/cloudera-scm-agent/config.ini
server_host=CDH1

2.4 安装cloudera-manager-server

[root@CDH1 cloudera-manager]# rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
警告：cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm: 头V3 RSA/SHA256 Signature, 密钥 ID b0b19c9f: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:cloudera-manager-server-6.3.1-146################################# [100%]
Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-server.service to /usr/lib/systemd/system/cloudera-scm-server.service.
[root@CDH1 cloudera-manager]#

2.5 上传指定文件到parcel-repo

## 如下三个文件皆为上传
[root@CDH1 parcel-repo]# ls
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel  CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1  manifest.json
[root@CDH1 parcel-repo]# mv CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha
[root@CDH1 parcel-repo]#

2.6 修改server的db.properties

[root@CDH1 parcel-repo]# vim /etc/cloudera-scm-server/db.properties
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=CDH1:3306
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=111213
com.cloudera.cmf.db.setupType=EXTERNAL

2.7 启动server服务


[root@CDH1 java]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm
Enter SCM password: scm
JAVA_HOME=/opt/apps/jdk1.8.0_261
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /opt/apps/jdk1.8.0_261/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!

[root@CDH1 schema]# systemctl start cloudera-scm-server

启动失败

## 查看启动状态，发现报错，而且在日志目录/var/log/cloudera-scm-server下也找不到相关文件
[root@CDH1 cm]# systemctl status cloudera-scm-server

## 使用如下命令查看原因，发现是Java找不到，但是我这里已经安装成功并可以使用，
## 经查询发现，他自动检查/usr/java目录下的jdk，所以这里需要重装一下
[root@CDH1 cm]# journalctl -xe
Error: Unable to find a compatible version of Java on this host,
 either because JAVA_HOME has not been set or because a   
 compatible version of Java is not installed.             
Please install either:

重装jdk

[root@CDH1 software]# mkdir /usr/java
[root@CDH1 software]# tar -zxvf jdk-8u261-linux-x64.tar.gz  -C /usr/java/

## CDH1安装jdk,配置环境
[root@CDH1 jdk1.8.0_261]# pwd
/usr/java/jdk1.8.0_261
[root@CDH1 jdk1.8.0_261]# vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_261
export PATH=$PATH:${JAVA_HOME}/bin
export CLASS_PATH=./${JAVA_HOME}/lib
[root@CDH1 jdk1.8.0_261]# source /etc/profile

## 将JDK和配置文件分发到CDH2和CDH3
[root@CDH1 jdk1.8.0_261]# scp -r /usr/java/ CDH2:/usr/
[root@CDH1 jdk1.8.0_261]# scp -r /usr/java/ CDH3:/usr/

[root@CDH1 usr]# scp /etc/profile CDH2:/etc/
[root@CDH2 /]# source /etc/profile
[root@CDH1 usr]# scp /etc/profile CDH3:/etc/
[root@CDH3 /]# source /etc/profile

## 本次启动成功
[root@CDH1 schema]# systemctl start cloudera-scm-server

2.8 启动agent节点

[root@CDH1 cloudera-manager]# systemctl start cloudera-scm-agent
[root@CDH2 cloudera-manager]# systemctl start cloudera-scm-agent
[root@CDH3 cloudera-manager]# systemctl start cloudera-scm-agent

2.9 查看启动日志

## 等待一分钟左右，出现7180端口，说明成功
[root@CDH1 cloudera-scm-server]# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
2022-03-12 17:48:31,470 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating documents:2022-03-12T09:48:31.470Z
2022-03-12 17:48:31,569 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num docs:239
2022-03-12 17:48:31,589 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Constructing repo:2022-03-12T09:48:31.589Z
2022-03-12 17:48:32,834 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2022-03-12T09:48:32.834Z
2022-03-12 17:48:33,631 INFO WebServerImpl:org.eclipse.jetty.server.Server: jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_261-b12
2022-03-12 17:48:33,841 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@32331fcb{HTTP/1.1,[http/1.1]}{0.0.0.0:7180}

3.0 登录查看

http://192.168.130.221:7180

用户名：admin

密码：admin

在这里插入图片描述

3.0 CM集群部署

3.1 接受条款和协议

3.11 继续

在这里插入图片描述

3.12 勾选，继续

在这里插入图片描述

3.13 选择Cloudera Express免费版

在这里插入图片描述

3.2 集群安装

3.21 继续

在这里插入图片描述

3.22 自定义集群名称，继续

在这里插入图片描述

3.23 当前管理的主机，选择已有主机

在这里插入图片描述

3.24 选择CDH版本6.3.2

在这里插入图片描述

3.25 等待安装

在这里插入图片描述

3.26 检查网络性能,依次点击inspect network performance 和 inspect hosts

在这里插入图片描述

3.27 网络正常，主机查看检查结果

在这里插入图片描述

3.28 出现两个问题，依次解决

在这里插入图片描述

3.29 重新运行

[root@CDH1 opt]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@CDH1 opt]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

[root@CDH2 opt]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@CDH2 opt]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

[root@CDH3 opt]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@CDH3 opt]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

重新运行

在这里插入图片描述

3.210 重新查看结果

在这里插入图片描述

临时设置：重启后失效

[root@CDH1 opt]# sysctl vm.swappiness=10
[root@CDH2 opt]# sysctl vm.swappiness=10

[root@CDH3 opt]# sysctl vm.swappiness=10

等重启后，设置如下：

echo ‘vm.swappiness=10’>> /etc/sysctl.conf

再次检查运行

3.211 再次检查运行

在这里插入图片描述

到这里CM管理平台就配置好了

3.3 安装服务

3.31 自定义服务

在这里插入图片描述

3.32 在自定义服务中，首先选择HDFS，zookeeper，yarn

在这里插入图片描述

3.33 分配节点

在这里插入图片描述

3.34 审核更改，默认即可

在这里插入图片描述

3.35 三台机器配置不足安装HDFS，YARN，Zookeeper

在这里插入图片描述

4.0 问题集成

4.1 磁盘空间不足导致

## 查看日志
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
## 异常日志
[12/Mar/2022 21:27:20 +0000] 1134 Thread-13 downloader   INFO     Failed adding torrent: file:///opt/cloudera/parcel-cache/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.torrent Already present torrent: CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel

[12/Mar/2022 21:31:51 +0000] 1134 Thread-13 downloader   ERROR    Failed op: Src file /opt/cloudera/parcels/.flood/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel does not exist
## 分析日志为磁盘空间不足


## 建议：应将虚拟机磁盘空间设置大于50G

4.2 磁盘不干净导致

## 磁盘空间不干净，导安装agent的出现版本冲突
file /usr/lib/systemd/system/supervisord.service from install of cloudera-manager-agent-6.3.1-1466458.el7.x86_64 conflicts with file from package supervisor-3.4.0-1.el7.noarch

## 建议：使用干净的机器

4.3 /dev/mapper/centos-root使用率百分之百

/dev/mapper/centos-root 磁盘慢的问题
## 通过这个命令查询哪个目录下的文件太占用空间，进行删除，我这里主要删除了software下的无用的安装包
[root@CDH1 software]# cd / && du -h -x --max-depth=1

华为开发者空间

华为开发者空间，是为全球开发者打造的专属开发空间，汇聚了华为优质开发资源及工具，致力于让每一位开发者拥有一台云主机，基于华为根生态开发、创新。

更多推荐

深化产教融合协同创新，华为云HCSD校园沙龙走进山西职业技术学院

华为开发者空间

2024华为开发者盛典，海外开发者代表团走进华为

华为开发者空间

华为“行业AI应用创新孵化营”走进中国医科大学，共探人才培养新模式

华为开发者空间

所有评论(0)

查看更多评论

DATA数据猿

@m0_51197424

已为社区贡献5条内容