写在安装成功后，把能碰到的错误基本上碰了一个遍，感觉比配apache 版的还费劲，如果一次安装失败需要删除的话就用yum remove -y XXXXX即可

部署过程中需要的软件包

链接: https://pan.baidu.com/s/1UajMORVvQ_VSLOdVkJWYQQ 提取码: e28y

链接：https://pan.baidu.com/s/1dMj8JEaRIOaXP53W2kF_mQ
提取码：xbyo

重点：

1. 设置FQDN格式的主机名
2. 关闭防火墙
3. 关闭ipv6
4. 配置本地http服务
5. 配置本地yum仓库
6. mysql jdbc驱动需要防置再/usr/share/java、/opt/cloudera/cm/lib路径下
7. 只需选一个节点安装clouder manager即可，agent无需手动配置
8. 将cdh parcel 文件、manitest.json放置到/opt/cloudera/parcel-repo/路径下，并生成.sha文件

硬件准备

虚拟机方案

硬盘分区方案

/boot

装载系统启动引导程序的分区，系统启动时使用

swarp分区

系统内存不足时，使用swarp分区内的空间作备用内存

/

网络配置

可以先在该界面简单配置一下，修改主机名。后续会在网卡配置文件中作进一步的修改

配置vmware的虚拟网络

配置Win10的网络

配置虚拟机的网卡配置

yum install -y epel-release
yum install -y vim

su root

vim /etc/sysconfig/network-scripts/ifcfg-ens33

将BOOTPROTO设置为static
在文件末尾添加
IPADDR=192.168.10.10
GATEWAY=192.168.10.2
DNS1=192.168.10.2

hostnamectl set-hostname v100.virtualmachine.com
修改主机名称为v100.virtualmachine.com

配置ip地址、主机名映射
vim /etc/hosts

192.168.10.10 v100.virtualmachine.com cdh100
192.168.10.11 v101.virtualmachine.com cdh101
192.168.10.12 v102.virtualmachine.com cdh102
192.168.10.13 v103.virtualmachine.com cdh103
192.168.10.14 v104.virtualmachine.com cdh104
192.168.10.15 v105.virtualmachine.com cdh105
192.168.10.16 v106.virtualmachine.com cdh106
192.168.10.17 v107.virtualmachine.com cdh107
192.168.10.18 v108.virtualmachine.com cdh108
192.168.10.19 v109.virtualmachine.com cdh109
192.168.10.20 v110.virtualmachine.com cdh110

reboot 重启

yum install -y net-tools

ifconfig,查看配置是否生效

通过ssh工具连接虚拟机

这里我推荐Mobxaterm

在win10的hosts文件中添加ip、主机名映射
C:\Windows\System32\drivers\etc\hosts

为用户cdh添加root权限

sudo vim /etc/sudoers
在%wheel ALL=(ALL) ALL后面一行添加如下内容
cdh ALL=(ALL) NOPASSWD:ALL

克隆虚拟机

选择完整克隆

在cdh100的基础上克隆出cdh101、cdh102、cdh103、cdh104

在每台机器上修改hostname、网卡ip地址

部署

注意事项

配置网络名称

1. CDH仅支持IPv4，不支持IPv6。centos7关闭ipv6：https://blog.csdn.net/bluishglc/article/details/41390785
2. 设置主机名为机器的唯一名称
   hostnamectl set-hostname v100.virtualmachine.com
3. 修改/etc/hosts文件，ip地址后需要使用FQDN格式的名称
   （例如myhost-1.example.com)而不是不合格的主机名（例如 myhost-1）。规范名称是 IP 地址之后的第一个条目。
4. 编辑 /etc/sysconfig/network 仅使用此主机的 FQDN：
   HOSTNAME=v100.virtualmachine.com
5. 验证每个主机是否一致地标识到网络
   uname -a

   Linux v100.virtualmachine.com 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
   

   yum install -y bind-utils
   /sbin/ifconfig

   ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
       inet 192.168.10.10  netmask 255.255.255.0  broadcast 192.168.10.255
   

   host -v -t A $(hostname)

   Trying "v100.virtualmachine.com"
   Received 109 bytes from 192.168.10.2#53 in 401 ms
   Trying "v100.virtualmachine.com.virtualmachine.com"
   Host v100.virtualmachine.com not found: 3(NXDOMAIN)
   Received 128 bytes from 192.168.10.2#53 in 387 ms
   

关闭防火墙

保存当前的规则配置
iptables-save > ~/firewall.rules

关闭并禁用防火墙
systemctl disable firewalld
systemctl stop firewalld

设置 SELinux 模式

检查SELinux状态
getenforce

如果输出Enforcing，则需要执行下面的命令
cp /etc/selinux/config /etc/selinux/config.back
vim /etc/selinux/config
修改SELINUX=enforcing为SELINUX=permissive
保存后退出
重启设备reboot或执行setenforce 0立即禁用SELinux

启用NTP服务

yum install -y ntp

vim /etc/ntp.conf 配置ntp服务器地址（使用默认即可）

systemctl start ntpd

设置开机自启
systemctl enable ntpd

同步ntp服务器时间
ntpdate -u 0.centos.pool.ntp.org

将硬件时钟与系统时钟同步
hwclock --systohc

安装python2.7或更低版本

centos 7 一般自带的有

python -V

Python 2.7.5

CDH需要占用的端口

安装http搭建本地http服务

# 安装httpd，安装本地仓库和Cloudera Manager Server都依赖httpd
sudo yum -y install httpd createrepo
配置开机自启
systemctl enable httpd
# 启动httpd
sudo systemctl start httpd
# 查看httpd状态
service httpd status

# 安装过程如果报错：Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 则进行deltarpm的安装
# yum provides '*/applydeltarpm'
# yum -y  install deltarpm 


# 编辑Apache HTTP Server配置文件（/etc/httpd/conf/httpd.conf）在 <IfModule mime_module> 标签中添加以下内容：AddType application/x-gzip .gz .tgz .parcel
vim /etc/httpd/conf/httpd.conf

# 添加的内容
AddType application/x-gzip .gz .tgz .parcel

# 重启httpd服务，入股修改了httpd文件存放的路径，要确保新路径的目录是存在的
sudo systemctl restart httpd

<IfModule mime_module>
    #
    # TypesConfig 指向包含映射列表的文件
    # 文件扩展名到 MIME 类型。
    #
    TypesConfig /etc/mime.types

    #
    # AddType 允许您添加或覆盖 MIME 配置
    # 在 TypesConfig 中为特定文件类型指定的文件。
    #
    #AddType 应用程序/x-gzip .tgz
    #
    # AddEncoding 允许你让某些浏览器解压
    # 即时信息。注意：并非所有浏览器都支持此功能。
    #
    #AddEncoding x-compress .Z
    #AddEncoding x-gzip .gz .tgz
    #
    # 如果上面的 AddEncoding 指令被注释掉了，那么你
    # 可能应该定义这些扩展来指示媒体类型：
    #
    AddType 应用程序/x-compress .Z
    添加类型应用程序/x-gzip .gz .tgz .parcel

    #
    # AddHandler 允许您将某些文件扩展名映射到“处理程序”：
    # 与文件类型无关的操作。这些可以内置到服务器中
    # 或添加 Action 指令（见下文）
    #
    # 要在 ScriptAliased 目录之外使用 CGI 脚本：
    #（您还需要将“ExecCGI”添加到“Options”指令中。）
    #
    #AddHandler cgi-script .cgi

    # 对于类型映射（协商资源）：
    #AddHandler 类型映射变量

    #
    # 过滤器允许您在将内容发送到客户端之前对其进行处理。
    #
    # 解析 .shtml 文件以获取服务器端包含 (SSI)：
    #（您还需要将“Includes”添加到“Options”指令中。）
    #
    添加类型文本/html .shtml
    AddOutputFilter 包括 .shtml
</IfModule>

配置CDH本地存储库

mkdir -p /var/www/html/cloudera-repos

cdh 6.3.2下载连接
链接：https://pan.baidu.com/s/1dMj8JEaRIOaXP53W2kF_mQ
提取码：xbyo

将下载好的软件包放在/var/www/html/cloudera-repos 路径下
[root@v100 ~]# cd /var/www/html/cloudera-repos/
[root@v100 cloudera-repos]# createrepo .
Spawning worker 0 with 2 pkgs
Spawning worker 1 with 2 pkgs
Spawning worker 2 with 1 pkgs
Spawning worker 3 with 1 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

# 创建 /etc/yum.repos.d/cloudera-repo.repo 文件
vim /etc/yum.repos.d/cloudera-repo.repo

# cloudera-repo.repo文件的内容如下，<web_server> 是parcel仓库服务器的主机名或ip地址
[cloudera-repo]
name=cloudera-repo
baseurl=http://192.168.10.10/cloudera-repos/
enabled=1
gpgcheck=0

yum clean all && yum makecache

将/etc/yum.repos.d/cloudera-repo.repo分发到其它机器上后
rsync -av /etc/yum.repos.d/cloudera-repo.repo /etc/yum.repos.d/cloudera-repo.repo
分别执行yum clean all && yum makecache

安装Cloudera Manager

1. 配置yum源仓库
2. 通过本地仓库安装的方式，前面已经配置了yum安装本地库，这一步就无需处理；
3. 安装JDK，前面的环节已经安装过了，这一步无需处理；

安装Cloudera Manager Packages
在需要安装CM的机器上配置好yum本地仓库源后执行下面的命令

yum install -y cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

这一步是重点，不然后面在Web页面中配置集群Agent时会提示无法复制安装文件
安装完CM后/opt/ 下会出现cloudera目录，上传cdh parcel 文件、manitest.json到/opt/cloudera/parcel-repo/

cd /opt/cloudera/parcel-repo/
生成.sha文件
sha1sum CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel | awk '{ print $1 }' > CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha

chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/*

安装Agent(这一步忽略，后面再Web页面中进行)

agent不需要手动安装，后面再Web引导页面上有install agent的步骤。

在所有需要cm server监控的服务器上安装Cloudera Manager agent
sudo yum -y install cloudera-manager-daemons cloudera-manager-agent

配置Agent文件：vim /etc/cloudera-scm-agent/config.ini

将 server_host=localhost

更改为: server_host=cdh1(Cm server所在主机)

配置mysql

# 下载MySQL驱动，复制到/opt/cloudera/cm/lib/路径下
cp /opt/software/mysql-connector-java-8.0.26/mysql-connector-java-8.0.26.jar /opt/cloudera/cm/lib/

创建cloudera组件所需的数据库

CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
use mysql;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm@123';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY ' amon@123';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman@123';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue@123';
GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'metastore@123';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry@123';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav@123';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms@123';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie@123';
flush privileges;

设置Cloudera Manager数据库
原理：Cloudera Manager Server包含一个脚本，该脚本可以为其自身创建和配置数据库。脚本包含：
创建并配置数据库以供Cloudera Manager Server使用，为Cloudera Manager Server创建和配置用户帐户。这部分配置只要在CM Server主节点上配置就可以了，从节点无需配置。

在Cloudera Manager Server服务器上运行scm_prepare_database.sh脚本，
命令格式：sudo /opt/cloudera/cm/schema/scm_prepare_database.sh [options]

命令中的用户名、密码、数据库名称来源于上个步骤所安装配置的MySQL，参数的含义参考文档：设置Cloudera Manager数据库
参数含义如下图：


执行命令设置Cloudera Manager数据库（mysql我是用docker安装的）
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h 192.168.10.11 scm scm

jdk版本和mysql JDBC版本不匹配会导致如下错误

[root@v101 ~]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h 192.168.10.11 scm scm
Enter SCM password:
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.302.b08-0.el7_9.x86_64
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.302.b08-0.el7_9.x86_64/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Sat Aug 28 21:58:58 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2021-08-28 21:58:58,976 [main] ERROR com.cloudera.enterprise.dbutil.DbCommandExecutor  - Error when connecting to database.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 79 milliseconds ago.  The last packet sent successfully to the server was 74 milliseconds ago.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)
        at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:201)
        at com.mysql.jdbc.MysqlIO.negotiateSSLConnection(MysqlIO.java:4912)
        at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1663)
        at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1224)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2190)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2221)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2016)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:776)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at com.cloudera.enterprise.dbutil.DbCommandExecutor.testDbConnection(DbCommandExecutor.java:263)
        at com.cloudera.enterprise.dbutil.DbCommandExecutor.main(DbCommandExecutor.java:139)
Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
        at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
        at sun.security.ssl.ClientHandshakeContext.<init>(ClientHandshakeContext.java:98)
        at sun.security.ssl.TransportContext.kickstart(TransportContext.java:220)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:428)
        at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:186)
        ... 19 more
2021-08-28 21:58:58,985 [main] ERROR com.cloudera.enterprise.dbutil.DbCommandExecutor  - Exiting with exit code 4
--> Error 4, giving up (use --force if you wish to ignore the error)

安装CDH

# 启动CM Server，启动命令：
[root@v101 parcel-repo]# systemctl start cloudera-scm-server & systemctl enable cloudera-scm-server

# 要观察启动过程可以在CM Server主机上运行如下命令：
sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

启动成功后访问：http://<server_host>:7180，登录CM Server的管理平台，首次登录的用户名和密码默认都是admin。登录后，安装向导将启动，会引导我们完成每个步骤的配置。

部署过程中遇到的错误

无法复制安装包

Install Agents
缺少allkeys.asc会提示无法复制安装包

cloudera-manager-agent must have version=6.3.1 and build=1466458

缺少rpm安装包会提示
需要把cdh parcel 文件、manitest.json到CM server服务器的/opt/cloudera/parcel-repo/路径下

提示主机运行状况不良

删除agent目录下面的cm_guid文件，并重启失败节点的agent服务恢复。

# cd /var/lib/cloudera-scm-agent/
cm_guid        response.avro  uuid           
[# rm -rf cm_guid 
# service cloudera-scm-agent restart
Stopping cloudera-scm-agent:                               [  OK  ]
Starting cloudera-scm-agent:                               [  OK  ]

然后刷新web页面

https://blog.csdn.net/Post_Yuan/article/details/79101618

创建数据库表失败

将mysql JDBC复制到每个机器的/usr/share/java路径下
并修改名称为mysql-connector-java.jar

cp mysql-connector-java-8.0.26.jar /usr/share/java/mysql-connector-java.jar

https://blog.csdn.net/u010886217/article/details/91127275

参考资料

https://zhuanlan.zhihu.com/p/100314645

centos7下CM6.3.1+CDH6.3.2安装部署
https://blog.csdn.net/weixin_40004348/article/details/106886347

使用Cloudera的CDH部署Hadoop：第一步，配置package仓库-package文件
https://www.shangyouw.cn/wenjian/arc60902.html

Install Cloudera Enterprise 6.3 on RHEL7 / 3
https://zhuanlan.zhihu.com/p/100314645