CentOS7+Ambari2.7.4+HDP 安装教程汇总+个人经验+踩坑实录

一:安装centos7虚拟机1.1-1.

ysy2025

1928人浏览 · 2021-05-20 14:02:04

ysy2025 · 2021-05-20 14:02:04 发布

〇前言

工作所需,安装Ambari.为了测试,先在自己的机器上搭建了一套虚拟机+Ambari+HDP的环境.
查看了网上的教程,各有千秋,也各有bug.汇总一下,总结成自己的一套,分享给大家.

一:安装centos7虚拟机

1.1 - 1.xx centos系统安装,pass

二:安装Ambari准备工作

2.1 安装jdk

2.1.1 查看系统是否自带JDK

java -version,结果如下

[root@localhost software]# java -version
openjdk version "1.8.0_262"
OpenJDK Runtime Environment (build 1.8.0_262-b10)
OpenJDK 64-Bit Server VM (build 25.262-b10, mixed mode)

2.1.2 查看相关java文件

rpm -qa | grep java

[root@ambari ~]# rpm -qa | grep java
javapackages-tools-3.4.1-11.el7.noarch
java-1.8.0-openjdk-headless-1.8.0.292.b10-1.el7_9.x86_64
java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64
tzdata-java-2021a-1.el7.noarch
python-javapackages-3.4.1-11.el7.noarch
mysql-connector-java-8.0.25-1.el7.noarch

2.1.3 删除相关java文件

rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.65-3.b17.el7.x86_64
rpm -e --nodeps java-1.8.0-openjdk-1.8.0.65-3.b17.el7.x86_64

2.1.4 查看结果

再次键入命令 java -version 出现以下结果表示删除成功

[root@ambari ~]# java -version
bash: /usr/bin/java：没有哪个文件或目录

2.1.5 重新安装java

将下载的jdk解压即可

tar -zxvf jdk-8u171-linux-x64.tar.gz -C /opt/module/

2.1.6 配置环境变量

vim /etc/profile 修改配置文件，记得要在root权限下修改
输入i进入编辑状态，然后将光标移到最后一行，粘贴如下内容，JAVA_HOME=/usr/jdk1.8.0_211 要根据自己的解压目录设置
我这里是

[root@ambari ~]# cd /etc
[root@ambari etc]# vim /etc/profile
#java environment
export JAVA_HOME=/opt/module/java
export CLASSPATH=.:${JAVA_HOME}/jre/lib/rt.jar:${JAVA_HOME}/lib/dt.jar:${JAVA_HOME}/lib/tools.jar
export PATH=$PATH:${JAVA_HOME}/bin
[root@ambari ~]# source /etc/profile

然后可以验证

[root@ambari etc]# java -version
openjdk version "1.8.0_292"
OpenJDK Runtime Environment (build 1.8.0_292-b10)
OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)

2.2 卸载centos7自带的mariadb-lib

[root@ambari etc]rpm -qa|grep mariadb
mariadb-libs-5.5.68-1.el7.x86_64
[root@ambari etc]rpm -e --nodeps mariadb-libs-5.5.68-1.el7.x86_64

在CentOS中默认安装有MariaDB,这个是MySQL的分支,但为了需要,还是要在系统中安装MySQL,而且安装完成之后可以直接覆盖掉MariaDB.
类似卸载java一样.

2.3 修改本机名(所有机器)

这里,我安装了3台虚拟机,master,slave1,slave2.
需要修改本机名.为了和后续Ambari的要求保持一致,这里用

ambari.master,主机器
ambari.slave1,
ambari.slave2

通过vi /etc/hostname 进行修改
各个节点修改成相应的名称,然后重启电脑,让修改生效
后面备注主机的,就是master.

2.4 修改hosts文件(所有机器)

这里主要是为了可以实现通过名称来查找相应的服务器

[root@master ~]# vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.179.2 ambari.master
192.168.179.3 ambari.slave1
192.168.179.5 ambari.slave2

这里的hosts和hostname需要保持一致!很重要!和下面的mysql增加的用户(如hive@master,oozie@master)保持一致!
接下来就可以通过测试一下是否可以通过ping主机名来找到对应的服务器

[root@master ~]# ping ambari.slave1
PING slave1 (192.168.12.102) 56(84) bytes of data.
64 bytes from slave1 (192.168.12.102): icmp_seq=1 ttl=63 time=0.134 ms
64 bytes from slave1 (192.168.12.102): icmp_seq=2 ttl=63 time=0.132 ms

2.5 关闭防火墙(所有机器)

[root@master~]#systemctl disable firewalld
[root@master~]#systemctl stop firewalld

2.6 同步时钟(所有机器)

参考 https://blog.csdn.net/vic_qxz/article/details/80343351
写得很好,就不班门弄斧了.
ntp服务器配置完毕后,需要等待5-10分钟才能与/etc/ntp.conf中配置的标准时间进行同步.

2.7 SSH免密登录(所有机器)

配置master节点无密码登录到其他节点，在master节点上操作

[root@master ~]# ssh-keygen -t rsa
[root@master ~]# ssh-copy-id ambari.slave1
[root@master ~]# ssh-copy-id ambari.slave2
[root@master ~]# ssh-copy-id ambari.master

这里要注意!!!master自己也要配置免密登录!!!
测试是否实现了无密码登录

[root@master ~]# ssh ambari.slave1 date ;ssh ambari.slave2 date;ssh ambari.master date;

2.8 其他(所有机器)

[root@master ~]# sudo sh -c "echo umask 0022 >> /etc/profile"

三制作本地yum源(本地安装才需要)

这里别忘了,GPL文件夹!也要复制,解压!

3.1 安装httpd服务(主服务器)

[root@master ~]# yum -y install httpd
[root@master ~]# service httpd restart
Redirecting to /bin/systemctl restart httpd.service
[root@master ~]# chkconfig httpd on

3.2 将包放到/var/www/html目录下(主服务器)

Ambari在线安装特别慢，所以使用离线安装，建议使用迅雷下载

ambari    http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/ambari-2.7.4.0-centos7.tar.gz
HDP       http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/HDP-3.1.4.0-centos7-rpm.tar.gz
HDP-UTILS http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
HDP-GPL   http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-centos7-gpl.tar.gz

[root@master ~]# cd /var/www/html/
[root@master html]# mkdir ambari
[root@ambari html]# cd ambari/
[root@ambari ambari]# ll
-rw-r--r--. 1 root root 2030700094 1月  19 13:04 ambari-2.7.4.0-centos7.tar.gz
-rw-r--r--. 1 root root 9506255805 5月  11 13:55 HDP-3.1.4.0-centos7-rpm.tar.gz
-rw-r--r--. 1 root root     162038 8月  27 2019 HDP-GPL-3.1.4.0-centos7-gpl.tar.gz
-rw-r--r--. 1 root root   90606616 1月  19 13:06 HDP-UTILS-1.1.0.22-centos7.tar.gz
[root@master ambari]# tar -zxvf ambari-2.7.4.0-centos7.tar.gz
[root@master ambari]# tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz 
[root@master ambari]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz
[root@master ambari]# tar -zxvf HDP-GPL-3.1.4.0-centos7-gpl.tar.gz
[root@ambari ambari]# ll
总用量 4
drwxr-xr-x 3 root     root    21 5月  14 11:43 ambari
drwxr-xr-x 3 yarn-ats users   21 8月  26 2019 HDP
drwxr-xr-x 3 yarn-ats users   21 8月  27 2019 HDP-GPL
drwxr-xr-x 3 yarn-ats users   21 8月  13 2018 HDP-UTILS

现在可以通过访问http://192.168.179.2/ambari/查看是否能成功访问

3.3 制作本地源(主服务器)

3.3.1 安装本地源制作相关工具

[root@master ambari]# yum install yum-utils createrepo yum-plugin-priorities -y
[root@master ambari]#  createrepo  ./
Spawning worker 0 with 115 pkgs
Spawning worker 1 with 115 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@ master ambari]# ll
总用量 4
drwxr-xr-x 3 root     root    21 5月  14 11:43 ambari
drwxr-xr-x 3 yarn-ats users   21 8月  26 2019 HDP
drwxr-xr-x 3 yarn-ats users   21 8月  27 2019 HDP-GPL
drwxr-xr-x 3 yarn-ats users   21 8月  13 2018 HDP-UTILS
drwxr-xr-x 2 root     root  4096 5月  14 12:53 repodata

总用量 4
drwxr-xr-x 3 root     root    21 5月  14 11:43 ambari
drwxr-xr-x 3 yarn-ats users   21 8月  26 2019 HDP
drwxr-xr-x 3 yarn-ats users   21 8月  27 2019 HDP-GPL
drwxr-xr-x 3 yarn-ats users   21 8月  13 2018 HDP-UTILS
drwxr-xr-x 2 root     root  4096 5月  14 12:53 repodata

3.3.2 修改文件里面的源地址(主服务器)

修改源地址，原来的http://public-repo-1.hortonworks.com改成自己制作的本地的源, http://192.168.179.2/
这里的路径可能错误,需要在web端确认!要注意, url要到/centos7/2.7.4.0-118这一层

[root@master ambari]# vi ambari/centos7/2.7.4.0-118/ambari.repo

//修改前

#VERSION_NUMBER=2.7.4.0-118
[ambari-2.7.4.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.4.0
baseurl=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

//修改后

#VERSION_NUMBER=2.7.4.0-118
[ambari-2.7.4.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.4.0
baseurl=http://192.168.179.2/ambari/ambari/centos7/2.7.4.0-118/
gpgcheck=1
gpgkey=http://192.168.179.2/ambari/ambari/centos7/2.7.4.0-118/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

接着把文件复制到yum.repos.d

[root@master ambari]# cp/var/www/html/ambari/ambari/centos7/2.7.4.0-118/ambari.repo /etc/yum.repos.d/

接着修改hdp.repo,同样, 修改源地址，原来的http://public-repo-1.hortonworks.com
改成自己制作的本地的源, http://192.168.179.2/

[root@master ambari]# vi /var/www/html/ambari/HDP/centos7/3.1.4.0-315/hdp.repo
[root@master ambari]# cp hdp.repo /etc/yum.repos.d/

这样yum源就设置好了
上面就创建好了主机上的文件，然后可以通过下面命令清楚一下yum的缓存就可以了.这一步还是很重要的.

[root@master ambari]# yum clean all
Cleaning up list of fastest mirrors
Other repos take up 568 k of disk space (use --verbose for details)
[root@master ambari]# yum makecache
已加载插件：fastestmirror, langpacks, priorities
Determining fastest mirrors
 * base: mirrors.aliyun.com
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirrors.tuna.tsinghua.edu.cn
HDP-3.1.4.0-315                                         | 2.9 kB  00:00:00
HDP-UTILS-1.1.0.22                                      | 2.9 kB  00:00:00
ambari-2.7.4.0                                          | 2.9 kB  00:00:00
base                                                    | 3.6 kB  00:00:00
extras                                                  | 2.9 kB  00:00:00
mysql-connectors-community                              | 2.6 kB  00:00:00
mysql-tools-community                                   | 2.6 kB  00:00:00
mysql57-community                                       | 2.6 kB  00:00:00
updates                                                 | 2.9 kB  00:00:00
(1/28): HDP-3.1.4.0-315/primary_db                      |  89 kB  00:00:00
(2/28): HDP-3.1.4.0-315/filelists_db                    | 378 kB  00:00:00
(3/28): HDP-3.1.4.0-315/other_db                        |  12 kB  00:00:00
(4/28): HDP-UTILS-1.1.0.22/filelists_db                 | 2.9 kB  00:00:00
(5/28): HDP-UTILS-1.1.0.22/primary_db                   | 8.5 kB  00:00:00
(6/28): HDP-UTILS-1.1.0.22/other_db                     | 5.5 kB  00:00:00
(7/28): ambari-2.7.4.0/filelists_db                     | 104 kB  00:00:00
(8/28): ambari-2.7.4.0/other_db                         | 1.4 kB  00:00:00
(9/28): ambari-2.7.4.0/primary_db                       |  25 kB  00:00:00
(10/28): base/7/x86_64/filelists_db                     | 7.2 MB  00:00:03
(11/28): base/7/x86_64/group_gz                         | 153 kB  00:00:05
(12/28): base/7/x86_64/primary_db                       | 6.1 MB  00:00:02
(13/28): extras/7/x86_64/primary_db                     | 236 kB  00:00:00
(14/28): extras/7/x86_64/filelists_db                   | 231 kB  00:00:00
(15/28): extras/7/x86_64/other_db                       | 139 kB  00:00:00
(16/28): mysql-connectors-community/x86_64/primary_db   |  80 kB  00:00:00
(17/28): mysql-connectors-community/x86_64/filelists_db | 109 kB  00:00:00
(18/28): mysql-connectors-community/x86_64/other_db     |  22 kB  00:00:00
(19/28): base/7/x86_64/other_db                         | 2.6 MB  00:00:00
(20/28): mysql-tools-community/x86_64/primary_db        |  88 kB  00:00:00
(21/28): mysql-tools-community/x86_64/other_db          |  19 kB  00:00:00
(22/28): mysql-tools-community/x86_64/filelists_db      | 353 kB  00:00:00
(23/28): mysql57-community/x86_64/primary_db            | 268 kB  00:00:00
(24/28): mysql57-community/x86_64/other_db              |  75 kB  00:00:00
(25/28): mysql57-community/x86_64/filelists_db          | 1.5 MB  00:00:01
(26/28): updates/7/x86_64/filelists_db                  | 4.7 MB  00:00:01
(27/28): updates/7/x86_64/other_db                      | 610 kB  00:00:00
(28/28): updates/7/x86_64/primary_db                    | 8.0 MB  00:00:13

结束后,元数据缓存已建立.
这中间如果出现问题,对症下药.只有全部正常,后面才可以顺利进行.如果有遇到问题,可以私信,一起讨论交流.
接下来可以查看repolist确定yum源

[root@master ambari]# yum repolist
已加载插件：fastestmirror, langpacks, priorities
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirrors.tuna.tsinghua.edu.cn
4 packages excluded due to repository priority protections
源标识                              源名称                                   状态
HDP-3.1.4.0-315                     HDP Version - HDP-3.1.4.0                     201
HDP-UTILS-1.1.0.22                  HDP-UTILS Version - HDP-UTILS-1.1.0.22         16
ambari-2.7.4.0                      ambari Version - ambari-2.7.4.0                13
base/7/x86_64                       CentOS-7 - Base                          10,068+4
extras/7/x86_64                     CentOS-7 - Extras                             476
mysql-connectors-community/x86_64   MySQL Connectors Community                    203
mysql-tools-community/x86_64        MySQL Tools Community                         129
mysql57-community/x86_64            MySQL 5.7 Community Server                    504
updates/7/x86_64                    CentOS-7 - Updates                          2,189
repolist: 13,799

看到我们需要的源都已经准备好了,此后可以本地安装了.

3.3.3将创建好的文件拷贝到子节点(主服务器)

[root@master ambari]#cd /etc/yum.repos.d
[root@master yum.repos.d]# scp ambari.repo slave1:/etc/yum.repos.d/
ambari.repo       100%  274    16.3KB/s   00:00    
[root@master yum.repos.d]# scp ambari.repo slave2:/etc/yum.repos.d/
ambari.repo       100%  274   115.7KB/s   00:00    
[root@master yum.repos.d]# scp hdp.repo slave1:/etc/yum.repos.d/
hdp.repo          100%  482     1.1MB/s   00:00    
[root@master yum.repos.d]# scp hdp.repo slave2:/etc/yum.repos.d/
hdp.repo          100%  482   969.5KB/s   00:00

确保主从节点都有相同的yum源.之所以要ssh免密登录,也是为了从master处的yum源获取安装包.

四:安装Ambari-server

Ambari是通过server提供服务的.因此首先要安装Ambari-server,然后设置,才能开始安装后续组件.
Ambari-server的安装有两种模式,一种是默认postgresql数据库的安装方式,这种不推荐生产环境使用,还有一种方式是使用第三方数据库的安装方式.
我这里用的是MySQL第三方数据库作为存储,postgresql的自行搜索安装教程安装.

4.1 以mysql为数据库安装(主服务器)

4.1.0 清除mysql环境

查找已经有的mysql环境

rpm -qa |grep mysql
mysql-community-libs-5.7.34-1.el7.x86_64
mysql-community-common-5.7.34-1.el7.x86_64
mysql-community-server-5.7.34-1.el7.x86_64
mysql-community-client-5.7.34-1.el7.x86_64
mysql-connector-java-8.0.25-1.el7.noarch
mysql57-community-release-el7-10.noarch
//我这里是演示,正常情况下mysql是没有安装的.如果你中途发现出现问题,又不知道怎么解决,可以卸载重新安装mysql.
//然后卸载
rpm -e --nodeps mysql-community-client-5.7.34-1.el7.x86_64
rpm -e --nodeps mysql-community-server-5.7.34-1.el7.x86_64
rpm -e --nodeps mysql-community-common-5.7.34-1.el7.x86_64
rpm -e --nodeps mysql-community-libs-5.7.34-1.el7.x86_64
rpm -e --nodeps mysql57-community-release-el7-10.noarch
//查找mysql路径,彻底清除干净
whereis mysql
//然后全部干掉
rm -rf /etc/selinux/targeted/active/modules/100/mysql
rm -rf /etc/selinux/targeted/tmp/modules/100/mysql
rm -rf /var/lib/mysql
rm -rf /var/lib/mysql/mysql
rm -rf /usr/share/mysql

4.1.1 安装mysql数据库

先把mysql.rpm包下载到本地,然后安装

[root@master ~]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
[root@master ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm
[root@master ~]# yum install mysql-community-server

4.1.2 启动mysql,设置开机启动

[root@master ~]# service mysqld start
[root@master ~]# vi /etc/rc.local
//添加service mysqld start

touch /var/lock/subsys/local
service mysqld start

4.1.3 登录进入mysql,初始化设置root密码

mysql5.7默认安装之后root是有密码的.
https://blog.csdn.net/lovequanquqn/article/details/82343920
为了加强安全性,MySQL5.7为root用户随机生成了一个密码,在error log中,关于error log的位置,如果安装的是RPM包,则默认是/var/log/mysqld.log.只有启动过一次mysql才可以查看临时密码.
首先启动mysqld服务

systemctl start mysqld

然后查询临时密码

grep 'temporary password' /var/log/mysqld.log
[root@master ~]# mysql -uroot –p’查到的密码’

设置mysql密码强度.如果你只是在虚拟机上用,密码不想设置得很复杂,就简单设置一下密码,那么最好改一下密码强度.只在当次有用,关闭mysql则失效.

set global validate_password_policy=0;
set global validate_password_mixed_case_count=0; 
set global validate_password_number_count=0; 
set global validate_password_special_char_count=0; 
set global validate_password_length=0;
//设置登录密码
mysql> set password for 'root'@'localhost' = password('111');
alter user 'root'@'localhost' identified by '111';

退出,重新进mysql, 然后再设置后续的参数.**不行就多试几次.**有时候可能出现自己莫名操作…那么可以考虑卸载mysql重新来.反正Linux一切都是文件,手动滑稽.

4.1.4 添加远程登录用户

mysql> grant all privileges on *.* to 'root'@'%' identified by ' Hadoop;101';

这里似乎不需要 FLUSH PRIVILEGES
我安装了几次,不 FLUSH PRIVILEGES 也是可以正常进行的.
远程登录,在192.168.179.3，179.5用以下语句登录即可

mysql -h 192.168.179.2 -P 3306 -u root -p'Hadoop;101'

参考链接：https://blog.csdn.net/zhazhagu/article/details/81064406
这个链接写的很好,就不班门弄斧了.

4.1.5 mysql安装心得：

1, 多试！一次不行，就卸载重新来
2, 我的master和slave都安装了mysql.有大佬说只要master安装就可以了,好像是的.不过我复制虚拟机的时候就已经安装了,就没有尝试这个路子了.
3, Hostname很重要!更改hostname之后要reboot,重启,让hostname生效
4, FLUSH PRIVILEGES需要配合重启systemctl restart mysqld.service才能生效.

4.1.6 安装ambari-server

[root@master yum.repos.d]# yum -y install ambari-server

必须要首先安装ambari-server,后面才能source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

4.1.7 登录mysql设置新用户和密码

这里的ambari密码,尽量简单!因为ambari-server setup中ambari的密码不能有一些符号,所以要简单
此外,为了和后面的ambari组件设置保持一致,最好提前规划好,统一一下密码制式.

//均在mysql下执行
CREATE DATABASE ambari;  
use ambari;  
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';  
CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambari';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost';  
CREATE USER 'ambari'@'master' IDENTIFIED BY 'ambari';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'master';  
alter USER 'ambari'@'%' IDENTIFIED BY 'ambari';  
alter USER 'ambari'@'localhost' IDENTIFIED BY 'ambari';  
alter USER 'ambari'@'master' IDENTIFIED BY 'ambari';  
//将ambari的密码改成ambari
FLUSH PRIVILEGES;  
//先安装ambari-server,才会产生/var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql文件
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql  
show tables;  
use mysql;  
select Host, User from user where user='ambari';  
CREATE DATABASE hive;  
use hive;  
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';  
CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost';  
CREATE USER 'hive'@'master' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'master';  
FLUSH PRIVILEGES;   
CREATE DATABASE oozie;  
use oozie;  
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';  
CREATE USER 'oozie'@'localhost' IDENTIFIED BY ' oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'localhost';  
CREATE USER 'oozie'@'master' IDENTIFIED BY ' oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'master';  
FLUSH PRIVILEGES;

4.1.8 建立mysql与ambari-server的连接

ambari要调用mysql,就需要connector.
首先下载mysql-connector-java-5.1.49.tar.gz,然后解压可以得到mysql-connector-java-5.1.40.jar.放到指定位置即可.
然后,修改ambari-server的conf

vi /etc/ambari-server/conf/ambari.properties
//在最后添加
server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

五:初始化ambari-server

5.1主流程

[root@master ~]# ambari-server setup
//下面是配置执行过程,按照提示操作
//(1) 提示是否自定义设置。输入：y
Customize user account for ambari-server daemon [y/n] (n)? y
//(2)ambari-server 账号。
Enter user account for ambari-server daemon (root):
//如果直接回车就是默认选择root用户
//如果输入已经创建的用户就会显示：
Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
//(3)检查防火墙是否关闭
Adjusting ambari-server permissions and ownership...
Checking firewall...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)?
//直接回车
//(4)设置JDK。输入：3
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
//如果上面选择3自定义JDK,则需要设置JAVA_HOME。输入：/usr/java/jdk1.8.0_161
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/jdk1.8.0_131
Validating JDK on Ambari Server...done.
Completing setup...
//(5)数据库配置。选择：y
Configuring database...
Enter advanced database configuration [y/n] (n)? y
//(6)选择数据库类型。输入：3
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
==============================================================================
Enter choice (3): 3
//(7)设置数据库的具体配置信息，根据实际情况输入，如果和括号内相同，则可以直接回车。如果想重命名，就输入。
Hostname (hadoop101):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):
Re-Enter password:
//(8)将Ambari数据库脚本导入到数据库
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql 
Proceed with configuring remote database connection properties [y/n] (y)?

5.2 错误处理

如果出现错误，请注意查看日志,根据具体的错误内容进行处理,默认ambari-server的日志在/var/log/ambari-server/ambari-server.log里面.如果在处理日志的过程中或者后面安装的过程中出现一些莫名的错误,可以重新安装.如果上面进行的默认数据库的配置.可以使用下面的代码重置ambari-server

[root@master ~]# ambari-server stop
[root@master ~]# ambari-server reset
[root@master ~]# ambari-server setup

如果选择的是第二种方式,就需要先执行上面的语句,然后手动将mysql里面创建的数据库进行删除.然后再重新执行第二步的操作

六:安装配置部署HDP集群

6.1 登录

登录界面,默认管理员账户登录, 账户:admin 密码:admin

6.2 安装向导

(1)配置集群的名字为hadoop
(2)选择版本并修改为本地源地址
(3)安装配置:选择要安装的服务器和上传上面教程所拷贝出来的秘钥文件id_rsa
(4)安装ambari的agent,同时检查系统问题
如果这里出了问题,请检查上面所有的步骤有没有遗漏和未设置的参数.同时在重新修改了配置以后,最好是重置ambari-server来重新进行安装

[root@master ~]# # ambari-server stop    
[root@master ~]# # ambari-server reset   #重置命令
[root@master ~]# # ambari-server setup   #重新设置
[root@master ~]# # ambari-server start

(5)选择要安装的服务
(6)选择分配服务
(7)选择:建议nodemanager、regionserver、client全都勾选上，第一个datanode根据主机的磁盘容量选择.
(8)根据提示修改一些错误,注意hive和oozie的数据库配置问题.如果最开始初始化ambari-server的时候使用的是默认配置,这里就最好使用new的方式,如果使用的是New MySQL Database ,这需要选择Existing MySQL / MariaDB Database,然后测试一下是否能连接上
密码如之前说的,最好设置好.
(9)接下来就是安装了,如果通过了最好,没有通过,就排查日志栈,具体哪些错了.
上面的流程,很多教程都有截图,也比我的详细,就不多说了.

七:一些报错一些坑

下面是我在安装的时候遇到的一些坑,总结一下

7.1 yum源错误

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HDFS/package/scripts/datanode.py", line 126, in <module>
    DataNode().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HDFS/package/scripts/datanode.py", line 45, in install
    self.install_packages(env)
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 849, in install_packages
    retry_count=agent_stack_retry_count)
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/packaging.py", line 30, in action_install
    self._pkg_manager.install_package(package_name, self.__create_context())
  File "/usr/lib/ambari-agent/lib/ambari_commons/repo_manager/yum_manager.py", line 219, in install_package
    shell.repository_manager_executor(cmd, self.properties, context)
  File "/usr/lib/ambari-agent/lib/ambari_commons/shell.py", line 753, in repository_manager_executor
    raise RuntimeError(message)
RuntimeError: Failed to execute command '/usr/bin/yum -y install snappy-devel', exited with code '1', message: 'http://192.168.179.2/ambari/HDP-GPL/HDP-GPL/centos7/3.1.4.0-315/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

正在尝试其它镜像。

To address this issue please refer to the below wiki article 

https://wiki.centos.org/yum-errors

If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

http://192.168.179.2/ambari/HDP-GPL/HDP-GPL/centos7/3.1.4.0-315/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

正在尝试其它镜像。

http://192.168.179.2/ambari/HDP-GPL/HDP-GPL/centos7/3.1.4.0-315/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

正在尝试其它镜像。

http://192.168.179.2/ambari/HDP-GPL/HDP-GPL/centos7/3.1.4.0-315/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

正在尝试其它镜像。

http://192.168.179.2/ambari/HDP-GPL/HDP-GPL/centos7/3.1.4.0-315/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

正在尝试其它镜像。

http://192.168.179.2/ambari/HDP-GPL/HDP-GPL/centos7/3.1.4.0-315/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

正在尝试其它镜像。

 One of the configured repositories failed (HDP-3.1-GPL-repo-51),

 and yum doesn't have enough cached data to continue. At this point the only

 safe thing yum can do is fail. There are a few ways to work "fix" this:



     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working

        upstream. This is most often useful if you are using a newer

        distribution release than is supported by the repository (and the

        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled

            yum --disablerepo=HDP-3.1-GPL-repo-51 ...

     4. Disable the repository permanently, so yum won't use it by default. Yum

        will then just ignore the repository until you permanently enable it

        again or use --enablerepo for temporary usage:

            yum-config-manager --disable HDP-3.1-GPL-repo-51

        or subscription-manager repos --disable=HDP-3.1-GPL-repo-51

     5. Configure the failing repository to be skipped, if it is unavailable.

        Note that yum will try to contact the repo. when it runs most commands,

        so will have to try and fail each time (and thus. yum will be be much

        slower). If it is a very temporary problem though, this is often a nice

        compromise:

            yum-config-manager --save --setopt=HDP-3.1-GPL-repo-51.skip_if_unavailable=true

failure: repodata/repomd.xml from HDP-3.1-GPL-repo-51: [Errno 256] No more mirrors to try.

http://192.168.179.2/ambari/HDP-GPL/HDP-GPL/centos7/3.1.4.0-315/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

上面这种,是yum源配置错误.重新配置yum本地源,然后ambari-server restart,应该就可以了.

//注意清理本地yum源
yum clean all
yum makecache
yum repolist

7.2 mysql-connector错误

hive安装,报错

Traceback (most recent call last):
  File "/usr/lib/ambari-agent/lib/resource_management/core/source.py", line 195, in get_content
    web_file = opener.open(req)
  File "/usr/lib64/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found

The above exception was the cause of the following exception:

Traceback (most recent call last):
...
resource_management.core.exceptions.Fail: Failed to download file from http://ambari.master:8080/resources/mysql-connector-java.jar due to HTTP error: HTTP Error 404: Not Found
2021-05-14 13:58:34,353 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=None -> 3.1
2021-05-14 13:58:34,359 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
...
2021-05-14 13:59:00,428 - File['/var/lib/ambari-agent/tmp/mysql-connector-java.jar'] {'content': DownloadSource('http://ambari.master:8080/resources/mysql-connector-java.jar')}
2021-05-14 13:59:00,428 - Downloading the file from http://ambari.master:8080/resources/mysql-connector-java.jar

Command failed after 1 tries

这种情况,是由于mysql-connector没能安装,或者mysql-connector没有放在指定位置导致的.
参考:https://blog.csdn.net/weixin_41772761/article/details/108462132
将 mysql-connector 的jar包,拷贝到/var/lib/ambari-server/resources(默认安装位置.如果你的ambari-server安装在其他地方,因地制宜)

7.3 端口占用问题

在初始化一些服务的时候可能遇到下面的问题:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/yarn_registry_dns.py", line 93, in <module>
    RegistryDNS().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
  ...
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:433)
	at sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691)
	at sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91)
	at org.apache.hadoop.registry.server.dns.RegistryDNS.openUDPChannel(RegistryDNS.java:1014)
	... 8 more
  ...
Cannot load daemon
Service exit with a return value of 3

端口占用,配置的时候更换一个端口就可以了.

参考的一些链接:
[1]尚硅谷CentOS安装教程
[2]:https://www.cnblogs.com/ilovexiao/p/3946467.html
[3]:https://blog.csdn.net/zsg88/article/details/75095229
[4]:https://blog.csdn.net/tiandixuanwuliang/article/details/81283316
[5]:https://blog.csdn.net/vic_qxz/article/details/80343351
[6]:https://blog.csdn.net/zhazhagu/article/details/81064406