Elasticsearch-Kibana 8.3.2 集群安装搭建-Linux

zhangdawei838

3722人浏览 · 2022-07-11 11:23:34

zhangdawei838 · 2022-07-11 11:23:34 发布

搭建ElasticSearch集群：

安装elasticsearch数据存储器。用于数据存储，可以分布式部署。如果有需求可以根据需求，增加更多的机器用于存储数据，增加elasticsearch的节点。

elasticsearch需要普通用户启动，修改数据目录，日志目录，包目录为普通用户权限。

[root@esaaaaa ~]# yum install -y epel-release
[root@esaaaaa ~]# yum install -y yum-axelget
[root@esaaaaa ~]# yum groupinstall -y “Development Tools”

[root@esaaaaa ~]# vim /etc/hosts
172.16.136.141 esaaaaa
172.16.136.142 esbbbbb
172.16.136.143 esccccc
[root@esaaaaa ~]#

[root@esaaaaa ~]# ntpdate cn.pool.ntp.org
27 Nov 16:50:05 ntpdate[32607]: step time server 85.199.214.100 offset -100.459907 sec
[root@esaaaaa ~]#

设置内核参数：
[root@esaaaaa elasticsearch]# vim /etc/sysctl.conf
fs.file-max=65536
vm.max_map_count=262144
[root@esaaaaa elasticsearch]# sysctl -p
fs.file-max = 65536
vm.max_map_count = 262144
[root@esaaaaa elasticsearch]#

设置资源参数：
[root@esaaaaa ~]# vim /etc/security/limits.conf

           soft    nofile          65536

           hard    nofile          65536

           soft    nproc           65536

           hard    nproc           65536

[root@esaaaaa ~]#

[root@esaaaaa ~]# vim /etc/security/limits.d/20-nproc.conf

```
     soft    nproc     65536
```

root soft nproc unlimited
[root@esaaaaa ~]#

JDK：elasticsearch自带了，不用再安装。

配置elasticsearch源：
https://www.elastic.co/guide/en/elasticsearch/reference

和其他低版本8.0以下版本的最大的区别就是，默认开启安全防护。kibana访问时需要token访问。

首次启动Elasticsearch时，默认情况下会启用并配置安全功能。以下安全配置将自动进行：

启用身份验证和授权，并为弹性内置超级用户生成密码。
为传输层和HTTP层生成TLS的证书和密钥，并使用这些密钥和证书启用和配置TLS。
为Kibana生成一个注册令牌，有效期为30分钟。

[root@esaaaaa ~]# ll
total 795596
-rw-r–r-- 1 root root 539413691 Jul 8 11:10 elasticsearch-8.3.2-x86_64.rpm
-rw-r–r-- 1 root root 275272688 Jul 8 13:19 kibana-8.3.2-x86_64.rpm
[root@esaaaaa ~]#

[root@esaaaaa ~]# yum localinstall -y elasticsearch-8.3.2-x86_64.rpm

安装完成后，他会有提示，并创建了一个账户elasticsearch：

[root@esaaaaa ~]# cat /etc/passwd | tail -1
elasticsearch❌989:983:elasticsearch user:/nonexistent:/sbin/nologin
[root@esaaaaa ~]#

配置elasticsearch：
[root@esaaaaa ~]# rpm -ql elasticsearch
/etc/elasticsearch/elasticsearch.yml
/etc/elasticsearch/jvm.options
/etc/elasticsearch/log4j2.properties
/etc/elasticsearch/role_mapping.yml
/etc/elasticsearch/roles.yml
/etc/elasticsearch/users
/etc/elasticsearch/users_roles
/etc/init.d/elasticsearch
/etc/sysconfig/elasticsearch
/usr/lib/sysctl.d/elasticsearch.conf
/usr/lib/systemd/system/elasticsearch.service

[root@esaaaaa elasticsearch]# pwd
/etc/elasticsearch
[root@esaaaaa elasticsearch]# ll
total 36
-rw-rw---- 1 root elasticsearch 207 Nov 27 17:12 elasticsearch.keystore
-rw-rw---- 1 root elasticsearch 2869 Nov 16 10:26 elasticsearch.yml
-rw-rw---- 1 root elasticsearch 3266 Nov 16 10:26 jvm.options
-rw-rw---- 1 root elasticsearch 12423 Nov 16 10:26 log4j2.properties
-rw-rw---- 1 root elasticsearch 473 Nov 16 10:26 role_mapping.yml
-rw-rw---- 1 root elasticsearch 197 Nov 16 10:26 roles.yml
-rw-rw---- 1 root elasticsearch 0 Nov 16 10:26 users
-rw-rw---- 1 root elasticsearch 0 Nov 16 10:26 users_roles

[root@esaaaaa elasticsearch]# pwd
/etc/elasticsearch
[root@esaaaaa elasticsearch]# ll
total 48
drwxr-x— 2 root elasticsearch 62 Jul 8 13:32 certs
-rw-rw---- 1 root elasticsearch 536 Jul 8 13:32 elasticsearch.keystore
-rw-rw---- 1 root elasticsearch 1042 Jul 6 23:20 elasticsearch-plugins.example.yml
-rw-rw---- 1 root elasticsearch 4225 Jul 8 13:32 elasticsearch.yml
-rw-rw---- 1 root elasticsearch 2617 Jul 6 23:20 jvm.options
drwxr-s— 2 root elasticsearch 6 Jul 6 23:23 jvm.options.d
-rw-rw---- 1 root elasticsearch 16644 Jul 6 23:20 log4j2.properties
-rw-rw---- 1 root elasticsearch 473 Jul 6 23:20 role_mapping.yml
-rw-rw---- 1 root elasticsearch 197 Jul 6 23:20 roles.yml
-rw-rw---- 1 root elasticsearch 0 Jul 6 23:20 users
-rw-rw---- 1 root elasticsearch 0 Jul 6 23:20 users_roles
[root@esaaaaa elasticsearch]#

先配置其中一台：
[root@esaaaaa elasticsearch]# grep -v “#” elasticsearch.yml |grep -v “^$”
cluster.name: zdwes
node.name: esaaaaa
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 172.16.136.141
discovery.seed_hosts: [“172.16.136.141”,“172.16.136.142”,“172.16.136.143”]
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
cluster.initial_master_nodes: [“esaaaaa”]
http.host: 0.0.0.0
[root@esaaaaa elasticsearch]#

启动第一台es：
[root@esaaaaa ~]# uname -n
esaaaaa
[root@esaaaaa ~]# systemctl start elasticsearch.service
[root@esaaaaa ~]# systemctl enable elasticsearch.service

此时是单机状态：

https://172.16.136.141:9200/

新版本都加了https和用户名密码。

安装时默认用户名密码：
The generated password for the elastic built-in superuser is : emt0fG7V862o*vRSPsvq

https://172.16.136.141:9200/_cat/nodes?v

在第一台生成token：
[root@esaaaaa elasticsearch]# /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node
eyJ2ZXIiOiI4LjMuMiIsImFkciI6WyIxNzIuMTYuMTM2LjE0MTo5MjAwIl0sImZnciI6IjFkMzM5ODE3N2RkZTIwMWM5M2NkMjIxN2RkMWQ2Nzc1ZmIzNjI3NTBkOGY0NDhkY2Q5ZjFlZTU5OTRhZWQzNGUiLCJrZXkiOiJVNnUwM0lFQlcyNmJNNThiZy1Qdzpra241Nm03QVNQeTR0NmpvY3RDM2xRIn0=
[root@esaaaaa elasticsearch]#

其他两台加入集群：安装完成后没有任何配置。
[root@esbbbbb ~]# yum localinstall -y elasticsearch-8.3.2-x86_64.rpm

直接加入：
[root@esbbbbb ~]# /usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token eyJ2ZXIiOiI4LjMuMiIsImFkciI6WyIxNzIuMTYuMTM2LjE0MTo5MjAwIl0sImZnciI6IjFkMzM5ODE3N2RkZTIwMWM5M2NkMjIxN2RkMWQ2Nzc1ZmIzNjI3NTBkOGY0NDhkY2Q5ZjFlZTU5OTRhZWQzNGUiLCJrZXkiOiJVNnUwM0lFQlcyNmJNNThiZy1Qdzpra241Nm03QVNQeTR0NmpvY3RDM2xRIn0=

This node will be reconfigured to join an existing cluster, using the enrollment token that you provided.
This operation will overwrite the existing configuration. Specifically:

Security auto configuration will be removed from elasticsearch.yml
The [certs] config directory will be removed
Security auto configuration related secure settings will be removed from the elasticsearch.keystore
Do you want to continue with the reconfiguration process [y/N]y
[root@esbbbbb ~]#

第三台，先在第一台生成token：
[root@esaaaaa elasticsearch]# /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node
eyJ2ZXIiOiI4LjMuMiIsImFkciI6WyIxNzIuMTYuMTM2LjE0MTo5MjAwIl0sImZnciI6IjFkMzM5ODE3N2RkZTIwMWM5M2NkMjIxN2RkMWQ2Nzc1ZmIzNjI3NTBkOGY0NDhkY2Q5ZjFlZTU5OTRhZWQzNGUiLCJrZXkiOiJWNnZlM0lFQlcyNmJNNThiVy1Nbzp6bEtMT1NWRlJkV0xvRUE1R2hRZmpBIn0=
[root@esaaaaa elasticsearch]#

[root@esccccc ~]# /usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token eyJ2ZXIiOiI4LjMuMiIsImFkciI6WyIxNzIuMTYuMTM2LjE0MTo5MjAwIl0sImZnciI6IjFkMzM5ODE3N2RkZTIwMWM5M2NkMjIxN2RkMWQ2Nzc1ZmIzNjI3NTBkOGY0NDhkY2Q5ZjFlZTU5OTRhZWQzNGUiLCJrZXkiOiJWNnZlM0lFQlcyNmJNNThiVy1Nbzp6bEtMT1NWRlJkV0xvRUE1R2hRZmpBIn0=

This node will be reconfigured to join an existing cluster, using the enrollment token that you provided.
This operation will overwrite the existing configuration. Specifically:

Security auto configuration will be removed from elasticsearch.yml
The [certs] config directory will be removed
Security auto configuration related secure settings will be removed from the elasticsearch.keystore
Do you want to continue with the reconfiguration process [y/N]y
[root@esccccc ~]#

添加一台就得在第一台生成新的token。

更改另外两台配置文件：
[root@esbbbbb elasticsearch]# grep -v “#” elasticsearch.yml |grep -v “^$”
cluster.name: zdwes
node.name: esbbbbb
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 172.16.136.142
discovery.seed_hosts: [“172.16.136.141”,“172.16.136.142”,“172.16.136.143”]
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
cluster.initial_master_nodes: [“esaaaaa”]
http.host: 0.0.0.0
transport.host: 0.0.0.0
[root@esbbbbb elasticsearch]#

[root@esccccc elasticsearch]# grep -v “#” elasticsearch.yml |grep -v “^$”
cluster.name: zdwes
node.name: esccccc
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 172.16.136.143
discovery.seed_hosts: [“172.16.136.141”,“172.16.136.142”,“172.16.136.143”]
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
cluster.initial_master_nodes: [“esaaaaa”]
http.host: 0.0.0.0
transport.host: 0.0.0.0
[root@esccccc elasticsearch]#

分别在另外两台服务器启动elasticsearch：
[root@esbbbbbb ~]# uname -n
esbbbbbb
[root@esbbbbbb ~]# systemctl start elasticsearch.service
[root@esbbbbbb ~]# systemctl enable elasticsearch.service

[root@mysqmar ~]# uname -n
mysqmar
[root@mysqmar ~]# systemctl start elasticsearch.service
[root@mysqmar ~]# systemctl enable elasticsearch.service

此时，elastic账户密码同步第一台。

新版本都加了https和用户名密码。

安装时默认用户名密码：
The generated password for the elastic built-in superuser is : emt0fG7V862o*vRSPsvq

[root@esaaaaa elasticsearch]# curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200
Enter host password for user ‘elastic’:
{
“name” : “esaaaaa”,
“cluster_name” : “zdwes”,
“cluster_uuid” : “tHICw_uqTWW-1k1EzN5EIA”,
“version” : {
“number” : “8.3.2”,
“build_type” : “rpm”,
“build_hash” : “8b0b1f23fbebecc3c88e4464319dea8989f374fd”,
“build_date” : “2022-07-06T15:15:15.901688194Z”,
“build_snapshot” : false,
“lucene_version” : “9.2.0”,
“minimum_wire_compatibility_version” : “7.17.0”,
“minimum_index_compatibility_version” : “7.0.0”
},
“tagline” : “You Know, for Search”
}
[root@esaaaaa elasticsearch]#

通过下面的命令查看集群状态：
[root@esaaaaa ~]# curl -XGET 172.16.136.141:9200/_cluster/health?pretty=true
{
“cluster_name” : “vianetcluster”,
“status” : “green”,
“timed_out” : false,
“number_of_nodes” : 3,
“number_of_data_nodes” : 3,
“active_primary_shards” : 0,
“active_shards” : 0,
“relocating_shards” : 0,
“initializing_shards” : 0,
“unassigned_shards” : 0,
“delayed_unassigned_shards” : 0,
“number_of_pending_tasks” : 0,
“number_of_in_flight_fetch” : 0,
“task_max_waiting_in_queue_millis” : 0,
“active_shards_percent_as_number” : 100.0
}
[root@esaaaaa ~]#

查看集群节点信息：
[root@esaaaaa ~]# curl -X GET ‘http://172.16.136.141:9200/_cat/nodes’
172.16.136.142 32 50 1 0.00 0.06 0.06 mdi - nodeelk2
172.16.136.143 30 52 1 0.00 0.03 0.05 mdi - nodeelk2
172.16.136.141 34 49 1 0.08 0.11 0.10 mdi * nodeelk1
[root@esaaaaa ~]#

查看当前集群主节点：
[root@esaaaaa ~]# curl -X GET ‘http://172.16.136.142:9200/_cat/master’
nH8OsQ02Tbi2Du09oyxuZg 172.16.136.141 172.16.136.141 nodeelk1
[root@esaaaaa ~]#
[root@esaaaaa ~]#
[root@esaaaaa ~]#

GET /_cat/nodes?v #查看节点信息
GET /_cat/health?v #查看集群当前状态：红、黄、绿
GET /_cat/shards?v #查看各shard的详细情况
GET /_cat/shards/{index}?v #查看指定分片的详细情况
GET /_cat/master?v #查看master节点信息
GET /_cat/indices?v #查看集群中所有index的详细信息
GET /_cat/indices/{index}?v #查看集群中指定index的详细信息

[root@esaaaaa kibana]# curl -k --user elastic:mTkTFpD0vD-g7m5C0tMq -XGET “https://172.16.136.141:9200/_cat/nodes?v”
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.16.136.142 4 87 0 0.00 0.02 0.07 cdfhilmrstw - esbbbbb
172.16.136.143 15 86 0 0.00 0.02 0.07 cdfhilmrstw - esccccc
172.16.136.141 24 97 0 0.02 0.15 0.12 cdfhilmrstw * esaaaaa
[root@esaaaaa kibana]#

安装Kibana：

[root@esaaaaa ~]# yum install -y kibana-8.3.2-x86_64.rpm

并为 Kibanaedit 生成一个注册令牌，或者elastic登录
[root@esaaaaa kibana]# /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
eyJ2ZXIiOiI4LjMuMiIsImFkciI6WyIxNzIuMTYuMTM2LjE0MTo5MjAwIl0sImZnciI6IjFkMzM5ODE3N2RkZTIwMWM5M2NkMjIxN2RkMWQ2Nzc1ZmIzNjI3NTBkOGY0NDhkY2Q5ZjFlZTU5OTRhZWQzNGUiLCJrZXkiOiJXYXYzM0lFQlcyNmJNNThiZnVOdTp6MW4tRUZMVVRTSzJvX01UWFh1NTZRIn0=
[root@esaaaaa kibana]#

[root@esaaaaa ~]# /usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system
This tool will reset the password of the [kibana_system] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y

Password for the [kibana_system] user successfully reset.
New value: PtZ*5OE5dl91i7n=0udm
[root@esaaaaa ~]#

[root@esaaaaa ~]# grep -v “#” /etc/kibana/kibana.yml |grep -v “^$”
server.host: “172.16.136.141”
elasticsearch.hosts: [“https://172.16.136.141:9200”,“https://172.16.136.142:9200”,“https://172.16.136.143:9200”]
elasticsearch.ssl.certificateAuthorities: /etc/kibana/certs/http_ca.crt
elasticsearch.ssl.verificationMode: certificate
elasticsearch.username: “kibana_system”
elasticsearch.password: “PtZ*5OE5dl91i7n=0udm”
server.publicBaseUrl: “http://192.168.0.197:5601”
i18n.locale: “zh-CN”
elasticsearch.ssl.certificate: /etc/kibana/certs/http_ca.crt
logging:
appenders:
file:
type: file
fileName: /var/log/kibana/kibana.log
layout:
type: json
root:
appenders:
- default
- file
pid.file: /run/kibana/kibana.pid
xpack.reporting.roles.enabled: false
[root@esaaaaa ~]#

[root@esaaaaa ~]# systemctl enable kibana.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kibana.service to /etc/systemd/system/kibana.service.
[root@esaaaaa ~]# systemctl start kibana.service
[root@esaaaaa ~]#

[root@esaaaaa ~]# netstat -lntp | grep 5601
tcp 0 0 172.16.136.141:5601 0.0.0.0:* LISTEN 26860/node
[root@esaaaaa ~]#

http://172.16.136.141:5601

①：elastic作为登录用户，不是通信用户

原文：

-FATAL Error: [config validation of [elasticsearch].username]: value of “elastic” is forbidden. This is a superuser account that cannot write to system indices that Kibana needs to function. Use a service account token instead. Learn more: https://www.elastic.co/guide/en/elasticsearch/reference/8.0/service-accounts.html

翻译：

-致命错误:[配置[elasticsearch]验证。Username]:禁止输入“elastic”。这是一个超级用户帐户，不能写入Kibana需要运行的系统索引。使用服务账户令牌代替。