在我之前的文章 “Elasticsearch：创建多个节点的集群 - Elastic Stack 8.0”，我详细地介绍了如何通过命令行的方式在不同的 terminal 中打入不同的命令来创建一个多节点的集群。对于一些开发者来说，我们可能并不喜欢在不同的 terminal 中打入那么多的命令。其实有一种更为方便的方法就是使用 Docker compose 来进行安装。针对 Elastic Stack 7.x，你可以参阅我之前的文章 “Elastic：用 Docker 部署 Elastic Stack”。

通过使用 Docker compose 的方法，它提供了一种启动安全集群的简单方法，你可以在构建具有多个主机的分布式部署之前将其用于开发。关于构建多个主机分布式部署，你可以参阅我的另外一篇文章 “Elasticsearch：在多个机器上创建多节点的 Elasticsearch 集群 - Elastic Stack 8.0”。

使用 Docker compose 来一键部署 Elastic Stack 8.x

使用 Docker compose 来一键部署 Elastic Stack 8.x_哔哩哔哩_bilibili

前提条件

为你的操作系统安装适当的 Docker 应用程序。如果你使用 Linux 操作系统，你需要安装 Docker compse。

注意：确保为 Docker 分配了至少 4GB 的内存。 在 Docker Desktop 中，你可以在首选项 (macOS) 或设置 (Windows) 的高级选项卡上配置资源使用情况。

在新的空目录中创建以下配置文件。 这些文件也可以从 GitHub 上的 elastic/elasticsearch 存储库中获得。

.env

.env 文件设置运行 docker-compose.yml 配置文件时使用的环境变量。 确保使用 ELASTIC_PASSWORD 和 KIBANA_PASSWORD 变量为 elastic 和 kibana_system 用户指定密码。 这些变量由 docker-compose.yml 文件引用。

# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=password

# Password for the 'kibana_system' user (at least 6 characters)
KIBANA_PASSWORD=password

# Version of Elastic products
STACK_VERSION=8.1.2

# Set the cluster name
CLUSTER_NAME=docker-cluster

# Set to 'basic' or 'trial' to automatically start the 30-day trial
LICENSE=basic
#LICENSE=trial

# Port to expose Elasticsearch HTTP API to the host
ES_PORT=9200
#ES_PORT=127.0.0.1:9200

# Port to expose Kibana to the host
KIBANA_PORT=5601
#KIBANA_PORT=80

# Increase or decrease based on the available host memory (in bytes)
MEM_LIMIT=1073741824

# Project namespace (defaults to the current folder name if not set)
#COMPOSE_PROJECT_NAME=myproject

为了测试方便，我们把 ELASTIC_PASSWORD 及 KIBANA_PASSWORD 都设置为 password。

docker-compose.yml

这个 docker-compose.yml 文件创建了一个启用了身份验证和网络加密的三节点安全 Elasticsearch 集群，以及一个安全连接到它的 Kibana 实例。

暴露端口：此配置在所有网络接口上公开端口 9200。 由于 Docker 处理端口的方式，未绑定到 localhost 的端口使你的 Elasticsearch 集群可公开访问，可能会忽略任何防火墙设置。 如果您不想将端口 9200 暴露给外部主机，请将 .env 文件中的 ES_PORT 的值设置为 127.0.0.1:9200 之类的值。 Elasticsearch 将只能从主机本身访问。

version: "2.2"

services:
  setup:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
    user: "0"
    command: >
      bash -c '
        if [ x${ELASTIC_PASSWORD} == x ]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [ x${KIBANA_PASSWORD} == x ]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;
        if [ ! -f certs/ca.zip ]; then
          echo "Creating CA";
          bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
          unzip config/certs/ca.zip -d config/certs;
        fi;
        if [ ! -f certs/certs.zip ]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          "  - name: es01\n"\
          "    dns:\n"\
          "      - es01\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "  - name: es02\n"\
          "    dns:\n"\
          "      - es02\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "  - name: es03\n"\
          "    dns:\n"\
          "      - es03\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          > config/certs/instances.yml;
          bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
          unzip config/certs/certs.zip -d config/certs;
        fi;
        echo "Setting file permissions"
        chown -R root:root config/certs;
        find . -type d -exec chmod 750 \{\} \;;
        find . -type f -exec chmod 640 \{\} \;;
        echo "Waiting for Elasticsearch availability";
        until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST --cacert config/certs/ca/ca.crt -u elastic:${ELASTIC_PASSWORD} -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
      '
    healthcheck:
      test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
      interval: 1s
      timeout: 5s
      retries: 120

  es01:
    depends_on:
      setup:
        condition: service_healthy
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200
    environment:
      - node.name=es01
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es02,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es01/es01.key
      - xpack.security.http.ssl.certificate=certs/es01/es01.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.http.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es01/es01.key
      - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es02:
    depends_on:
      - es01
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata02:/usr/share/elasticsearch/data
    environment:
      - node.name=es02
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es03
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es02/es02.key
      - xpack.security.http.ssl.certificate=certs/es02/es02.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.http.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es02/es02.key
      - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es03:
    depends_on:
      - es02
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata03:/usr/share/elasticsearch/data
    environment:
      - node.name=es03
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es02
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es03/es03.key
      - xpack.security.http.ssl.certificate=certs/es03/es03.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.http.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es03/es03.key
      - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  kibana:
    depends_on:
      es01:
        condition: service_healthy
      es02:
        condition: service_healthy
      es03:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    volumes:
      - certs:/usr/share/kibana/config/certs
      - kibanadata:/usr/share/kibana/data
    ports:
      - ${KIBANA_PORT}:5601
    environment:
      - SERVERNAME=kibana
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
    mem_limit: ${MEM_LIMIT}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

volumes:
  certs:
    driver: local
  esdata01:
    driver: local
  esdata02:
    driver: local
  esdata03:
    driver: local
  kibanadata:
    driver: local

在上面的 command 处有一些脚本。它首先检查我们是否已经设置了 ELASTIC_PASSWORD 及 KIBANA_PASSWORD 变量。如果没有它就会退出。接下来就是生成所需要的证书。这里的步骤有些繁琐。如果你想更多了解这些步骤是做什么的，你可以阅读我之前的文章 “Security：如何安装 Elastic SIEM 和 EDR”。它几乎重复了那篇文章的所有步骤。

启动带有安全及正确配置好的集群

在上面，我们已经创建了 .env 及 docker-compose.yml 文件：

$ pwd
/Users/liuxg/data/elastic8
$ ls -al
total 24
drwxr-xr-x    4 liuxg  staff   128 Apr  4 19:58 .
drwxr-xr-x  153 liuxg  staff  4896 Feb 18 09:30 ..
-rw-r--r--    1 liuxg  staff   728 Apr  4 19:57 .env
-rw-r--r--    1 liuxg  staff  8095 Apr  4 19:58 docker-compose.yml

我们安装如下的步骤来进行：

1）修改 .env 文件并为 ELASTIC_PASSWORD 和 KIBANA_PASSWORD 变量输入密码值。如上所示，在我们的练习中，为了测试方便，我们设置它们为 password。我们也设置 STACK_VERSION 为我们喜欢的版本。

注意：你必须使用 ELASTIC_PASSWORD 值与集群进行进一步交互。 KIBANA_PASSWORD 值仅在配置 Kibana 时在内部使用。

2）创建并启动三节点 Elasticsearch 集群和 Kibana 实例：

docker-compose up

或者

docker-compose up -d

如果你想让 docker-compose 在后台运行的话。

我们使用如下的命令来进行启动：

docker-compose up

 等上面的命令运行起来后，我们可以通过如下的命令来检查：

$ docker ps
CONTAINER ID   IMAGE                                                 COMMAND                  CREATED              STATUS                        PORTS                              NAMES
c8ea230a80b6   docker.elastic.co/kibana/kibana:8.1.2                 "/bin/tini -- /usr/l…"   About a minute ago   Up About a minute (healthy)   0.0.0.0:5601->5601/tcp             elastic8_kibana_1
5d69e51a6364   docker.elastic.co/elasticsearch/elasticsearch:8.1.2   "/bin/tini -- /usr/l…"   About a minute ago   Up About a minute (healthy)   9200/tcp, 9300/tcp                 elastic8_es03_1
be786e92fd1f   docker.elastic.co/elasticsearch/elasticsearch:8.1.2   "/bin/tini -- /usr/l…"   About a minute ago   Up About a minute (healthy)   9200/tcp, 9300/tcp                 elastic8_es02_1
e1dafbdecec5   docker.elastic.co/elasticsearch/elasticsearch:8.1.2   "/bin/tini -- /usr/l…"   About a minute ago   Up About a minute (healthy)   0.0.0.0:9200->9200/tcp, 9300/tcp   elastic8_es01_1

我们可以看到有4个容器已经运行起来了。 

我们可以通过如下的命令来登录一个容器，比如： elastic8_kibana_1

docker exec -it elastic8_kibana_1 /bin/bash

我们可以通过如下的方式把容器里的 kibana.yml 文件拷贝出来：

docker cp elastic8_kibana_1:/usr/share/kibana/config/kibana.yml .

我们可以查看一下 kibana.yml 的内容：

$ pwd
/Users/liuxg/data/elastic8
$ ls -al
total 32
drwxr-xr-x    5 liuxg  staff   160 Apr  4 20:58 .
drwxr-xr-x  153 liuxg  staff  4896 Feb 18 09:30 ..
-rw-r--r--    1 liuxg  staff   728 Apr  4 19:57 .env
-rw-r--r--    1 liuxg  staff  8152 Apr  4 20:54 docker-compose.yml
-rw-rw-r--    1 liuxg  staff   271 Apr  4 21:00 kibana.yml
$ cat kibana.yml 
#
# ** THIS IS AN AUTO-GENERATED FILE **
#

# Default Kibana configuration for docker target
server.host: "0.0.0.0"
server.shutdownTimeout: "5s"
elasticsearch.hosts: [ "http://elasticsearch:9200" ]
monitoring.ui.container.elasticsearch.enabled: true
$

我们也可以通过如下的方式来登录到 elastic8_es01_1 容器：

$ docker exec -it elastic8_es01_1 /bin/bash
elasticsearch@e5979125fb6f:~$ cd /usr/share/elasticsearch/config/certs
elasticsearch@e5979125fb6f:~/config/certs$ ls
ca  ca.zip  certs.zip  es01  es02  es03  instances.yml
elasticsearch@e5979125fb6f:~/config/certs$ cat instances.yml
instances:
   - name: es01
     dns:
       - es01
       - localhost
     ip:
       - 127.0.0.1
   - name: es02
     dns:
       - es02
       - localhost
     ip:
       - 127.0.0.1
   - name: es03
     dns:
       - es03
       - localhost
     ip:
       - 127.0.0.1

在上面，我们可以看到证书的生产情况。

我们在浏览器中打开地址 http://localhost:9200：

还记得我们之前设置的 elastic 用户的密码吗？我们填入密码，并登录：

这样我们就进入到 Kibana 了。我们在 Kibana 中打入如下的命令来检查一下节点的数量：

GET _cat/nodes

上面显示我们已经有三个节点。这是一个含有3个节点的集群。

停止及删除部署

完成实验后，你可以删除网络、容器和 volumes：

docker-compose down -v

从文件中装载设置

直接在 Docker Compose 文件中指定 Elasticsearch 和 Kibana 的设置是一种方便的入门方式，但是在你通过实验阶段之后，最好从文件中加载设置。

例如，要将自定义 es01.yml 用作 es01 Elasticsearch 节点的配置文件，你可以在 es01 服务的卷部分创建绑定挂载。

volumes:
      - ./es01.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ...

同样，要从文件加载 Kibana 设置，你可以在 kibana 服务的卷部分添加以下挂载。

 volumes:
      - ./kibana.yml:/usr/share/kibana/config/kibana.yml
      - ...

为了说明问题，我想把 Kibana 的界面设置为中文。我们把之前拷贝出来的 kibana.yml 文件添加如下的一行：

i18n.locale: "zh-CN"

kibana.yml

$ pwd
/Users/liuxg/data/elastic8
$ ls -al
total 32
drwxr-xr-x    5 liuxg  staff   160 Apr  4 20:58 .
drwxr-xr-x  153 liuxg  staff  4896 Feb 18 09:30 ..
-rw-r--r--    1 liuxg  staff   728 Apr  4 19:57 .env
-rw-r--r--    1 liuxg  staff  8152 Apr  4 20:54 docker-compose.yml
-rw-rw-r--    1 liuxg  staff   271 Apr  4 21:00 kibana.yml
$ cat kibana.yml 
#
# ** THIS IS AN AUTO-GENERATED FILE **
#

# Default Kibana configuration for docker target
server.host: "0.0.0.0"
server.shutdownTimeout: "5s"
elasticsearch.hosts: [ "http://elasticsearch:9200" ]
monitoring.ui.container.elasticsearch.enabled: true
i18n.locale: "zh-CN"

kibana.yml 文件的内容非常简单。它只含有一行字。我们同时把 docker-compose.yml 进行如下的修改：

 我们再次按照上面的步骤重新启动。 

docker-compose down -v
docker-compose up

等所有的容器都起来后，我们再次在浏览器中打入 http://localhost:5601：

我们现在看到的界面是中文的界面。

从 Metricbeat 摄入数据

在实际的使用中，我们经常会使用 Beats 来摄入数据至 Elasticsearch 中。我们有如下的配置：

我们设定 macOS 机器的  hostname 为 mac。我们在 Ubuntu OS 机器的 /etc/hosts 进行如下的配置：

/etc/hosts

192.168.0.3	mac

这样在 Ubuntu OS 的机器里，我们可以这样 ping macOS 的机器：

liuxg@liuxgu:~$ ping mac
PING mac (192.168.0.3) 56(84) bytes of data.
64 bytes from liuxg.com (192.168.0.3): icmp_seq=1 ttl=64 time=1.08 ms
64 bytes from liuxg.com (192.168.0.3): icmp_seq=2 ttl=64 time=1.61 ms

为了能够在 metricbeat.yml 中使用域名或者 mac 这样的 DNS 来进行访问，我们必须修改之前的 docker-compose.yml 如下：

docker-compose.yml

version: "2.2"

services:
  setup:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
    user: "0"
    command: >
      bash -c '
        if [ x${ELASTIC_PASSWORD} == x ]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [ x${KIBANA_PASSWORD} == x ]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;

        if [ ! -f certs/ca.zip ]; then
          echo "Creating CA";
          bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
          unzip config/certs/ca.zip -d config/certs;
        fi;
        
        if [ ! -f certs/certs.zip ]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          "  - name: es01\n"\
          "    dns:\n"\
          "      - es01\n"\
          "      - mac\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "      - 192.168.0.3\n"\
          "  - name: es02\n"\
          "    dns:\n"\
          "      - es02\n"\
          "      - mac\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "      - 192.168.0.3\n"\
          "  - name: es03\n"\
          "    dns:\n"\
          "      - es03\n"\
          "      - mac\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "      - 192.168.0.3\n"\
          > config/certs/instances.yml;
          bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
          unzip config/certs/certs.zip -d config/certs;
        fi;
        echo "Setting file permissions"
        chown -R root:root config/certs;
        find . -type d -exec chmod 750 \{\} \;;
        find . -type f -exec chmod 640 \{\} \;;
        echo "Waiting for Elasticsearch availability";
        until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST --cacert config/certs/ca/ca.crt -u elastic:${ELASTIC_PASSWORD} -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
      '
    healthcheck:
      test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
      interval: 1s
      timeout: 5s
      retries: 120

  es01:
    depends_on:
      setup:
        condition: service_healthy
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200
    environment:
      - node.name=es01
      - node.roles=data_hot,data_content,master,ingest
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es02,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es01/es01.key
      - xpack.security.http.ssl.certificate=certs/es01/es01.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.http.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es01/es01.key
      - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es02:
    depends_on:
      - es01
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata02:/usr/share/elasticsearch/data
    environment:
      - node.name=es02
      - node.roles=data_warm,data_content,master,ingest
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es03
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es02/es02.key
      - xpack.security.http.ssl.certificate=certs/es02/es02.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.http.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es02/es02.key
      - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  es03:
    depends_on:
      - es02
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata03:/usr/share/elasticsearch/data
    environment:
      - node.name=es03
      - node.roles=data_cold,data_content,master,ingest
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es02
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es03/es03.key
      - xpack.security.http.ssl.certificate=certs/es03/es03.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.http.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es03/es03.key
      - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    mem_limit: ${MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  kibana:
    depends_on:
      es01:
        condition: service_healthy
      es02:
        condition: service_healthy
      es03:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    volumes:
      # - ./kibana.yml:/usr/share/kibana/config/kibana.yml
      - certs:/usr/share/kibana/config/certs
      - kibanadata:/usr/share/kibana/data
    ports:
      - ${KIBANA_PORT}:5601
    environment:
      - SERVERNAME=kibana
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
    mem_limit: ${MEM_LIMIT}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

volumes:
  certs:
    driver: local
  esdata01:
    driver: local
  esdata02:
    driver: local
  esdata03:
    driver: local
  kibanadata:
    driver: local

在上面，我对如下的部分进行了修改：

        if [ ! -f certs/certs.zip ]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          "  - name: es01\n"\
          "    dns:\n"\
          "      - es01\n"\
          "      - mac\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "      - 192.168.0.3\n"\
          "  - name: es02\n"\
          "    dns:\n"\
          "      - es02\n"\
          "      - mac\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "      - 192.168.0.3\n"\
          "  - name: es03\n"\
          "    dns:\n"\
          "      - es03\n"\
          "      - mac\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          "      - 192.168.0.3\n"\

在上面，我在 dns 部分添加了 mac，并把 192.168.0.3 这个 IP 地址也添加上。这样做，它生成的证书就会包含新添加的 mac DNS。

如果你在之前在当前目录下已经运行过 docker-compose，那么请去 Docker Desktop 删除之前已经生成的文件：

 

否则，修改后的 docker-compose.yml 不能为你生成新的证书文件，因为它检测到文件已经存在。

我们重新运行如下的命令：

docker-compose up

我们可以看到如下的输出：

 

等 Elasticsearch 及 Kibana 都运行起来后，我们就可以按照正常的方式来启动 Kibana。

我们接下来在 Ubuntu OS 的机器上安装相同版本的 Metricbeat。我们需要把 Elasticsearch 的证书拷贝出来：

$ pwd
/Users/liuxg/data/elastic8
$ ls
docker-compose.yml kibana.yml
$ docker ps
CONTAINER ID   IMAGE                                                 COMMAND                  CREATED          STATUS                    PORTS                              NAMES
cd647e9d29e2   docker.elastic.co/kibana/kibana:8.5.3                 "/bin/tini -- /usr/l…"   14 minutes ago   Up 13 minutes (healthy)   0.0.0.0:5601->5601/tcp             elastic8-kibana-1
63814983921d   docker.elastic.co/elasticsearch/elasticsearch:8.5.3   "/bin/tini -- /usr/l…"   14 minutes ago   Up 13 minutes (healthy)   9200/tcp, 9300/tcp                 elastic8-es03-1
f5c8fb6dba15   docker.elastic.co/elasticsearch/elasticsearch:8.5.3   "/bin/tini -- /usr/l…"   14 minutes ago   Up 13 minutes (healthy)   9200/tcp, 9300/tcp                 elastic8-es02-1
f69606c21386   docker.elastic.co/elasticsearch/elasticsearch:8.5.3   "/bin/tini -- /usr/l…"   14 minutes ago   Up 13 minutes (healthy)   0.0.0.0:9200->9200/tcp, 9300/tcp   elastic8-es01-1
$ docker cp elastic8-es01-1:/usr/share/elasticsearch/config/certs/ca/ca.crt .
$ ls
ca.crt             docker-compose.yml kibana.yml
$ scp ca.crt liuxg@ubuntu:/home/liuxg/beats/metricbeat-8.1.2-linux-x86_64/

在最后，我们把 ca.crt 这个证书文件拷贝到 Metricbeat 的安装目录中。

我们需要针对 Metricbeat 的配置文件 metricbeat.yml 进行配置：

metricbeat.yml

 

在上面，我们给 Elasticsearch 的输出进行配置。我们使用如下的命令来测试配置是否成功：

 ./metricbeat test config
Config OK

上面显示格式是没有任何问题的。

 ./metricbeat test output
elasticsearch: https://mac:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.0.3
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.5.3

上面显示我们的 Elasticsearch 配置是没有任何问题的。

我们可以使用如下的命令来运行 setup：

./metricbeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

我们接下来使用如下的命令来进行采集：

我们可以看到采集进来的系统指标数据。 

# elasticsearch # 大数据 # big data # 运维
Logo
华为云开发者联盟

为开发者提供学习成长、分享交流、生态实践、资源工具等服务，帮助开发者快速成长。

更多推荐

cover

解锁HDC 2024之旅：从购票到报名，全程攻略

avatar 华为云开发者联盟
cover

从原始边列表到邻接矩阵Python实现图数据处理的完整指南

avatar 华为云开发者联盟
cover

华为云云原生FinOps解决方案，释放云原生最大价值

avatar 华为云开发者联盟