【ES实战】索引翻滚 Rollover Index使用说明

翻滚索引指的就是对满足特定条件的拥有别名的索引，进行采用旧索引的配置创建新索引，并对将新索引别名下的`is_write_index`设为`true`。

顧棟

4259人浏览 · 2021-12-21 20:18:45

顧棟 · 2021-12-21 20:18:45 发布

文章目录

Rollover Index

Rollover Index

翻滚索引指的就是对满足特定条件的拥有别名的索引，进行采用旧索引的配置创建新索引，并对将新索引别名下的is_write_index设为true。

使用前提

索引拥有别名
翻滚时需要为新索引指定mapping和settings

功能说明

翻滚场景

别名指向单个索引且is_write_index未配置时，翻滚之后，别名会指向新索引，并从旧索引中移除别名。
别名指向单个索引且is_write_index为true时，翻滚之后，别名会同时指向新旧索引，旧索引上的别名is_write_index设置为false，新索引上的别名is_write_index设置为false。
别名指向多个索引，在这种情况下，写入索引将其翻转别名is_write_index设置为false，而新创建的索引现在将翻转别名指向它作为写入索引，is_write_index为 true。

翻滚条件

在使用_rollover翻转时可以用条件conditions

conditions 的参数

Name	Description
max_age	索引的最大年龄
max_docs	索引的所有主分片的文档数和的最大值，到达设定值就可以翻滚
max_size	索引的主分片的最大的大小

自定义翻滚

如果现有索引的名称以-和数字结尾，例如 logs-000001— 那么新索引的名称将遵循相同的模式，增加数字（logs-000002）。无论旧索引名称如何，该数字都用零填充，长度为 6。如果旧名称与此模式不匹配，则必须指定新索引的名称。

定义新索引名，在API的URL中新增新索引的名称
重写settings，在请求体中新增settings参数来重写新索引的settings

POST /logs_write/_rollover/new_index_name
{
  "conditions" : {
    "max_age": "7d",
    "max_docs": 1000,
    "max_size": "5gb"
  },
  "settings": {
    "index.number_of_shards": 2
  }
}

试运行模式

rollover API 支持试运行模式，可以在不执行实际rollover 的情况下检查请求条件，通过URL中增加参数dry_run实现。

POST /logs_write/_rollover?dry_run
{
  "conditions" : {
    "max_age": "7d",
    "max_docs": 1000,
    "max_size": "5gb"
  }
}

举例说明

创建索引，未指明is_write_index，URL编码化的<gudong-logs-{now/d}-1>

PUT /%3Cgudong-logs-%7Bnow%2Fd%7D-1%3E 
{
  "aliases": {
    "logs_write": {}
  },
  "mappings": {
            "_doc": {
                "properties": {
                    "create_date": {
                        "type": "date",
                        "format": "yyyy-MM-dd HH:mm:ss"
                    },
                    "message": {
                        "type": "keyword"
                    }
                }
            }
        },
  "settings":{
    "number_of_shards": 1
  }
}

插入数据

PUT logs_write/_doc/1
{
"message":"1111"
}
PUT logs_write/_doc/2
{
"message":"2222"
}

翻滚索引，

POST /logs_write/_rollover 
{
  "conditions": {
    "max_age":   "7d",
    "max_docs":  1,
    "max_size":  "5gb"
  }
}

此时max_docs生效。新建gudong-logs-2021.12.21-000002，如果旧索引的命名是以-数字结尾，新索引是以的旧索引数字累加补前置补0，形成6位数。

{
    "acknowledged": true,
    "shards_acknowledged": true,
    "old_index": "gudong-logs-2021.12.21-1",
    "new_index": "gudong-logs-2021.12.21-000002",
    "rolled_over": true,
    "dry_run": false,
    "conditions": {
        "[max_size: 5gb]": false,
        "[max_docs: 1]": true,
        "[max_age: 7d]": false
    }
}

查询索引情况

{
    "gudong-logs-2021.12.21-000002": {
        "aliases": {
            "logs_write": {}
        },
        "mappings": {
            "_doc": {
                "properties": {
                    "message": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    }
                }
            }
        },
        "settings": {
            "index": {
                "creation_date": "1640087157307",
                "number_of_shards": "5",
                "number_of_replicas": "1",
                "uuid": "kz5isQ_MRT6dYZzJN9CK3A",
                "version": {
                    "created": "6070299"
                },
                "provided_name": "<gudong-logs-{now/d}-000002>"
            }
        }
    },
    "gudong-logs-2021.12.21-1": {
        "aliases": {},
        "mappings": {
            "_doc": {
                "properties": {
                    "create_date": {
                        "type": "date",
                        "format": "yyyy-MM-dd HH:mm:ss"
                    },
                    "message": {
                        "type": "keyword"
                    }
                }
            }
        },
        "settings": {
            "index": {
                "creation_date": "1640087134488",
                "number_of_shards": "1",
                "number_of_replicas": "1",
                "uuid": "SyhQs_zXTlumeuNazEcoqw",
                "version": {
                    "created": "6070299"
                },
                "provided_name": "<gudong-logs-{now/d}-1>"
            }
        }
    }
}

发现两个索引在settings与mappings上均不一致。

重新翻滚

POST /logs_write/_rollover 
{
  "conditions": {
    "max_age":   "7d",
    "max_docs":  1,
    "max_size":  "5gb"
  },
  "mappings": {
            "_doc": {
                "properties": {
                    "create_date": {
                        "type": "date",
                        "format": "yyyy-MM-dd HH:mm:ss"
                    },
                    "message": {
                        "type": "keyword"
                    }
                }
            }
        },
  "settings":{
    "number_of_shards": 1
  }
}

查询索引情况

{
    "gudong-logs-2021.12.21-000002": {
        "aliases": {
            "logs_write": {}
        },
        "mappings": {
            "_doc": {
                "properties": {
                    "create_date": {
                        "type": "date",
                        "format": "yyyy-MM-dd HH:mm:ss"
                    },
                    "message": {
                        "type": "keyword"
                    }
                }
            }
        },
        "settings": {
            "index": {
                "creation_date": "1640087430710",
                "number_of_shards": "1",
                "number_of_replicas": "1",
                "uuid": "c8t1BKmSQGmlGDkvPqeuQQ",
                "version": {
                    "created": "6070299"
                },
                "provided_name": "<gudong-logs-{now/d}-000002>"
            }
        }
    },
    "gudong-logs-2021.12.21-1": {
        "aliases": {},
        "mappings": {
            "_doc": {
                "properties": {
                    "create_date": {
                        "type": "date",
                        "format": "yyyy-MM-dd HH:mm:ss"
                    },
                    "message": {
                        "type": "keyword"
                    }
                }
            }
        },
        "settings": {
            "index": {
                "creation_date": "1640087420246",
                "number_of_shards": "1",
                "number_of_replicas": "1",
                "uuid": "qagvEifqSnKTywA7NJsChA",
                "version": {
                    "created": "6070299"
                },
                "provided_name": "<gudong-logs-{now/d}-1>"
            }
        }
    }
}