elasticsearch中设置index.highlight.max_analyzed_offset

Caused by: java.lang.IllegalArgumentException: 
The length of [message] field of [VqCNQHkBFO-4K4CU-7tq] doc of [xxxx-node02-xxxx-2021.05.06] 
index has exceeded [1000000] - maximum allowed to be analyzed for highlighting. 
This maximum can be set by changing the [index.highlight.max_analyzed_offset] index level setting.
For large texts, indexing with offsets or term vectors is recommended!

浏览官网可知:

突出显示没有偏移或术语向量的索引文本,需要在搜索请求期间实时分析内存中的该文本。对于大文本,此分析可能需要大量时间和内存。为了防止这种情况,将分析的最大字符数限制为 1000000。可以使用 index 设置为特定索引更改此默认限制index.highlight.max_analyzed_offset

max_analyzed_offset 用于高亮请求的可分析最大字符数量,仅对无offsetsterm vectors的文本高亮请求有效。默认值1000000

1、通过kibana进行配置:

PUT /_settings
 {
    "index" : {
        "highlight.max_analyzed_offset" : 100000000
    }
}

2、通过java进行配置

完整的配置类,我这里采用的是更新设置的请求方法,也可使用新增,主要得根据自己的情况。
因为我的创建索引是在实体类中以@Document(indexName = INDEXNAME,createIndex = true)来创建。所以我设置以更新的方式UpdateSettingsRequest.

import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.settings.Settings;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.time.Duration;

/**
 * es配置
 */

@Component
public class ESConfig {

    @Value("${es.ip}")
    private String ip;

    @Value("${es.port}")
    private int port;

    @Value("${es.indexHighlightMaxAnalyzedOffset}")
    private Long indexHighlightMaxAnalyzedOffset;

    @Bean
    public RestHighLevelClient restHighLevelClient() throws IOException {
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost(ip, port, "http")
                )
                        .setHttpClientConfigCallback(httpClientBuilder -> {
                            httpClientBuilder.setKeepAliveStrategy(((httpResponse, httpContext) -> Duration.ofMinutes(5).toMillis()));
                            httpClientBuilder.disableAuthCaching();
                            return httpClientBuilder;
                        })
        );

       //索引偏移量设置/更新  index.highlight.max_analyzed_offset
        UpdateSettingsRequest updateSettingsRequest = new UpdateSettingsRequest(INDEXNAME);
        updateSettingsRequest.settings(Settings.builder().put("index.highlight.max_analyzed_offset",indexHighlightMaxAnalyzedOffset));
        client.indices().putSettings(updateSettingsRequest, RequestOptions.DEFAULT);

        return client;
    }
}

index.highlight.max_analyzed_offset的大小虽然不能过小,但建议也不要过大,会对内存带来负担。

Logo

为开发者提供学习成长、分享交流、生态实践、资源工具等服务,帮助开发者快速成长。

更多推荐