elasticsearch实现根据某一个字段去重后并求总数
版本说明:elasticsearch:elasticsearch-7.12.1kibana:kibana-7.12.1-windows-x86_64原始数据:_index_type_id▲_scoreaddressagebiridname
版本说明:
elasticsearch:elasticsearch-7.12.1
kibana:kibana-7.12.1-windows-x86_64
原始数据:
_index | _type | _id | _score | address | age | bir | id | name |
---|---|---|---|---|---|---|---|---|
jason_doc_10 | _doc | L2aqNoABfXKo_Zq7BPkO | 1 | 中国深圳 | 69 | 1650184553376 | daZhaoLiu | 大赵六 |
jason_doc_10 | _doc | MGaqNoABfXKo_Zq7mfma | 1 | 中国上海市宝山区 | 20 | 1649862303342 | user001 | 大张三 |
jason_doc_10 | _doc | MWaqNoABfXKo_Zq7mfma | 1 | 中国上海市浦东区 | 50 | 1649862303342 | user00100 | 小张三 |
jason_doc_10 | _doc | MmaqNoABfXKo_Zq7mfma | 1 | 中国上海市黄埔 | 34 | 1649862303342 | 00user1 | 李四 |
jason_doc_10 | _doc | M2aqNoABfXKo_Zq7mfma | 1 | 上海普陀区 | 45 | 1649862303342 | user0011 | 大李四 |
jason_doc_10 | _doc | NGaqNoABfXKo_Zq7mfma | 1 | 中国北京市朝阳区 | 62 | 1649862303342 | user2201 | 小李四 |
jason_doc_10 | _doc | NWaqNoABfXKo_Zq7mfma | 1 | 中国北京市东城 | 42 | 1649862303342 | 78user01 | 王五 |
jason_doc_10 | _doc | NmaqNoABfXKo_Zq7mfma | 1 | 中国北京市西城区 | 12 | 1649862303342 | 22 | 大王五 |
jason_doc_10 | _doc | N2aqNoABfXKo_Zq7mfma | 1 | 北京市海淀区 | 18 | 1649862303342 | user01 | 小王五 |
jason_doc_10 | _doc | OGasNoABfXKo_Zq77Pkx | 1 | 10 | test_save_01 | 测试save用户 | ||
jason_doc_10 | _doc | OWasNoABfXKo_Zq77Pmu | 1 | 20 | test_save_02 | 测试save用户_02 | ||
jason_doc_10 | _doc | OmasNoABfXKo_Zq77Pmu | 1 | 30 | test_save_03 | 测试save用户_03 | ||
jason_doc_10 | _doc | Q2aeO4ABfXKo_Zq7TvlP | 1 | 广东省广州市天河区 | 55 | 1650267672018 | daZhaoLiu | 王海 |
因为数据的原因,我这里只有年龄age字段有重复的,所以根据age去重。
DSL请求:
GET /jason_doc_10/_search
{
"size": 0,
"aggs": {
"distic_age": {
"cardinality": {
"field": "age"
}
}
}
}
响应结果:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 13,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"distic_age" : {
"value" : 12
}
}
}
java中的实现:
首先pom依赖:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
<version>2.6.6</version>
</dependency>
<!-- 注意:这里使用的ES版本是7.12.1,所以我本地安装的ES和kibana也是7.12.1 -->
配置的客户端:HighLevelClient
@SpringBootConfiguration
public class RestHingLevelClient {
@Bean
public RestClientBuilder restClientBuilder() {
return RestClient.builder(createHttpHost("127.0.0.1:9200"));
}
@Bean
public RestHighLevelClient restHighLevelClient(@Autowired RestClientBuilder restClientBuilder) {
return new RestHighLevelClient(restClientBuilder);
}
private HttpHost createHttpHost(String ipPort) {
String[] split = ipPort.split(":");
String ip = split[0];
int port = Integer.parseInt(split[1]);
return new HttpHost(ip, port, "http");
}
}
测试类:
@SpringBootTest
public class TestQueryDocument {
@Autowired
@Qualifier(value = "restHighLevelClient")
RestHighLevelClient client;
@Test
public void testDistinctByAge() throws IOException {
// 创建查询请求
SearchRequest request = new SearchRequest();
request.indices("jason_doc_10");
// 创建查询条件
SearchSourceBuilder builder = new SearchSourceBuilder();
// 根据年龄去重,并起别名为:distic_age
builder.aggregation(AggregationBuilders.cardinality("distic_age").field("age"));
request.source(builder);
// 查询
SearchResponse response = client.search(request, RequestOptions.DEFAULT);
Aggregations aggregations = response.getAggregations();
ParsedCardinality cardinality = (ParsedCardinality) aggregations.asList().get(0);
long count = cardinality.getValue();
System.out.println("去重后的总数:" + count);
}
更多推荐
所有评论(0)