让学习成为一种习惯！--------magic_guo

之前的文章搭建了rabbitMq、es、ik和kibana环境，这次实现es的搜索模块；

在一个电商项目中数据量和并发量很大，如果直接去数据库查询数据，会给数据库造成很大的压力，有可能导致数据库的宕机；
当然解决办法也有很多，其中最典型的两种解决办法就是使用缓存和搜索引擎：
1、数据库 + redis
2、数据库 + 搜索引擎

此模块的业务构架：

es模块的maven依赖：
需要注意的是，es依赖版本要和服务器的es版本要一致；

 <dependencies>

        <dependency>
            <groupId>com.guo</groupId>
            <artifactId>shop-common</artifactId>
        </dependency>

        <dependency>
            <groupId>com.guo</groupId>
            <artifactId>shop-feign</artifactId>
        </dependency>

        <dependency>
            <groupId>commons-beanutils</groupId>
            <artifactId>commons-beanutils</artifactId>
        </dependency>

        <!--1. elasticsearch-->
        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>7.2.0</version>
        </dependency>

        <!--2. elasticsearch的高级API-->
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>7.2.0</version>
        </dependency>

        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-amqp</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

    </dependencies>

模块中要用到两个配置：
rabbitMq的配置：
交换机、队列，然后将队列绑定到交换机上：

package com.guo.config;

import com.guo.constants.ShopConstants;
import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.core.TopicExchange;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;


@Configuration
public class RabbitMQConfig {

    /**
     * 1.创建一个队列
     * 第一个参数：队列的名字；第二个参数：是否消息持久化；第三个参数：是否独占队列；第四个参数：是否自动删除
     * 独占队列的意思是，此队列必须要一直在监听状态，如果没有被监听，则此队列会自动删除；
     * @return 返回一个
     */
    @Bean
    public Queue goodsQueue() {
        return new Queue(ShopConstants.GOODS_QUEUE, true, false, false);
    }

    // 2.创建一个交换机
    @Bean
    public TopicExchange goodsExchange() {
        return new TopicExchange(ShopConstants.GOODS_EXCHANGE, true, false);
    }

    // 3.将队列绑定到交换机上
    @Bean
    public Binding goodsQueueToGoodsExchange () {
        return BindingBuilder.bind(goodsQueue()).to(goodsExchange()).with("goods.*");
    }

}

ES的配置：
es的配置文件：

server:
  port: 8006
spring:
  application:
    name: shop-search
es:
  host: xxxxxxxxxx
  port: 9200
  index: xxxxxx
  type: xxxxxx

返回一个es的客户端：

package com.guo.config;

import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class EsConfig {

    @Value("${es.host}")
    private String esHost;

    @Value("${es.port}")
    private Integer esPort;

    @Bean
    public RestHighLevelClient restClient() {
        HttpHost httpHost = new HttpHost(esHost, esPort);
        RestClientBuilder restClientBuilder = RestClient.builder(httpHost);

        RestHighLevelClient client = new RestHighLevelClient(restClientBuilder);
        return client;
}

接下来创建一个关于商品的索引：
可以在kibana平台直接创建、也可以写一个接口，来创建；我是在测试模块创建的，顺便可以测试一下es的连接状态：
创建索引的步骤：

@SpringBootTest
class ShopSearchApplicationTests {

    @Autowired
    private RestHighLevelClient client;

    /**
     * 创建索引的过程
     * MySQL => Databases => Tables => Columns/Rows
     * Elasticsearch => Indices => Types => Documents with Properties
     * @throws IOException
     */
    @Test
    void contextLoads() throws IOException {

        // 1.创建一个索引请求
        CreateIndexRequest indexRequest = new CreateIndexRequest();
        // 设置索引的名字
        indexRequest.index("shop-index");

        // 2.创建一个Settings
        Settings.Builder settings = Settings.builder();
        settings.put("number_of_shards", "3");  // 设置三个分片
        settings.put("number_of_replicas", "1");  // 设置一个备份

        // 3.把settings设置给request对象
        indexRequest.settings(settings);

        // 4.创建一个mappings
        XContentBuilder mappings = JsonXContent.contentBuilder();
        mappings.
                startObject()
                    .startObject("properties")
                        .startObject("id")
                            .field("type", "integer")
                        .endObject()
                        .startObject("gname")
                            .field("type", "text")
                            .field("analyzer", "ik_max_word")
                        .endObject()
                        .startObject("gdesc")
                            .field("type", "text")
                            .field("analyzer", "ik_max_word")
                        .endObject()
                        .startObject("gtype")
                            .field("type", "integer")
                        .endObject()
                        .startObject("gprice")
                            .field("type", "double")
                        .endObject()
                        .startObject("gpng")
                            .field("type", "keyword")
                        .endObject()
                    .endObject()
                .endObject();
        // 5.把mappings设置给index
        indexRequest.mapping("shop-type");

        // 6.客户端开始发送请求
        CreateIndexResponse response = client.indices().create(indexRequest, RequestOptions.DEFAULT);
        System.out.println(response.toString());
    }
}

添加商品测试：

    @Test
    public void testAddGoods() throws IOException {
        // 初始化商品
        Goods goods = new Goods();
        goods.setGname("华为手机");
        goods.setGdesc("手机 huawei mate30");
        goods.setGprice(BigDecimal.valueOf(5000.0));
        goods.setGtype(2);
        goods.setId(10);
        goods.setTempPng("a.png|b.png|c.png");

        // 创建一个index请求
        IndexRequest indexRequest = new IndexRequest("shop-index");

        // 将对象转成json
        String json = new Gson().toJson(goods);

        indexRequest.source(json, XContentType.JSON);

        IndexResponse indexResponse = client.index(indexRequest, RequestOptions.DEFAULT);

        System.out.println(indexResponse.toString());
        System.out.println(indexRequest);
    }

以上测试都没问题的话，开始编写service层和controller层：
service层：
接口：

public interface ISearchGoodsService {

    void addGoods(Goods goods) throws Exception;

    List<Goods> searchGoodsList(String keyword, Integer psort) throws Exception;
}

实现类：
这里在搜索接口中，添加了搜索结果的高亮和排序功能：

@Service
public class SearchGoodsSearchImpl implements ISearchGoodsService {

    @Resource
    private RestHighLevelClient client;

    @Value(("${es.index}"))
    private String esIndex;

    @Value("${es.type}")
    private String esType;

    @Override
    public void addGoods(Goods goods) throws Exception{
        System.out.println(goods);
        // 创建一个请求
        IndexRequest indexRequest = new IndexRequest(esIndex);
        System.out.println(indexRequest);
        // 对象格式化
        String json = new Gson().toJson(goods);

        // 将对象放进request中
        indexRequest.source(json, XContentType.JSON);
        // 发送请求
        IndexResponse index = client.index(indexRequest, RequestOptions.DEFAULT);
    }

    // 商品名字和商品描述都要匹配关键字
    @Override
    public List<Goods> searchGoodsList(String keyword, Integer psort) throws Exception {

        // 创建一个search请求
        SearchRequest searchRequest = new SearchRequest(esIndex);

        // 设置请求条件
        SearchSourceBuilder builder = new SearchSourceBuilder();

        // 只要是gname或者gdesc中包含关键字都可以查询，设置查询条件
        if (StringUtils.isEmpty(keyword)) {
            // 如果没有关键字 则查询全部
            builder.query(QueryBuilders.matchAllQuery());
        } else {
            builder.query(QueryBuilders.multiMatchQuery(keyword, "gname", "gdesc"));
        }

        // 设置高亮属性
        HighlightBuilder highlightBuilder = new HighlightBuilder();
        highlightBuilder.field("gname", 30);
        highlightBuilder.preTags("<font color = 'red'>");
        highlightBuilder.postTags("</font>");

        // 设置排序规则(默认值， 价格(1), 名字(2))
        if (psort != null) {
            if ("1".equals(psort.toString())) {
                builder.sort("gprice", SortOrder.DESC);
            } else if ("2".equals(psort.toString())) {
                builder.sort("name", SortOrder.DESC);
            }
        }

        // 把高亮的属性设置builder
        builder.highlighter(highlightBuilder);

        // 查询条件设置到request中
        searchRequest.source(builder);

        // 发送请求
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

        // 处理结果集
        // 准备一个集合
        List<Goods> goodsList = new ArrayList<>();

        // 获取查询出来的结果集
        for (SearchHit documentFields : searchResponse.getHits().getHits()) {

            // 获取商品的信息
            Map<String, Object> sourceAsMap = documentFields.getSourceAsMap();
            // 初始化一个商品
            Goods goods = new Goods();

            // 将map中的信息copy到goods中
            BeanUtils.populate(goods, sourceAsMap);

            // 获取高亮字段的信息, 但是有些数据是没有高亮信息的
            Map<String, HighlightField> highlightFields = documentFields.getHighlightFields();
            HighlightField gnameHigh = highlightFields.get("gname");
            if (gnameHigh != null) {
                // 说明当前这条数据有高亮字段
                goods.setGname(gnameHigh.getFragments()[0].toString());
            }
            goodsList.add(goods);
        }

        return goodsList;
    }
}

controller层：

@RestController
@RequestMapping("/searchGoodsController")
@Slf4j
public class SearchGoodsController {

    @Autowired
    private ISearchGoodsService searchGoodsService;

    @RequestMapping("/searchGoodsList")
    public ResultEntity searchGoodsList(String keyword, Integer psort, ModelMap modelMap) throws Exception {

        List<Goods> goodsList = searchGoodsService.searchGoodsList(keyword, psort);
        modelMap.put("goodsList", goodsList);
        return ResultEntity.success(goodsList);
//        return "searchList"; // 展示搜索到的视图页面
    }
}

然后还得有一个关于添加商品时将商品同步到es中，对于MQ消息的监听：

@Configuration
@Slf4j
public class GoodsQueueListener {

    // 创建一个线程池
    private ExecutorService executorService = Executors.newFixedThreadPool(5);

    @Autowired
    private ISearchGoodsService searchGoodsService;

    @RabbitListener(queues = ShopConstants.GOODS_QUEUE)
    public void addGoodsToEs(Goods goods, Channel channel, Message message) {
        System.out.println(message);
        executorService.submit(new Runnable() {
            @Override
            public void run() {
                try {
                    // 添加商品到es
                    searchGoodsService.addGoods(goods);
                    // 手动ACK
                    channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        });
    }
}

goods模块的消息发送：

    public ResultEntity addGoods(@RequestBody Goods goods) {
        log.info("{}", goods);
        // 1.将商品插入数据库
        boolean insert = goodsService.insert(goods);

        // 2.将商品同步到es
        rabbitTemplate.convertAndSend(ShopConstants.GOODS_EXCHANGE, ShopConstants.GOODS_ROUTING_KEY, goods);

        return ResultEntity.responseClient(insert);
    }

在测试之前，需要保证后台模块使用feign调用商品模块的流程是没问题的；

postman测试添加商品：

postman测试查询数据：

另外如果es模块是后来搭建的，则可以写一个脚本将数据的数据拿出来再同步到es中：

@Autowired
    private IGoodsSearchService goodsSearchService;

    @Autowired
    private IGoodsService goodsService;

    @Test
    public void syncGoodsTOEs() throws Exception {

        Page<Goods> goodsPage = goodsService.getGoodsPage(new Page<>());

        List<Goods> records = goodsPage.getRecords();

        for (Goods goods : records) {
            List<GoodsPic> goodsPicList = goods.getGoodsPicList();
            for (GoodsPic goodsPic : goodsPicList) {
                String png = goodsPic.getPng();
                String tempPng = goods.getTempPng();
                if (StringUtils.isEmpty(tempPng)) {
                    goods.setTempPng(png);
                } else {
                    goods.setTempPng(goods.getTempPng() + "|" + png);
                }
            }
            System.out.println(goods);
            goodsSearchService.addGoods(goods);
        }
    }

清除es中的所有数据：

    @Test
    public void clearGoods() throws IOException {
        DeleteByQueryRequest deleteByQueryRequest = new DeleteByQueryRequest("shop-index");
        deleteByQueryRequest.setDocTypes("shop-type");
        deleteByQueryRequest.setQuery(QueryBuilders.matchAllQuery());

        BulkByScrollResponse response = client.deleteByQuery(deleteByQueryRequest, RequestOptions.DEFAULT);
        System.out.println(response);
    }

使用kibana的语句：

 POST /shop-index/shop-type/_delete_by_query?pretty
{
  "query": {
  "match_all": {
    }
  }
}

---

本文章教学视频来自：https://www.bilibili.com/video/BV1tb4y1Q74E?p=3&t=125

---

静下心，慢慢来，会很快！