Springboot整合Neo4j和Mysql多数据源整合方案及其解决冲突问题方法

1. 背景最近数仓系统的元数据分析和任务依赖分析用到Neo4J，原有的后台模块使用了Mysql，需要在同一个模块中同时操作Mysql和Neo4J，整合的方案网上也有不少，感觉别人好像不会有整合的问题一样，而我在整合的过程中，碰到一系列问题。2. 整合步骤1. 添加Jar包依赖<dependency><groupId>org.springframework.boot</

mlwise

6074人浏览 · 2022-01-18 17:29:22

mlwise · 2022-01-18 17:29:22 发布

1. 背景

最近数仓系统的元数据分析和任务依赖分析用到Neo4J，原有的后台模块使用了Mysql，需要在同一个模块中同时操作Mysql和Neo4J，整合的方案网上也有不少，感觉别人好像不会有整合的问题一样，而我在整合的过程中，碰到一系列问题。

2. 整合步骤

1. 添加Jar包依赖

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-data-neo4j</artifactId>
</dependency>

2. 添加Neo4J数据源配置（我贴自己的配置文件）

spring:
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource
    driverClassName: com.mysql.cj.jdbc.Driver
    url: jdbc:mysql://10.200.1.112:3306/datacenter?useUnicode=true&useSSL=false&characterEncoding=utf8&serverTimezone=UTC
    username: root
    password: 123123
    druid:
      initial-size: 8
      min-idle: 3
      max-active: 20
      max-wait: 60000
      time-between-eviction-runsMillis: 60000
      min-evictable-idle-timeMillis: 300000
      validation-query: SELECT 1 FROM DUAL
      test-while-idle: true
      test-on-borrow: false
      test-on-return: false
      pool-prepared-statements: true
      max-open-prepared-statements: 20
      max-pool-prepared-statement-per-connection-size: 20
      filters: stat,wall,slf4j
      connection-properties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000
      use-global-data-source-stat: true
      web-stat-filter:
        enabled: true
        url-pattern: /*
        exclusions: '*.js,*.gif,*.jpg,*.bmp,*.png,*.css,*.ico,/druid/*'
      stat-view-servlet:
        enabled: true
        allow:
        reset-enable: 'false'
        url-pattern: '/druid/*'
        login-username: ''
        login-password: ''
  quartz:
    job-store-type: jdbc
    properties:
      org:
        quartz:
          scheduler:
            instanceName: ClusteredScheduler
            instanceId: AUTO
          jobStore:
            class: org.quartz.impl.jdbcjobstore.JobStoreTX
            driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
            tablePrefix: QRTZ_
            isClustered: true
            useProperties: false
            dontSetAutoCommitFalse: true
            misfireThreshold: 600000
          threadPool:
            class: org.quartz.simpl.SimpleThreadPool
            threadCount: 10
            threadPriority: 5
            threadsInheritContextClassLoaderOfInitializingThread: true
  redis:
    host: 10.200.1.112
    port: 6379
    password:
    timeout: 30000
    jedis:
      pool:
        max-active: 200
        max-wait: -1ms
        max-idle: 8
        min-idle: 0
  data:
    neo4j:
      uri: bolt://10.200.1.111:7687
      username: neo4j
      password: test

3. 创建实体Entity

创建实体类和关系实体类时，建议单独放到一个包或者说文件夹下，便于后面配置EntityScan，千万不要和Mysql相关的类放到一个文件夹，避免引起冲突。例子的包名为com.github.datacenter.neo4j.entity

package com.github.datacenter.neo4j.entity
// 实体类
@NodeEntity(label = "dept")
@Data
@Builder
public class Dept {

    @Id
    @GeneratedValue
    private Long id;

    @Property(name = "deptName")
    private String deptName;

}

package com.github.datacenter.neo4j.entity
// 关系实体类
@RelationshipEntity(type = "relationShip")
@Data
@Builder
public class RelationShip {

    @Id
    @GeneratedValue
    private Long id;

    @StartNode
    private Dept parent;

    @EndNode
    private Dept child;
}

3. 创建操作类

创建操作类时，建议单独放到一个包或者说文件夹下，便于后面配@EnableNeo4jRepositories

，千万不要和Mysql相关Mapper类放到一个文件夹，避免引起冲突。例子的包名为com.github.datacenter.neo4j.repositories

package com.github.datacenter.neo4j.repositories
@Repository
public interface DeptRepository extends Neo4jRepository<Dept, Long> {

    Dept findByName(String name);

}

4. 配置Neo4J，避免冲突

package com.github.datacenter.neo4j.config

import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.neo4j.ogm.session.SessionFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.domain.EntityScan;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.neo4j.repository.config.EnableNeo4jRepositories;
import org.springframework.data.neo4j.transaction.Neo4jTransactionManager;
import org.springframework.data.transaction.ChainedTransactionManager;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;

import javax.sql.DataSource;

@Configuration
@EnableTransactionManagement
@EnableNeo4jRepositories(basePackages = {"com.github.datacenter.neo4j.repositories"})
@EntityScan(basePackages = {"com.github.datacenter.neo4j.entity"} )
public class Neo4JConfig {
    // 1. 此处为了修改默认事务，必须改。加载了Neo4J依赖库之后，transactionManager变成Neo4jTransactionManager，不增加此处，启动会报错，Mysql无法使用。
    @Bean("transactionManager")
    @Primary  
    public DataSourceTransactionManager transactionManager(DataSource dataSource) {
        return new DataSourceTransactionManager(dataSource);
    }
    // 2. Neo4J的事务管理
    @Bean("neo4jTransactionManager")
    public Neo4jTransactionManager neo4jTransactionManager(SessionFactory sessionFactory) {
        return new Neo4jTransactionManager(sessionFactory);
    }
    // 3. 此处可选，非必须。需要使用多种事务时，需要加。
    @Autowired
    @Bean(name = "multiTransactionManager")
    public PlatformTransactionManager multiTransactionManager(
            Neo4jTransactionManager neo4jTransactionManager,
            DataSourceTransactionManager mysqlTransactionManager) {
        return new ChainedTransactionManager(
                neo4jTransactionManager, mysqlTransactionManager);
    }
}

1. @EnableNeo4jRepositories是为了配置Neo4J的扫描路径，不配置，会把所有的文件夹中的类都当作Neo4jRepository来扫描

2. @EntityScan为了配置实体和实体关系节点扫描路径，不配置，查询的时候找不到类，会报错。

3. 事务管理器说明

@Bean("transactionManager")
@Primary（有两个事务管理器时，最好加上这个注解@Primary）
public DataSourceTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}

如果想单独引用neo4j的事务也可以这样引用，代码如下

@Transactional(value="neo4jTransactionManager")

3. 碰到的问题

我特地试了一下缺少上面的某个代码段，会报什么样子的错误。下面某些问题，注释掉部分代码实验得到的，有些是真的碰到的。

1. 加载Neo4J依赖库，导致了默认transactionManager被Neo4J的事务管理局替换。现象如下：

Description:

Field dataSourceTransactionManager in xxxxxx required a bean of type 'org.springframework.jdbc.datasource.DataSourceTransactionManager' that could not be found.

The injection point has the following annotations:
	- @org.springframework.beans.factory.annotation.Autowired(required=true)

The following candidates were found but could not be injected:
	- Bean method 'transactionManager' in 'DataSourceTransactionManagerAutoConfiguration.DataSourceTransactionManagerConfiguration' not loaded because @ConditionalOnMissingBean (types: org.springframework.transaction.PlatformTransactionManager; SearchStrategy: all) found beans of type 'org.springframework.transaction.PlatformTransactionManager' transactionManager


Action:

Consider revisiting the entries above or defining a bean of type 'org.springframework.jdbc.datasource.DataSourceTransactionManager' in your configuration.

解决方案：

@Bean("transactionManager")
@Primary （@Primary注解可选）
public DataSourceTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}

2. neo4jTransactionManager事务管理器问题

===2022-01-18 17:00:41.204 [http-nio-8080-exec-1] ERROR xxxx Line:108 - No bean named 'neo4jTransactionManager' available: No matching TransactionManager bean found for qualifier 'neo4jTransactionManager' - neither qualifier match nor bean name match!
org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'neo4jTransactionManager' available: No matching TransactionManager bean found for qualifier 'neo4jTransactionManager' - neither qualifier match nor bean name match!
	at org.springframework.beans.factory.annotation.BeanFactoryAnnotationUtils.qualifiedBeanOfType(BeanFactoryAnnotationUtils.java:136)
	at org.springframework.beans.factory.annotation.BeanFactoryAnnotationUtils.qualifiedBeanOfType(BeanFactoryAnnotationUtils.java:95)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.determineQualifiedTransactionManager(TransactionAspectSupport.java:492)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.determineTransactionManager(TransactionAspectSupport.java:470)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:335)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
	at org

解决方案：

@Bean("neo4jTransactionManager")
public Neo4jTransactionManager neo4jTransactionManager(SessionFactory sessionFactory) {
return new Neo4jTransactionManager(sessionFactory);
}

3. 无法找到实体节点类和实体关系类

org.springframework.data.mapping.MappingException: Couldn't find PersistentEntity for type com.github.datacenter.neo4j.entity.Dept!
	at org.springframework.data.mapping.context.MappingContext.getRequiredPersistentEntity(MappingContext.java:119)
	at org.springframework.data.mapping.context.PersistentPropertyPathFactory.createPersistentPropertyPath(PersistentPropertyPathFactory.java:203)
	at org.springframework.data.mapping.context.PersistentPropertyPathFactory.lambda$getPersistentPropertyPath$2(PersistentPropertyPathFactory.java:183)
	at java.util.concurrent.ConcurrentMap.computeIfAbsent(ConcurrentMap.java:324)
	at org.springframework.data.mapping.context.PersistentPropertyPathFactory.getPersistentPropertyPath(PersistentPropertyPathFactory.java:182)
	at org.springframework.data.mapping.context.PersistentPropertyPathFactory.from(PersistentPropertyPathFactory.java:97)

解决方案：

@EntityScan(basePackages = {"com.github.datacenter.neo4j.entity"} )

4. 无法找到Neo4J操作类

此种错误一般就是因为没有扫描到此组件导致的。

Description:

Field Dept in xxxx required a bean of type 'com.github.datacenter.neo4j.repositories.DeptRepository' that could not be found.

The injection point has the following annotations:
	- @org.springframework.beans.factory.annotation.Autowired(required=true)


Action:

Consider defining a bean of type 'com.github.datacenter.neo4j.repositories.DeptRepository' in your configuration.

解决方案：

@EnableNeo4jRepositories(basePackages = {"com.github.datacenter.neo4j.repositories"})

华为云开发者联盟

为开发者提供学习成长、分享交流、生态实践、资源工具等服务，帮助开发者快速成长。

更多推荐

openEuler系统通过shell脚本安装openGauss 5.0.0企业版

华为云开发者联盟

一文了解npm install -g和npm install --save-dev的关系

华为云开发者联盟

【FAQ】HarmonyOS SDK 闭源开放能力 —Map Kit（2）

华为云开发者联盟

所有评论(0)

查看更多评论

mlwise

@mlwise

已为社区贡献1条内容