RedisTemplate.execute执行lua脚本踩坑记录

一波三折目录EvalSha is not supported in cluster environmentNo way to dispatch this command to Redis Cluster because keys have different slotsJedis does not support password protected Redis Cluster configura

李有乾

12003人浏览 · 2021-09-27 17:06:56

李有乾 · 2021-09-27 17:06:56 发布

一波三折

EvalSha is not supported in cluster environment

No way to dispatch this command to Redis Cluster because keys have different slots

Jedis does not support password protected Redis Cluster configurations!

背景：最近项目新引入redis，代码中为了保证redis原子操作，大部分操作都使用了lua脚本，redisTemplate的API如下

redisTemplate.execute(RedisScript<T> script, List<String> keys, Object args[])

本地Redis单机模式下测试没问题

因生产环境下Cluster集群部署，稳妥起见本地搭一个三主三从集群测试，报错如下

EvalSha is not supported in cluster environment

org.springframework.dao.InvalidDataAccessApiUsageException: EvalSha is not supported in cluster environment.
	at org.springframework.data.redis.connection.jedis.JedisClusterConnection.evalSha(JedisClusterConnection.java:3568)

从redisTemplate.execute跟踪源码到org.springframework.data.redis.connection.jedis.JedisClusterConnection.evalSha，发现spring-redis的jedis集群连接不支持evalsha操作

jar包版本：spring-data-redis-1.7.1.RELEASE.jar

	public <T> T eval(byte[] script, ReturnType returnType, int numKeys, byte[]... keysAndArgs) {
		throw new InvalidDataAccessApiUsageException("Eval is not supported in cluster environment.");
	}

	public <T> T evalSha(String scriptSha, ReturnType returnType, int numKeys, byte[]... keysAndArgs) {
		throw new InvalidDataAccessApiUsageException("EvalSha is not supported in cluster environment.");
	}

	public <T> T evalSha(byte[] scriptSha, ReturnType returnType, int numKeys, byte[]... keysAndArgs) {
		throw new InvalidDataAccessApiUsageException("EvalSha is not supported in cluster environment.");
	}

解决：换种API，redisTemplate.execute(RedisCallback<T> call)

代码中获取jedis客户端的连接对象进行eval，如下代码

	@SuppressWarnings({ "rawtypes", "unchecked" })
	public <T> T execute(final RedisScript<T> script, final List<String> keys, final Object args[]) {
		return this.redisTemplate.execute(new RedisCallback<T>(){
		
			@Override
			public T doInRedis(RedisConnection connection) throws DataAccessException {
				Object nativeConnection = connection.getNativeConnection();
				
                // redis序列化key、value、lua脚本
				RedisSerializer keySerializer = redisTemplate.getKeySerializer();
				RedisSerializer valueSerializer = redisTemplate.getValueSerializer();
				RedisSerializer<String> stringSerializer = redisTemplate.getStringSerializer();
				
				List<byte[]> keys_ByteArr = new ArrayList<byte[]>(keys.size()); 
				List<byte[]> args_ByteArr = new ArrayList<byte[]>(args.length); 
				
				for (int i = 0; i < keys.size(); i++) {
					keys_ByteArr.add(keySerializer.serialize(keys.get(i)));
				}
				
				for (int j = 0; j < args.length; j++) {
					args_ByteArr.add(valueSerializer.serialize(args[j]));
				}
				
				byte[] scriptByte = stringSerializer.serialize(script.getScriptAsString());
				// 集群模式
				if (nativeConnection instanceof JedisCluster) {
					return (T) ((JedisCluster) nativeConnection).eval(scriptByte, keys_ByteArr, args_ByteArr);
				}
				// 单机模式
				else {
					return (T) ((Jedis) nativeConnection).eval(scriptByte, keys_ByteArr, args_ByteArr);
				}
			}
		});
	}

先连接redis单机测试，测试通过，再连接集群环境，其他lua都没问题，就只有下边的lua报错

No way to dispatch this command to Redis Cluster because keys have different slots

（两个redis操作, 1) set集合sadd添加一个对象元素的ID 2）保存对象数据为hash结构，对象ID为key, 对象属性为hashkey，属性值为hashvalue，为了保证原子操作，使用了lua脚本，入参key两个，入参argv数组每两个一对作为hashkey-hashvalue）

// lua脚本原子操作redis
		StringBuilder sb = new StringBuilder();
		sb.append(" local a = redis.call('sadd', KEYS[1], ARGV[1]) ");
		sb.append(" if (a == 1) then ");
		sb.append("    local hashkey ");
		sb.append("    for i,v in ipairs(ARGV) do ");// 循环传入的argv数组
		sb.append("        if (i > 1) then ");// 跳过第一个value，此后的每两个一对保存为hashkey-hashvalue
		sb.append("            if (i % 2 == 0) then ");
		sb.append("                hashkey = v ");
		sb.append("            else ");
		sb.append("                redis.call('hset', KEYS[2], hashkey, v) ");
		sb.append("            end ");
		sb.append("        end ");
		sb.append("    end ");
		sb.append(" else ");
		sb.append("    return 0 ");
		sb.append(" end ");
		sb.append(" return 1 ");

redis.clients.jedis.exceptions.JedisClusterException: No way to dispatch this command to Redis Cluster because keys have different slots.
	at redis.clients.jedis.JedisClusterCommand.runBinary(JedisClusterCommand.java:74)
	at redis.clients.jedis.BinaryJedisCluster.eval(BinaryJedisCluster.java:1242)
	at com.dw.pub.sicp3.si.center.RedisUtil$1.doInRedis(RedisUtil.java:102)

再次跟踪源码到redis.clients.jedis.JedisClusterCommand.runBinary()，jedis操作lua脚本入参key经过hash计算后的slot必须是相同的，否则就报错，因为上述lua脚本入参有两个，slot不同所以报错

  public T runBinary(int keyCount, byte[][] keys) {
    if ((keys == null) || (keys.length == 0)) {
      throw new JedisClusterException("No way to dispatch this command to Redis Cluster.");
    }

    if (keys.length > 1) {
      int slot = JedisClusterCRC16.getSlot(keys[0]);
      for (int i = 1; i < keyCount; i++) {
        int nextSlot = JedisClusterCRC16.getSlot(keys[i]);
        if (slot != nextSlot) {
          throw new JedisClusterException("No way to dispatch this command to Redis Cluster because keys have different slots.");
        }
      }

    }

    return runWithRetries(keys[0], this.redirections, false, false);
  }

// 如果key中存在{}，只crc大括号中的字符
  public static int getSlot(String key) {
    int s = key.indexOf("{");
    if (s > -1) {
      int e = key.indexOf("}", s + 1);
      if ((e > -1) && (e != s + 1)) {
        key = key.substring(s + 1, e);
      }

    }

    return getCRC16(key) & 0x3FFF;
  }

解决：从上方源码可以看出，计算slot只计算大括号{}中的串，因此判断如果是集群，key增加{}前缀

	private static String dealKeyPrefix(String str) {
		if (redisUtil.isCluster()) {
			str = "{cluster:}" + str;// 集群模式下使用
		}
		
		return str;
	}

生产库有密码，再在本机的集群加上密码再次测试，淦报错如下：

Jedis does not support password protected Redis Cluster configurations!

java.lang.IllegalArgumentException: Jedis does not support password protected Redis Cluster configurations!
	at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(JedisConnectionFactory.java:299)
	at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(JedisConnectionFactory.java:273)
	at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.afterPropertiesSet(JedisConnectionFactory.java:235)

跟源码spring-redis.jar，发现如果是集群，又发现设置了密码，直接报错如上

解决：百度之后，是jedis2.8之前不支持集群设置密码，升级jar包后解决

目前的报错jar包版本：

jedis-2.8.1.jar

spring-data-redis-1.7.1.RELEASE.jar

升级后版本：

jedis-2.9.0.jar

spring-data-redis-1.8.6.RELEASE.jar