redis计算指定key大小

memory usageredis4 及已上版本>SET""""OK> MEMORY USAGE ""(integer)51memoryusage 命令返回key和value占用内存的bytes--stat持续输出stats模式，在测试上可以做测试>redis-cli-apassword-p6379-hhostname--stat------- data ------ -----

大怀特

2299人浏览 · 2021-09-26 09:31:55

大怀特 · 2021-09-26 09:31:55 发布

memory usage

redis4 及已上版本

> SET "" ""OK> MEMORY USAGE ""(integer) 51

memory usage 命令返回key和value占用内存的bytes

--stat

持续输出stats模式，在测试上可以做测试

> redis-cli -a password  -p 6379 -h hostname --stat------- data ------ --------------------- load -------------------- - child -keys       mem      clients blocked requests            connections          35768376   6.98G    2021    0       2600414876 (+3)     3121642     35768376   6.98G    2021    0       2600414879 (+3)     3121642     35768376   6.98G    2021    0       2600414884 (+5)     3121642     35768376   6.98G    2021    0       2600414887 (+3)     3121642     35768376   6.98G    2021    0       2600414890 (+3)     3121642     35768376   6.98G    2021    0       2600414894 (+4)     3121642     35768376   6.98G    2021    0       2600414898 (+4)     3121642     35768376   6.98G    2021    0       2600414901 (+3)     3121642     35768376   6.98G    2021    0       2600414905 (+4)     3121642     35768376   6.98G    2021    0       2600414909 (+4)     3121642     35768376   6.98G    2021    0       2600414912 (+3)     3121642

通过第三方工具rdbtools来计算key大小

安装pip

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py# 用相对应版本python安装对应版本sudo python get-pip.pysudo python3 get-pip.py

安装rdbtools

pip install rdbtools

查看指定key大小

 redis-memory-for-key -s hostname [-a password]  -p 6379 -d 0 key1

rdb

# rdb --help
usage: usage: rdb [options] /path/to/dump.rdb
Example : rdb --command json -k "user.*" /var/redis/6379/dump.rdb
positional arguments:
  dump_file             RDB Dump file to process
optional arguments:
  -h, --help            show this help message and exit
  -c CMD, --command CMD
                        Command to execute. Valid commands are json, diff,
                        justkeys, justkeyvals, memory and protocol
  -f FILE, --file FILE  Output file
  -n DBS, --db DBS      Database Number. Multiple databases can be provided.
                        If not specified, all databases will be included.
  -k KEYS, --key KEYS   Keys to export. This can be a regular expression
  -o NOT_KEYS, --not-key NOT_KEYS
                        Keys Not to export. This can be a regular expression
  -t TYPES, --type TYPES
                        Data types to include. Possible values are string,
                        hash, set, sortedset, list. Multiple typees can be
                        provided. If not specified, all data types will be
                        returned
  -b BYTES, --bytes BYTES
                        Limit memory output to keys greater to or equal to
                        this value (in bytes)
  -l LARGEST, --largest LARGEST
                        Limit memory output to only the top N keys (by size)
  -e {raw,print,utf8,base64}, --escape {raw,print,utf8,base64}
                        Escape strings to encoding: raw (default), print,
                        utf8, or base64.
  -x, --no-expire       With protocol command, remove expiry from all keys
  -a N, --amend-expire N
                        With protocol command, add N seconds to key expiry
                        time

rdb -c memory /tmp/dump.rdb 
database,type,key,size_in_bytes,encoding,num_elements,len_largest_element,expiry
0,hash,data:index_flow_yingshi,10492,hashtable,1,8992,2019-01-14T08:20:10.236000
0,hash,data:index_movie,22068,hashtable,7,2896,2019-01-14T07:29:19.685000
0,string,block:index_module_novel,8296,string,7694,7694,2019-01-13T00:27:46.128000
0,string,block:index_bottom_baike_aikan,8296,string,7632,7632,2019-01-14T02:27:11.850000
0,string,block:index_bottom_tools,5224,string,4549,4549,2019-01-13T01:02:09.171000
0,string,block:index_module_travel,7272,string,6408,6408,2019-01-13T00:43:39.478000
...

输出了db，数据类型，key, 大小, 编码等多列信息

在这种特殊的模式下，redis-cli会对key进行空间分析，扫描出占用空间大的key的集，并且提供关于数据类型和数组组成构造

只要知道最大的N个key, 可以使用-l选项

[@sjs_73_171 ~]$ rdb -c memory -l 3 /tmp/dump.rdb  
database,type,key,size_in_bytes,encoding,num_elements,len_largest_element,expiry
0,hash,city_tong,724236,hashtable,3113,216,2019-01-14T01:10:59.407000
0,hash,iplocsearch,406292,hashtable,383,180190,2019-01-30T05:37:56.082000
0,hash,weather_tong3,583844,hashtable,319,1658,2019-01-07T10:22:33.742000

redis-cli --bigkeys

redis-cli -a password -p 6379 -h hostname --bigkeys# Scanning the entire keyspace to find biggest keys as well as# average sizes per key type.  You can use -i 0.1 to sleep 0.1 sec# per 100 SCAN commands (not usually needed).[00.00%] Biggest string found so far '100000004602269' with 156 bytes[00.00%] Biggest string found so far '100000006251981' with 225 bytes[00.00%] Biggest string found so far '100000011594328' with 226 bytes[00.00%] Biggest string found so far '100000000095649' with 227 bytes[00.01%] Biggest string found so far '100000003257493' with 228 bytes...

debug object key

redis的命令，可以查看某个key序列化后的长度。例：

连接上redis后执行如下命令

b.redis:1959> hmset myhash k1 v1 k2 v2 k3 v3
OK
b.redis:1959> debug object myhash
Value at:0x7f005c6920a0 refcount:1 encoding:ziplist serializedlength:36 lru:3341677 lru_seconds_idle:2

Value at：key的内存地址
refcount：引用次数
encoding：编码类型
serializedlength：序列化长度
lru_seconds_idle：空闲时间

几个需要注意的问题 serializedlength是key序列化后的长度(redis在将key保存为rdb文件时使用了该算法)，并不是key在内存中的真正长度。这就像一个数组在json_encode后的长度与其在内存中的真正长度并不相同。不过，它侧面反应了一个key的长度，可以用于比较两个key的大小。 serializedlength会对字串做一些可能的压缩。如果有些字串的压缩比特别高，那么在比较时会出现问题。比如下列：

b.redis:1959> set str1 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
OK
b.redis:1959> set str2 abcdefghijklmnopqrstuvwxyz1234
OK
b.redis:1959> debug object str1
Value at:0x7f007c035b80 refcount:1 encoding:embstr serializedlength:12 lru:3342615 lru_seconds_idle:13
b.redis:1959> debug object str2
Value at:0x7f00654df400 refcount:1 encoding:embstr serializedlength:31 lru:3342622 lru_seconds_idle:7

两个字串的实际长度都是30, 但str1的serializedlength为12， str2的为31。