Redis_note

安装

CentOS

1
2
3


sudo yum install epel-release
sudo yum update
sudo yum install redis

Ubuntu

1

sudo apt install redis-server

源码安装

需要注意要安装 redis 5.0 以上的版本。源码下载地址 [https://redis.io/download]。

在 centos 7 上直接执行 make 会出现以下问题

In file included from adlist.c:34:
zmalloc.h:50:10: fatal error: jemalloc/jemalloc.h: No such file or directory
   50 | #include <jemalloc/jemalloc.h>
      |          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.

解决办法

make MALLOC=libc

然后出现问题：

linux redis release.c:37:10: fatal error: release.h: No such file or directory

解决办法

cd src
chmod +x mkreleasehdr.sh

cd src && make all
make[1]: Entering directory `/home/reas/Downloads/redis-6.2.6/src'
    CC Makefile.dep
make[1]: Leaving directory `/home/reas/Downloads/redis-6.2.6/src'
make[1]: Entering directory `/home/reas/Downloads/redis-6.2.6/src'
    CC release.o
    CC networking.o
    CC util.o
    CC object.o
    CC db.o
    CC replication.o
    CC rdb.o
    CC t_string.o
    CC t_list.o
    CC t_set.o
    CC t_zset.o
    CC t_hash.o
    CC config.o
    CC aof.o
    CC pubsub.o
    CC multi.o
    CC debug.o
debug.c: In function ‘debugCommand’:
debug.c:474:22: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  474 |         *((char*)-1) = 'x';
      |         ~~~~~~~~~~~~~^~~~~
    CC sort.o
    CC intset.o
    CC syncio.o
    CC cluster.o
    CC crc16.o
    CC endianconv.o
    CC slowlog.o
    CC scripting.o
    CC bio.o
    CC rio.o
    CC rand.o
    CC memtest.o
    CC crcspeed.o
    CC crc64.o
    CC bitops.o
    CC sentinel.o
    CC notify.o
    CC setproctitle.o
    CC blocked.o
    CC hyperloglog.o
    CC latency.o
    CC sparkline.o
    CC redis-check-rdb.o
    CC redis-check-aof.o
    CC geo.o
    CC lazyfree.o
    CC module.o
    CC evict.o
    CC expire.o
    CC geohash.o
    CC geohash_helper.o
    CC childinfo.o
    CC defrag.o
    CC siphash.o
    CC rax.o
    CC t_stream.o
    CC listpack.o
    CC localtime.o
    CC lolwut.o
    CC lolwut5.o
    CC lolwut6.o
    CC acl.o
    CC gopher.o
    CC tracking.o
    CC connection.o
    CC tls.o
    CC sha256.o
    CC timeout.o
    CC setcpuaffinity.o
    CC monotonic.o
    CC mt19937-64.o
    LINK redis-server
    INSTALL redis-sentinel
    CC redis-cli.o
redis-cli.c: In function ‘_serverAssert’:
redis-cli.c:485:18: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  485 |     *((char*)-1) = 'x';
      |     ~~~~~~~~~~~~~^~~~~
    CC cli_common.o
    LINK redis-cli
    CC redis-benchmark.o
redis-benchmark.c: In function ‘_serverAssert’:
redis-benchmark.c:269:18: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  269 |     *((char*)-1) = 'x';
      |     ~~~~~~~~~~~~~^~~~~
    LINK redis-benchmark
    INSTALL redis-check-rdb
    INSTALL redis-check-aof

Hint: It's a good idea to run 'make test' ;)

make[1]: Leaving directory `/home/reas/Downloads/redis-6.2.6/src'

上述输出则通过。

可进入 src 文件夹执行。/redis-server 进行测试

(base) [root@reas src]# ./redis-server
65252:C 28 Dec 2021 10:23:02.533 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
65252:C 28 Dec 2021 10:23:02.533 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=65252, just started
65252:C 28 Dec 2021 10:23:02.533 # Warning: no config file specified, using the default config. In order to specify a config file use ./redis-server /path/to/redis.conf
65252:M 28 Dec 2021 10:23:02.534 * Increased maximum number of open files to 10032 (it was originally set to 1024).
65252:M 28 Dec 2021 10:23:02.534 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 6.2.6 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 65252
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

65252:M 28 Dec 2021 10:23:02.535 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
65252:M 28 Dec 2021 10:23:02.535 # Server initialized
65252:M 28 Dec 2021 10:23:02.535 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
65252:M 28 Dec 2021 10:23:02.535 * Ready to accept connections

测试

1
2
3


> redis-server
> redis-cli ping
# 正常输出 PONG

配置

可以通过修改 /etc/redis/redis.conf 文件或使用 CONFIG set 命令来修改配置

CONFIG SET 命令基本语法：

1
2


redis 127.0.0.1:6379> CONFIG SET CONFIG_SETTING_NAME NEW_CONFIG_VALUE
redis 127.0.0.1:6379> CONFIG SET loglevel "notice"

redis-cluster

需要注意要安装 redis 5.0 以上的版本, 可能需要采用源码安装的方式。

cluster 的构建:

因为redis集群最少需要三主三从，所以我们启动6个实例，

新建 /usr/local/redis-cluster 目录并进入

1
2
3


cd /usr/local
mkdir redis-cluster
cd redis-cluster

1
2
3
4


mkdir conf data log
cd data
mkdir redis-6301 redis-6302 redis-6303 redis-6304 redis-6305 redis-6306
cd ../conf

文件 redis-6301.conf 修改后的文件内容如下，接下来5个配置文件分别按照6302, 6303, 6304, 6305, 6306来配置：

pidfile /var/run/redis/redis-6301-server.pid
port 6301
logfile /usr/local/redis-cluster/log/redis-6301-server.log
dir /usr/local/redis-cluster/data/redis-6301
dbfilename dump-6301.rdb

requirepass foobared
cluster-enabled yes
cluster-config-file nodes-6301.conf
cluster-node-timeout 15000

进入 /usr/local/redis-cluster 启动实例

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


sudo /usr/local/bin/redis-server conf/redis-6301.conf
sudo /usr/local/bin/redis-server conf/redis-6302.conf
sudo /usr/local/bin/redis-server conf/redis-6303.conf
sudo /usr/local/bin/redis-server conf/redis-6304.conf
sudo /usr/local/bin/redis-server conf/redis-6305.conf
sudo /usr/local/bin/redis-server conf/redis-6306.conf
ps -ef | grep redis
root       663     1  0 13:52 ?        00:00:00 redis-server 127.0.0.1:6379
root      5928     1  0 14:57 ?        00:00:00 /usr/local/bin/redis-server 127.0.0.1:6301 [cluster]
root      5935     1  0 14:57 ?        00:00:00 /usr/local/bin/redis-server 127.0.0.1:6302 [cluster]
root      5942     1  0 14:57 ?        00:00:00 /usr/local/bin/redis-server 127.0.0.1:6303 [cluster]
root      5949     1  0 14:57 ?        00:00:00 /usr/local/bin/redis-server 127.0.0.1:6304 [cluster]
root      5956     1  0 14:57 ?        00:00:00 /usr/local/bin/redis-server 127.0.0.1:6305 [cluster]
root      5963     1  0 14:57 ?        00:00:00 /usr/local/bin/redis-server 127.0.0.1:6306 [cluster]

此时执行 tree 命令应该得到以下目录结构

.
├── conf
│   ├── redis-6301.conf
│   ├── redis-6302.conf
│   ├── redis-6303.conf
│   ├── redis-6304.conf
│   ├── redis-6305.conf
│   └── redis-6306.conf
├── data
│   ├── redis-6301
│   │   └── nodes-6301.conf
│   ├── redis-6302
│   │   └── nodes-6302.conf
│   ├── redis-6303
│   │   └── nodes-6303.conf
│   ├── redis-6304
│   │   └── nodes-6304.conf
│   ├── redis-6305
│   │   └── nodes-6305.conf
│   └── redis-6306
│       └── nodes-6306.conf
└── log
    ├── redis-6301-server.log
    ├── redis-6302-server.log
    ├── redis-6303-server.log
    ├── redis-6304-server.log
    ├── redis-6305-server.log
    └── redis-6306-server.log

9 directories, 18 files

现在每个实例是单独的，需要把它们连接到一起

1

redis-cli -a foobared --cluster create 127.0.0.1:6301 127.0.0.1:6302 127.0.0.1:6303 127.0.0.1:6304 127.0.0.1:6305 127.0.0.1:6306 --cluster-replicas 1

若此步骤出现没有参数 -cluter 的错误，注意安装的 redis 版本问题 (需要 > 5)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53


redis-cli -a foobared --cluster create 127.0.0.1:6301 127.0.0.1:6302 127.0.0.1:6303 127.0.0.1:6304 127.0.0.1:6305 127.0.0.1:6306 --cluster-replicas 1

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:6305 to 127.0.0.1:6301
Adding replica 127.0.0.1:6306 to 127.0.0.1:6302
Adding replica 127.0.0.1:6304 to 127.0.0.1:6303
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 5d60bdad15a6f8ceec188a1081e9381f181a5c5e 127.0.0.1:6301
   slots:[0-5460] (5461 slots) master
M: 8c814d4a32763d47723398fcf8f596d7b6340afc 127.0.0.1:6302
   slots:[5461-10922] (5462 slots) master
M: 7bc53b512772c3a1df3217facca283ff9564d32d 127.0.0.1:6303
   slots:[10923-16383] (5461 slots) master
S: b39bdb5b6e720e9fbedd43e58b57661910dcc3d7 127.0.0.1:6304
   replicates 5d60bdad15a6f8ceec188a1081e9381f181a5c5e
S: 5fef2bedd430bf86cdff63cb2f852aeb21e1b18f 127.0.0.1:6305
   replicates 8c814d4a32763d47723398fcf8f596d7b6340afc
S: 0123b160087743a5296807145b426d9b9cefcf21 127.0.0.1:6306
   replicates 7bc53b512772c3a1df3217facca283ff9564d32d
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 127.0.0.1:6301)
M: 5d60bdad15a6f8ceec188a1081e9381f181a5c5e 127.0.0.1:6301
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: b39bdb5b6e720e9fbedd43e58b57661910dcc3d7 127.0.0.1:6304
   slots: (0 slots) slave
   replicates 5d60bdad15a6f8ceec188a1081e9381f181a5c5e
M: 7bc53b512772c3a1df3217facca283ff9564d32d 127.0.0.1:6303
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 5fef2bedd430bf86cdff63cb2f852aeb21e1b18f 127.0.0.1:6305
   slots: (0 slots) slave
   replicates 8c814d4a32763d47723398fcf8f596d7b6340afc
M: 8c814d4a32763d47723398fcf8f596d7b6340afc 127.0.0.1:6302
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 0123b160087743a5296807145b426d9b9cefcf21 127.0.0.1:6306
   slots: (0 slots) slave
   replicates 7bc53b512772c3a1df3217facca283ff9564d32d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

redis 集群有三种模式

主从复制模式:

数据库分为两类，一类是主数据库（master），另一类是从数据库(slave）。主数据库可以进行读写操作，当写操作导致数据变化时会自动将数据同步给从数据库。而从数据库一般是只读的，并接受主数据库同步过来的数据。一个主数据库可以拥有多个从数据库，而一个从数据库只能拥有一个主数据库。 2. Sentinel（哨兵）模式:

第一种主从同步/复制的模式，当主服务器宕机后，需要手动把一台从服务器切换为主服务器，这就需要人工干预，费事费力，还会造成一段时间内服务不可用。这不是一种推荐的方式，更多时候，我们优先考虑哨兵模式。

哨兵模式通过发送命令，让 Redis 服务器返回监控其运行状态，包括主服务器和从服务器；当哨兵监测到 master 宕机，会自动将 slave 切换成 master ，然后通过发布订阅模式通知其他的从服务器，修改配置文件，让它们切换主机；然而一个哨兵进程对Redis服务器进行监控，也可能会出现问题，为此，我们可以使用多个哨兵进行监控。各个哨兵之间还会进行监控，这样就形成了多哨兵模式。

优点：哨兵模式是基于主从模式的，所有主从的优点，哨兵模式都具有。主从可以自动切换，系统更健壮，可用性更高(可以看作自动版的主从复制)。缺点：Redis较难支持在线扩容，在集群容量达到上限时在线扩容会变得很复杂。

Cluster 集群模式: 在 redis3.0上加入了 Cluster 集群模式，实现了 Redis 的分布式存储，也就是说每台 Redis 节点上存储不同的内容。

Redis 集群有16384 个哈希槽，每个 key 通过 CRC16 校验后对 16384 取模来决定放置哪个槽。集群的每个节点负责一部分hash槽，举个例子，比如当前集群有3个节点，那么：

节点 A 包含 0 到 5460 号哈希槽节点 B 包含 5461 到 10922 号哈希槽节点 C 包含 10923 到 16383 号哈希槽

在 Redis 的每一个节点上，都有这么两个东西，一个是插槽（slot），它的的取值范围是：0-16383。还有一个就是 cluster，可以理解为是一个集群管理的插件。当我们的存取的 Key到达的时候，Redis 会根据 CRC16 的算法得出一个结果，然后把结果对 16384 求余数，这样每个 key 都会对应一个编号在 0-16383 之间的哈希槽，通过这个值，去找到对应的插槽所对应的节点，然后直接自动跳转到这个对应的节点上进行存取操作。

python

Connect to Redis

1
2
3
4
5
6


import redis

r = redis.Redis(
    host='hostname',
    port=port,
    password='password')

Once connected to Redis, you can read and write data with Redis command functions.

The following code snippet assigns the value bar to the Redis key foo, reads it back, and prints it:

1
2
3


r.set('foo', 'bar')
value = r.get('foo')
print(value)

The redis-py client natively supports SSL.

Use the SSLConnection class or instantiate your connection pool using a rediss:// URL and the from_url method, like so:

1
2
3
4
5
6


r = redis.Redis( url='rediss://:password@hostname:port/0',
    password='password',
    ssl_keyfile='path_to_keyfile',
    ssl_certfile='path_to_certfile',
    ssl_cert_reqs='required',
    ssl_ca_certs='path_to_ca_cert')

redis + cloudpickle

server

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


import redis
import cloudpickle as pk

C = 40
def func(data):
  return data + C


class A:
  def __init__(self):
    self.count = 3
  def increase(self):
    self.count = func(self.count)
  def show(self):
    print("count = ", self.count)


serialized_A = pk.dumps(A)
gcs = redis.Redis(host='127.0.0.1', port = 6379)
gcs.set('A', serialized_A)

client

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


import redis
import cloudpickle as pk

server = redis.Redis(host = "127.0.0.1", port = 6379)

serialied_A = server.get('A')
A = pk.loads(serialied_A)
a = A()
a.show()
a.increase()
a.show()

c++

使用C/C++调用进行redis 的连接，最常见的是 hiredis，但是它本身不提供集群的便利访问，像 MGET，MSET 这种操作不太支持。我们可以使用 hiredis-vip, 或者官方推荐使用的 hiredis-cluster , redis-plus-plus

hiredis

安装

$ git clone git@github.com:redis/hiredis.git
$ cd hiredis
$ make
$ make install

make install 的输出如下:

mkdir -p /usr/local/include/hiredis /usr/local/include/hiredis/adapters /usr/local/lib
cp -pPR hiredis.h async.h read.h sds.h /usr/local/include/hiredis
cp -pPR adapters/*.h /usr/local/include/hiredis/adapters
cp -pPR libhiredis.so /usr/local/lib/libhiredis.so.0.13
cd /usr/local/lib && ln -sf libhiredis.so.0.13 libhiredis.so
cp -pPR libhiredis.a /usr/local/lib
mkdir -p /usr/local/lib/pkgconfig
cp -pPR hiredis.pc /usr/local/lib/pkgconfig

复制动态链接库到lib：

1
2
3


cp libhiredis.so /usr/lib64
cp libhiredis.so /usr/lib
/sbin/ldconfig

连接

client_connect.cc

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


#include <cstdio>
#include <iostream>
#include <hiredis/hiredis.h>
using namespace std;
int main()
{
    redisContext* conn = redisConnect("127.0.0.1", 6303);
    if(conn->err)   printf("connection error:%s\n", conn->errstr);
    // 提供密码
    redisReply* reply = static_cast<redisReply*>(redisCommand(conn, "AUTH %s", "foobared"));
    reply = static_cast<redisReply*>(redisCommand(conn, "set foo 1234"));
    freeReplyObject(reply);

    reply = static_cast<redisReply*>(redisCommand(conn, "get foo"));

    printf("%s\n", reply->str);
    freeReplyObject(reply);

    redisFree(conn);

    return 0;
}

这里要注意，对于序列化的结构数据，string中保存的是二进制数据，c_str()方法返回的c字符串被二进制0值截断，会造成数据不完整，而hiredis提供%b作为格式化二进制的方法，需要提供起始地址和长度：

reply = (redisReply *)redisCommand(redisContext, "SET %s %b", key.c_str(), value.data(), value.length())；//写长byte流

编译

g++ client_connect.cc -o client -lhiredis

在使用上发现如果连接的是 6301, 则会输出 MOVED 12182 127.0.0.1:6303。

redis-plus-plus

官方的默认安装路径:

头文件在 /usr/local/include/sw/redis++
链接库文件在 /usr/local/lib/libredis++.a /usr/local/lib/libredis++.so

g++ -std=c++17 rediscc.cpp -o main /usr/local/lib/libredis++.a /usr/local/lib/libhiredis.a -pthread

使用

连接

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49


ConnectionOptions connection_options;
connection_options.host = "127.0.0.1";  // Required.
connection_options.port = 6666; // Optional. The default port is 6379.
connection_options.password = "auth";   // Optional. No password by default.
connection_options.db = 1;  // Optional. Use the 0th database by default.

// Optional. Timeout before we successfully send request to or receive response from redis.
// By default, the timeout is 0ms, i.e. never timeout and block until we send or receive successfuly.
// NOTE: if any command is timed out, we throw a TimeoutError exception.
connection_options.socket_timeout = std::chrono::milliseconds(200);

// Connect to Redis server with a single connection.
Redis redis1(connection_options);

ConnectionPoolOptions pool_options;
pool_options.size = 3;  // Pool size, i.e. max number of connections.

// Optional. Max time to wait for a connection. 0ms by default, which means wait forever.
// Say, the pool size is 3, while 4 threds try to fetch the connection, one of them will be blocked.
pool_options.wait_timeout = std::chrono::milliseconds(100);

// Optional. Max lifetime of a connection. 0ms by default, which means never expire the connection.
// If the connection has been created for a long time, i.e. more than `connection_lifetime`,
// it will be expired and reconnected.
pool_options.connection_lifetime = std::chrono::minutes(10);

// Connect to Redis server with a connection pool.
Redis redis2(connection_options, pool_options);

// Single connection to the given host and port.
Redis redis1("tcp://127.0.0.1:6666");

// Use default port, i.e. 6379.
Redis redis2("tcp://127.0.0.1");

// Connect to Redis with password, and default port.
Redis redis3("tcp://pass@127.0.0.1");

// Connect to Redis and select the 2nd (db number starts from 0) database.
Redis redis4("tcp://127.0.0.1:6379/2");

// Set keep_alive option to true with query string.
Redis redis5("tcp://127.0.0.1:6379/2?keep_alive=true");

// Set socket_timeout to 50 milliseconds, and connect_timeout to 1 second with query string.
Redis redis6("tcp://127.0.0.1?socket_timeout=50ms&connect_timeout=1s");

// Connect to Unix Domain Socket.
Redis redis7("unix://path/to/socket");

Send Command to Redis Server

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


// Delete a single key.
long long Redis::del(const StringView &key);

// Delete a batch of keys: [first, last).
template <typename Input>
long long Redis::del(Input first, Input last);

// Delete keys in the initializer_list.
template <typename T>
long long Redis::del(std::initializer_list<T> il);

issue

kill redis-server

/etc/init.d/redis-server stop

useful command

get all keys

redis-cli --scan --pattern '*'

redis-cli KEYS '*'

delete everything in redis

redis-cli flushall  # Deletes all keys from all databases.
redis-cli flushdb  # Deletes all keys from the connection's current database.