RedisCluster

1 .Redis Cluster a pragmatic approach to distribution Source: Redis .io

2 .Redis Cluster a pragmatic approach to distribution Source: Redis .io

3 .What nodes talk about? PING : are you ok dude? Im master for XYZ hash slots. Config is FF89X1JK Gossip : this are info about other nodes Im in touch with: A replies to my ping, I think its state is OK. B is idle, I guess its having problems but I need some ACK. PONG : Sure Im ok dude! Im master for XYZ hash slots. Config is FF89X1JK Gossip : I want to share with you some info about random nodes: C and D are fine and replied in time. But B is idle for me as well! IMHO its down!.

4 .Hash slots keyspace is divided into 4096 hash slots. But in this example well assume they are just ten, from 0 to 9

5 .Master and Slave nodes Nodes are all connected and functionally equivalent, but actually there are two kind of nodes: slave and master nodes:

6 .Redundancy In the example there are two replicas per every master node, so up to two random nodes can go down without issues. Working with two nodes down is guaranteed, but in the best case the cluster will continue to work as long as there is at least one node for every hash slot .

7 .What this means so far? Every key only exists in a single instance, plus N replicas that will never receive writes. So there is no merge, nor application-side inconsistency resolution . The price to pay is not resisting to net splits that are bigger than replicas-per-hashslot nodes down. Master and Slave nodes use the Redis Replication you already know. Every physical server will usually hold multiple nodes, both slaves and masters, but the redis-trib cluster manager program will try to allocate slaves and masters so that the replicas are in different physical servers.

8 .Client requests - dummy client 1. Client => A: GET foo 2. A => Client: -MOVED 8 192.168.5.21:6391 Client => B: GET foo B => Client: "bar" -MOVED 8 ... this error means that hash slot 8 is located at the specified IP/port, and the client should reissue the query there.

9 .Client requests - smart client Client => A: CLUSTER HINTS A => Client: ... a map of hash slots -> nodes Client => B: GET foo B => Client: "bar"

10 .Client requests Dummy, single-connection clients, will work with minimal modifications to existing client code base. Just try a random node among a list, then reissue the query if needed. Smart clients will take persistent connections to many nodes, will cache hashslot -> node info, and will update the table when they receive a -MOVED error. This schema is always horizontally scalable, and low latency if the clients are smart. Especially in large clusters where clients will try to have many persistent connections to multiple nodes, the Redis client object should be shared.

11 .Client requests Dummy, single-connection clients, will work with minimal modifications to existing client code base. Just try a random node among a list, then reissue the query if needed. Smart clients will take persistent connections to many nodes, will cache hashslot -> node info, and will update the table when they receive a -MOVED error. This schema is always horizontally scalable, and low latency if the clients are smart. Especially in large clusters where clients will try to have many persistent connections to multiple nodes, the Redis client object should be shared.

12 .Re-sharding - moving data All the new keys for slot 7 will be created / updated in D. All the old keys in C will be moved to D by redis-trib using the MIGRATE command. MIGRATE is an atomic command, it will transfer a key from C to D, and will remove the key in C when we get the OK from D. So no race is possible. p.s. MIGRATE is an exported command. Have fun... Open problem: ask C the next key in hash slot N, efficiently.

13 .Re-sharding - moving data All the new keys for slot 7 will be created / updated in D. All the old keys in C will be moved to D by redis-trib using the MIGRATE command. MIGRATE is an atomic command, it will transfer a key from C to D, and will remove the key in C when we get the OK from D. So no race is possible. p.s. MIGRATE is an exported command. Have fun... Open problem: ask C the next key in hash slot N, efficiently.

14 .Fault tolerance All nodes continuously ping other nodes... A node marks another node as possibly failing when there is a timeout longer than N seconds. Every PING and PONG packet contain a gossip section: information about other nodes idle times, from the point of view of the sending node.

15 .Fault tolerance All nodes continuously ping other nodes... A node marks another node as possibly failing when there is a timeout longer than N seconds. Every PING and PONG packet contain a gossip section: information about other nodes idle times, from the point of view of the sending node.

16 .Fault tolerance All nodes continuously ping other nodes... A node marks another node as possibly failing when there is a timeout longer than N seconds. Every PING and PONG packet contain a gossip section: information about other nodes idle times, from the point of view of the sending node.

17 .Fault tolerance All nodes continuously ping other nodes... A node marks another node as possibly failing when there is a timeout longer than N seconds. Every PING and PONG packet contain a gossip section: information about other nodes idle times, from the point of view of the sending node.