C# redis集群批量操作之slot计算出16384个字符串

引入一个大家都用的到的需求来说吧。

需求:要在三主三从的redis集群,存入数据,会对数据进行批量删除操作,数据要求要在redis集群负载均衡。

思路:

1.存入数据好办

1 var connect = ConnectionMultiplexer.Connect(redisConn);
2 var redisDb = connect.GetDatabase();
3 var res1 = redisDb.StringSet("1", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
4 var res2 = redisDb.StringSet("1111", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));

2.批量删除直接异常

1 redisDb.KeyDelete(new RedisKey[] { "1", "1111" });

Ex:“Multi-key operations must involve a single slot; keys can use ‘hash tags’ to help this, i.e. ‘{/users/12345}/account’ and ‘{/users/12345}/contacts’ will always be in the same slot”

 

3.查到异常是因为redis hash slot 机制导致的,什么是 hash slot?
hash slot 介绍:https://redis.io/topics/cluster-tutorial

4.加上hash slot 字符串,让key进入同一个slot
var res1 = redisDb.StringSet(“{myslot}key1”, DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
var res2 = redisDb.StringSet(“{myslot}key2”, DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
redisDb.KeyDelete(new RedisKey[] { “{myslot}key1”, “{myslot}key2” });
能进行批量操作,但是都被分配到了同一台服务器上的同一个槽点,不负载均衡。

5.如何负载均衡, 让Key分布到各个服务器,并且可以批量操作?
如果知道每个槽点对应的字符串,key可以按照算法计算出自己对应的字符串,加上后,就可以进行分组批量增删改操作。

6.hash slot 计算方法
HASH_SLOT = CRC16(key) mod 16384 (crc16-XMODEM)
介绍 :https://redis.io/topics/cluster-spec

7.net core 计算出16384个slot 字符串 算法例子和 结果模板

 1         static void Main(string[] args)
 2         {
 3             var data = new Dictionary<int, string>();
 4             var i = 0;
 5             while (data.Keys.Count < 16384)
 6             {
 7                 var temp = i.ToString("X");
 8                 var value = Crc16(Encoding.UTF8.GetBytes(temp)) % 16384;
 9                 data[int.Parse(value.ToString())] = temp;
10                 i++;
11 
12             }
13             var sb = new StringBuilder();
14             foreach (var item in data.OrderBy(s => s.Key))
15             {
16                 var temp = $"slot num:{item.Key}   str:{item.Value} rn";
17                 Console.WriteLine(temp);
18                 sb.Append(temp);
19             }
20             File.WriteAllText("data.txt", sb.ToString());
21             Console.ReadLine();
22         }
23         private static ushort Crc16(byte[] bytes)
24         {
25             ushort poly = 0x1021;
26             ushort[] table = new ushort[256];
27             ushort initialValue = 0x0;
28             ushort temp, a;
29             ushort crc = initialValue;
30             for (int i = 0; i < table.Length; ++i)
31             {
32                 temp = 0;
33                 a = (ushort)(i << 8);
34                 for (int j = 0; j < 8; ++j)
35                 {
36                     if (((temp ^ a) & 0x8000) != 0)
37                         temp = (ushort)((temp << 1) ^ poly);
38                     else
39                         temp <<= 1;
40                     a <<= 1;
41                 }
42                 table[i] = temp;
43             }
44             for (int i = 0; i < bytes.Length; ++i)
45             {
46                 crc = (ushort)((crc << 8) ^ table[((crc >> 8) ^ (0xff & bytes[i]))]);
47             }
48             return crc;
49         }

  data.txt 文件下载

8.校验算出来的字符串 对应 的slot位置 是否正确

C# redis集群批量操作之slot计算出16384个字符串插图

 

9.批量设置和批量删除方法
假定三主三从,那么三台服务器,取九个slot字符串,这九个是均分的位置(均分利于集群扩展)。即16384/10=1638 1638是第一位,1638*2是第二位,以此类推取字符串
共九个[“1A73F”,”18B13″,”1AAD3″,”184FF”,”143BF”,”18413″,”17B8D”,”18BFF”,”1B1C4″]
先分组,再批量插入,再批量删除

C# redis集群批量操作之slot计算出16384个字符串插图(1)

 1 try
 2             {
 3                 var redisConn = "{集群地址}";
 4                 var connect = ConnectionMultiplexer.Connect(redisConn);
 5                 var redisDb = connect.GetDatabase();
 6                 var res1 = redisDb.StringSet("1", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
 7                 var res2 = redisDb.StringSet("1111", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
 8                 redisDb.KeyDelete(new RedisKey[] { "1", "1111" });
 9 
10 
11                 var redisSlotKeyList = new string[] { "1A73F", "18B13", "1AAD3", "184FF", "143BF", "18413", "17B8D", "18BFF", "1B1C4" };
12                 var userIdArray = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 };
13                 //group
14                 var dic = new Dictionary<int, Dictionary<RedisKey,RedisValue>>();
15                 foreach (var userId in userIdArray)
16                 {
17                     var index = userId % redisSlotKeyList.Length;
18                     var slotKey = redisSlotKeyList[index];
19                     var redisKey = $"{{{slotKey}}}test_{userId}";
20                     Console.WriteLine($"{ redisKey}  {userId}");
21                     if (dic.ContainsKey(index))
22                     {
23                         dic[index].Add(redisKey, DateTime.Now.ToLongTimeString());
24                     }
25                     else 
26                     {
27                         dic[index] = new Dictionary<RedisKey, RedisValue> { { new RedisKey(redisKey), new RedisValue("values") } };
28                     }
29                 }
30 
31                 //set
32                 foreach (var item in dic)
33                 {
34                     var addRes = redisDb.StringSet(item.Value.ToArray());
35                     Console.WriteLine(addRes);
36                 }
37 
38 
39                 //delete
40                 foreach (var item in dic)
41                 {
42                     var deleteRes = redisDb.KeyDelete(item.Value.Keys.ToArray());
43                     Console.WriteLine(deleteRes);
44                 }
45 
46 
47 
48             }
49             catch (Exception ex)
50             {
51                 throw ex;
52             }

View Code

 

10.注意点
集群的分割slot配置不一定的均分的,提前先查看,命令:cluster nodes。 查看之后再根据实际情况取slot string

 

原创文章,作者:迷途资源,如若转载,请注明出处:https://www.mipng.com/439.html

这些信息可能会帮助到你: 下载帮助 | 报毒说明 | 进站必看

修改版本安卓软件,加群提示为修改者自留,非本站信息,注意鉴别

发表评论

邮箱地址不会被公开。 必填项已用*标注