MasterConnectionPool no available Redis entries

背景

线上报了 Rocketmq 消息堆积的报警，排查下来，发现是因为消费者那边调用 redis 导致的，具体的错误如下：

org.redisson.client.RedisConnectionException: MasterConnectionPool no available Redis entries.  Hosts disconnected due to `failedAttempts` limit reached: [xxxxx:6379]
        at org.redisson.connection.pool.ConnectionPool.get(ConnectionPool.java:196) ~[redisson-2.10.3.jar:na]
        at org.redisson.connection.pool.MasterConnectionPool.get(MasterConnectionPool.java:31) ~[redisson-2.10.3.jar:na]
        at org.redisson.connection.MasterSlaveEntry.connectionWriteOp(MasterSlaveEntry.java:417) ~[redisson-2.10.3.jar:na]
        at org.redisson.connection.MasterSlaveConnectionManager.connectionWriteOp(MasterSlaveConnectionManager.java:706) ~[redisson-2.10.3.jar:na]
        at org.redisson.command.CommandAsyncService.async(CommandAsyncService.java:506) ~[redisson-2.10.3.jar:na]
        at org.redisson.command.CommandAsyncService$8.run(CommandAsyncService.java:585) ~[redisson-2.10.3.jar:na]
        at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663) ~[netty-common-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738) ~[netty-common-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466) ~[netty-common-4.1.15.Final.jar:4.1.15.Final]
        at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_192]

 host xxxxx:6379 disconnected due to failedAttempts=3 limit reached
org.redisson.client.RedisTimeoutException: Command execution timeout for xxxxx:6379
        at org.redisson.client.RedisConnection$2.run(RedisConnection.java:212) ~[redisson-2.10.3.jar:na]
        at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) ~[netty-common-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120) ~[netty-common-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-common-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) [netty-common-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) [netty-common-4.1.15.Final.jar:4.1.15.Final]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_192]

分析

经过排查，发现这个是 RedissonClient 的一个 bug，官方已经在 2.10.4 版本修复了，但是我们系统用的版本是 2.10.3，官方问题地址：Issue #1092 · redisson/redisson

解决

对 redisson 的版本进行升级