RedisReactiveHealthIndicator should provide cluster details in cluster mode

[mhyeon-lee]

RedisReactiveHealthIndicator health is down with cluster mode.

Check different property if ReactiveRedisClusterConnection

private Health up(Health.Builder builder, Properties info) {
		return builder.up()
				.withDetail(RedisHealthIndicator.VERSION, info.getProperty(RedisHealthIndicator.REDIS_VERSION)).build();
	}

[wilkinsona]

@mp911de What's the equivalent of org.springframework.data.redis.connection.RedisClusterCommands.clusterGetClusterInfo() in the reactive API please?

[mp911de]

Redis Cluster commands (CLUSTER INFO) are not exposed through a reactive API. I filed DATAREDIS-1150 to provide a reactive variant of RedisClusterCommands.

[wilkinsona]

Thanks, @mp911de. Will that sneak into a Neumann SR or be a new feature in Ockham? If it's the latter, is there anything that we can do in the meantime?

[mp911de]

I need to discuss this with the team. Introducing new API in service releases is always subject to cause a certain degree of confusion. There's currently no workaround.

[marcus-bcl]

I've just hit this issue since upgrading to 2.3.0.RELEASE and have had to temporarily disable the Redis health endpoint, which isn't ideal (management.health.redis.enabled=false).

As a workaround until the change is implemented in spring-data-redis, would falling back to the non-reactive RedisHealthIndicator work? Something like:

@SpringBootApplication(exclude = RedisReactiveHealthContributorAutoConfiguration.class)

Appreciate any info you can provide - thanks!

[wilkinsona]

With thanks to @mp911de and the Data team, they've agreed to add the missing cluster-related APIs in a Spring Data Neumann service release. I believe it's currently slated for inclusion in Neumann-SR1 so, all being well, we should be able to fix this in Boot 2.3.1.

[andreacomo]

Waiting for Boot version 2.3.1, my workaround is:

1. disable default health check (management.health.redis.enabled=false)
2. Reimplement RedisReactiveHealthIndicator with a fixed version of up method like this:

private Health up(Health.Builder builder, Properties info, ReactiveRedisConnection connection) {
        if (connection instanceof ReactiveRedisClusterConnection) {
            List<Map<String, String>> details = getDetails(info);
            if (details.isEmpty()) {
                return builder.outOfService()
                        .build();
            } else {
                return builder.up()
                        .withDetail("nodes", details)
                        .build();
            }
        } else {
            return builder.up()
                    .withDetail("version", info.getProperty("redis_version"))
                    .build();
        }
    }

    private List<Map<String, String>> getDetails(Properties info) {
        return info.keySet().stream()
                        .map(String.class::cast)
                        .map(k -> k.substring(0, k.lastIndexOf(".")))
                        .distinct()
                        .sorted()
                        .map(node -> Map.of(
                                "node", node,
                                "redis_version", info.getProperty(node + ".redis_version"),
                                "role", info.getProperty(node + ".role"),
                                "uptime_in_days", info.getProperty(node + ".uptime_in_days")
                                )
                        )
                        .collect(Collectors.toList());
    }

Now my health check is UP.
Hope this can help

[christopher-nash]

We've experienced this same thing going from 2.2.7 to 2.2.8.

[wilkinsona]

@christopher-nash As far as I can tell, nothing has changed in this area between 2.2.7 and 2.2.8. If something was working for you in 2.2.7 and stopped working in 2.2.8, I don't think a fix for this issue will help. If you can reproduce the problem that you saw, please open a new issue with a minimal sample that works with 2.2.7 and fails with 2.2.8.

[wilkinsona]

This is no longer blocked as the changes have been merged into Spring Data Neumann and Ockham (2020.0.0).

[edwardsre]

@wilkinsona It looks like the properties returned in the cluster info in 2.2.8 are prefixed with the cluster node IP and port (e.g. 127.0.0.1:7001.redis_version) so getting the redis-version property returns null. This is failing on the same line indicated in this issue description.

In 2.2.7.RELEASE, the properties do not have the ip and port prefix.

[wilkinsona]

I can't explain that. The only Redis-related change in 2.2.8 was an upgrade to Spring Data Moore-SR8. As I said above, if you can reproduce the problem that you saw, please open a new issue with a minimal sample that works with 2.2.7 and fails with 2.2.8.

[mp911de]

Spring Data Redis returns in version 2.2.8 a different connection type which reports Redis stats prefixed with host and port. See #22061 for further reference.

[scottfrederick]

After some discussion, we've decided that we should not change the health response in a Spring Boot 2.3 patch release, but only change it in the 2.4 release. Boot 2.3 and earlier will continue to provide the version field in the reactive Redis health response in both clustered and non-clustered configurations. In 2.4 the reactive and non-reactive health responses are brought into alignment, with cluster details provided when Redis is running in a clustered configuration.