Redis prefix causes ever-growing redis cache size

[juliusknorr]

Whenever Nextcloud or an app is updated the Memcache prefix is changing and old stored keys without an expiry would just stay around forever. Since the prefix is also hashed and not easily genera table due to the app version list in place there is also currently no good way to clean them up manually.

server/lib/private/Server.php

Lines 712 to 726 in 7817845

	if (!$config->getSystemValueBool('log_query')) {
	$v = \OC_App::getAppVersions();
	} else {
	// If the log_query is enabled, we can not get the app versions
	// as that does a query, which will be logged and the logging
	// depends on redis and here we are back again in the same function.
	$v = [
	'log_query' => 'enabled',
	];
	}
	$v['core'] = implode(',', \OC_Util::getVersion());
	$version = implode(',', $v);
	$instanceId = \OC_Util::getInstanceId();
	$path = \OC::$SERVERROOT;
	$prefix = md5($instanceId . '-' . $version . '-' . $path);

Maybe we could consider having a predictable global prefix, so we could also introduce a nightly job to delete outdated keys?

On the other hand deleting keys by prefix is a rather heavy operation as it requires a KEYS and then a DEL for each key on the Redis side, so maybe a default expiry of some weeks would also be something that we could consider.

cc @nextcloud/server-backend

[come-nc]

maybe a default expiry of some weeks would also be something that we could consider.

Instinctively I would go for this, what is the usecase for non-expiring entries in redis?

[juliusknorr]

what is the usecase for non-expiring entries in redis?

At least I'm not aware that we have any, at least not if we consider an expiry > 1 day. In the end I think our memcache abstraction was always considered volatile.

[juliusknorr]

Note that one may configure redis to delete keys, but that only happens if the redis server reaches the configured memory limit then: https://redis.io/docs/manual/eviction/#eviction-policies

[joshtrichards]

Related: #38562