Environment
Description
- Problem
With a per-pod quota enabled (config/storage/backend/pod-quota-file.json),
every write — including writes to CSS-internal storage under /.internal/ —
goes through the quota chain:
QuotaValidator calls getAvailableSpace before and after every write
and createQuotaGuard mid-stream.
- Default
PodQuotaStrategy.createQuotaGuard calls getAvailableSpace (a
full pod walk) per stream chunk: chunks × O(pod size).
getAvailableSpace does pod discovery (searchPimStorage) + a full
recursive size walk via FileSizeReporter when the pod isn't cached;
FileSizeReporter also accumulates large dirent arrays in memory.
-
Observed impact (production)
The IDP KeyValueStorage (AuthorizationCode store at
/.internal/idp/adapter/) is backed by
ResourceStore → ResourceStore_Backend → FileDataAccessor, so login writes
hit the same chain:
[WrappedExpiringReadWriteLocker] error: Lock expired after 6000ms on
/.internal/idp/adapter/AuthorizationCode/
[WrappedExpiringStorage] error: ... Failed to remove expired entries ...
On a sufficiently large pod, any internal write exceeds the 6s lock expiry —
logins and refresh flows fail intermittently.
-
Root cause
ignoreFolders: ["^/\\.internal$"] only excludes .internal from the size
reporter's walk — it does not stop QuotaValidator from validating
internal writes. CSS-internal storage shares the user accessor chain with pod
data.
-
Suggested fix
Exempt /.internal/* at the quota strategy/validator level (return unlimited
space without pod discovery/walk), not only in the reporter's walk excludes —
as implemented in solid-contrib/pivot (FastQuotaStrategy.getAvailableSpace
+ delta-accessor skip).
Environment
Description
With a per-pod quota enabled (
config/storage/backend/pod-quota-file.json),every write — including writes to CSS-internal storage under
/.internal/—goes through the quota chain:
QuotaValidatorcallsgetAvailableSpacebefore and after every writeand
createQuotaGuardmid-stream.PodQuotaStrategy.createQuotaGuardcallsgetAvailableSpace(afull pod walk) per stream chunk:
chunks × O(pod size).getAvailableSpacedoes pod discovery (searchPimStorage) + a fullrecursive size walk via
FileSizeReporterwhen the pod isn't cached;FileSizeReporteralso accumulates large dirent arrays in memory.Observed impact (production)
The IDP
KeyValueStorage(AuthorizationCode store at/.internal/idp/adapter/) is backed byResourceStore → ResourceStore_Backend → FileDataAccessor, so login writeshit the same chain:
[WrappedExpiringReadWriteLocker] error: Lock expired after 6000ms on
/.internal/idp/adapter/AuthorizationCode/
[WrappedExpiringStorage] error: ... Failed to remove expired entries ...
On a sufficiently large pod, any internal write exceeds the 6s lock expiry —
logins and refresh flows fail intermittently.
Root cause
ignoreFolders: ["^/\\.internal$"]only excludes.internalfrom the sizereporter's walk — it does not stop
QuotaValidatorfrom validatinginternal writes. CSS-internal storage shares the user accessor chain with pod
data.
Suggested fix
Exempt
/.internal/*at the quota strategy/validator level (return unlimitedspace without pod discovery/walk), not only in the reporter's walk excludes —
as implemented in solid-contrib/pivot (
FastQuotaStrategy.getAvailableSpace+ delta-accessor skip).