Optimizing cloud economics with linear elastic caching

Google Research · 3 weeks ago

Google researchers developed linear elastic caching that dynamically adjusts cache size using lightweight machine learning to optimize the trade-off between memory costs and cache misses. In production testing on Spanner, the approach reduced memory usage by 15.5% and total cost of ownership by approximately 5% while increasing cache misses by only 5.5%. The system frames cache eviction as a ski rental problem where data can be kept in expensive RAM or evicted to slower storage, with a shallow decision tree predicting optimal retention times for each data page.