Reinforcement Learning-Driven Dynamic Cache Management with Hotspot-Aware Replacement

Abstract

This paper focuses on the problem of cache management in dynamic environments. It proposes a hotspot-aware dynamic cache replacement strategy based on reinforcement learning. The goal is to improve the intelligence and efficiency of cache systems under complex access scenarios. The method models service nodes, content requests, and access features in the cache system as a state space. It designs a multi-dimensional state representation that incorporates access frequency, content popularity, recent access time, and content lifespan. A policy network is introduced to score and select each cache replacement action. Under the reinforcement learning framework, the model dynamically evaluates cache hit behavior through a reward function. The policy is optimized to maximize long-term benefits. To validate the effectiveness of the proposed approach, extensive experiments are conducted on a real-world YouTube Cache dataset. The evaluation covers several key metrics, including hit rate, system latency, misreplacement ratio, and traffic control. Sensitivity analyses are also performed across different cache capacities and parameter settings. The experimental results show that the proposed strategy demonstrates superior adaptability and robustness under various conditions. It significantly outperforms traditional replacement policies and existing learning-based methods. This study not only enhances the decision-making ability of cache replacement strategies in real-world complex environments but also highlights the potential of reinforcement learning mechanisms for structural optimization in caching systems.