Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysis

arXiv cs.AI · 6 hours ago

Researchers introduced a framework for reinforcement learning in switching non-stationary environments where hidden Markov chains govern transitions between different MDPs, proving that temporal-difference learning, policy iteration, and Q-learning converge to optimal solutions despite persistent non-stationarity. The theoretical analysis shows that long-term effects of switching are equivalent to stationary dynamics parameterized by the hidden chain's stationary distribution. The approach was validated on a wireless communication network model with time-varying channel noise.