Viet-Anh on Software Logo

What is: Experience Replay?

Year1993
Data SourceCC BY-SA - https://paperswithcode.com

Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, e_t=(s_t,a_t,r_t,s_t+1)e\_{t} = \left(s\_{t}, a\_{t}, r\_{t}, s\_{t+1}\right) in a data-set D=e_1,,e_ND = e\_{1}, \cdots, e\_{N} , pooled over many episodes into a replay memory. We then usually sample the memory randomly for a minibatch of experience, and use this to learn off-policy, as with Deep Q-Networks. This tackles the problem of autocorrelation leading to unstable training, by making the problem more like a supervised learning problem.

Image Credit: Hands-On Reinforcement Learning with Python, Sudharsan Ravichandiran