This paper investigates the problem of $H_\infty$ state estimation of delayed recurrent memristive neural networks (DRMNNs) with both continuous-time and discrete-time cases. By utilizing Lyapunov-Krasovskii functional (LKF) and linear matrix inequalities (LMIs), two criterions are provided to guarantee the asymptotically stable of the estimation error systems with a $H_\infty$ performance. The connection weight parameters of DRMNNs are dealed with logical switching signals, which greatly reduces the computational complexity. The given conditions can be easily checked by solving LMIs, the obtained theoretical results are supported demonstrated by two numerical examples.