Data assimilation (DA) plays a pivotal role in diverse applications, ranging from climate predictions and weather forecasts to trajectory planning for autonomous vehicles. A prime example is the widely used ensemble Kalman filter (EnKF), which relies on linear updates to minimize variance among the ensemble of forecast states. Recent advancements have seen the emergence of deep learning approaches in this domain, primarily within a supervised learning framework. However, the adaptability of such models to untrained scenarios remains a challenge. In this study, we introduce a novel DA strategy that utilizes reinforcement learning (RL) to apply state corrections using full or partial observations of the state variables. Our investigation focuses on demonstrating this approach to the chaotic Lorenz ’63 system, where the agent’s objective is to minimize the root-mean-squared error between the observations and corresponding forecast states. Consequently, the agent develops a correction strategy, enhancing model forecasts based on available system state observations. Our strategy employs a stochastic action policy, enabling a Monte Carlo-based DA framework that relies on randomly sampling the policy to generate an ensemble of assimilated realizations. Results demonstrate that the developed RL algorithm performs favorably when compared to the EnKF. Additionally, we illustrate the agent’s capability to assimilate non-Gaussian data, addressing a significant limitation of the EnKF.

Ensemble Kalman Filters (EnKFs), which assimilate observations based on statistics derived from samples of ocean states called ensemble, have become the norm for ocean data assimilation (DA) and forecasting. These schemes are commonly implemented with inflation and localization techniques to increase their ensemble spread and to filter out spurious long-range correlations resulting from the limited-size ensembles imposed by computational burden constraints. Such ad hoc methods were found not necessary in ensemble DA experiments with simplified ocean/atmospheric models and large ensembles. Here, we conduct a series of 1-year-long ensemble experiments with a fully realistic EnKF-DA system in the Red Sea using tens-to-thousands of ensemble members. The system assimilates satellite and in-situ observations and accounts for model uncertainties by integrating a 4km-resolution ocean model with ECMWF atmospheric ensemble fields, perturbed internal physics and initial conditions for forecasting. Our results indicate that accounting for model uncertainties is more beneficial than simply increasing the ensemble size, with the improvements due to large ensemble leveling off at about 250 members. Besides, and in contrast to what is commonly observed with simplified models, the investigated ensemble DA system still required localization even when implemented with thousands of members. These findings are explained by (i) amplified spurious long-range correlations produced by the low-rank nature of the ECMWF atmospheric forcing ensemble, and (ii) non-Gaussianity generated by the perturbed internal physical parameterization schemes. Large ensemble forcing fields and non-Gaussian DA methods might be needed to take full benefits from large ensembles in ocean DA.