Physics-informed Actor-Critic for Coordination of Virtual Inertia from
Power Distribution Systems
Abstract
The vanishing inertia of synchronous generators in transmission systems
requires the utilization of renewables for inertial support. These are
often connected to the distribution system and their support should be
coordinated to avoid violation of grid limits. To this end, this paper
presents the Physics-informed Actor-Critic (PI-AC) algorithm for
coordination of Virtual Inertia (VI) from renewable Inverter-based
Resources (IBRs) in power distribution systems. Acquiring a model of the
distribution grid can be difficult, since certain parts are often
unknown or the parameters are highly uncertain. To favor model-free
coordination, Reinforcement Learning (RL) methods can be employed,
necessitating a substantial level of training beforehand. The PI-AC is a
RL algorithm that integrates the physical behavior of the power system
into the Actor-Critic (AC) approach in order to achieve faster learning.
To this end, we regularize the loss function with an aggregated power
system dynamics model based on the swing equation. Throughout this
paper, we explore the PI-AC functionality in a case study with the CIGRE
14-bus and IEEE 37-bus power distribution system in various grid
settings. The PI-AC is able to achieve better rewards and faster
learning than the exclusively data-driven AC algorithm and the
metaheuristic Genetic Algorithm (GA).