Deep Reinforcement Learning Enabled Inverters: Strengthening RES Integration in Grids with Electric Arc Furnaces

This paper presents development of a controller system for grid-supporting inverters to integrate renewable energy sources (RES) to the power grid for the challenging conditions of the existence of intermittent loads such as electric arc furnaces (EAFs). A deep-learning based method using Deep Deterministic Policy Gradient (DDPG), which is a Reinforcement Learning (RL) approach, is used for grid modeling, voltage and phase angle estimation, and control of the grid-supporting inverter. The goal is to develop a grid-supporting inverter which produces virtual inertia, stabilizes the grid frequency problems originating from intermittent loads and enables seamless integration of renewable energy sources (RES) to the power system. DDPG usage eliminates the need for some traditional estimation tools, such as Fast Fourier Transform (FFT), Synchronous Reference Frame (SRF) and low-pass filters, which are typically used methods for determining controller loop set-points. Moreover, with the proposed deep-learning based system, parameter tunings of the classical PID based controllers are avoided. The proposed system has been verified and tested in the simulation environment using actual field data collected at the transformer substations supplying EAF plants. It has been shown that the DDPG-based control system offers a fast and efficient control mechanism for maintaining the frequency stability of power systems. It is suggested that, this innovative approach can play a pivotal role in promoting the widespread adoption of RES to the system for the challenging conditions of intermittent loads.

Keyword: deep deterministic policy gradient (DDPG); electric arc furnace (EAF); grid-supporting inverter; power quality (PQ); power system frequency; power system inertia; reinforcement learning (RL); renewable energy sourses (RES); voltage fluctuation