Control of a hybrid electric-hydrogen energy storage islanded DC microgrid based on soft actor-critic algorithm

To address bus voltage fluctuations and state-of-charge (SOC) constraint violations in hybrid electric-hydrogen energy storage systems (HEH-ESS) for islanded DC microgrids, this paper proposes an intelligent control strategy based on the Soft Actor-Critic (SAC) algorithm. The objective is to achieve long-term system stability and optimized energy scheduling. First, a detailed DC microgrid model is developed, incorporating photovoltaic (PV) panels, fuel cells, electrolyzers, hydrogen storage tanks, lithium-ion batteries, and DC loads. Second, a hierarchical control architecture is designed. The primary control layer utilizes droop control and Virtual Synchronous Generator (VSG) techniques to enable grid-forming capabilities. The secondary control layer implements an SAC-based reinforcement learning strategy, modeled as a Markov Decision Process (MDP), to optimize global power distribution. To enable coordinated voltage stabilization and SOC management, a dynamic, weighted single-objective reward function is formulated. This reward function integrates system constraints such as power limits, storage capacity, bus voltage range, and ramp rate, enhancing both the convergence speed and robustness of the control strategy. Simulation results validate the effectiveness of the proposed method. The SAC-based control significantly improves voltage stability, reduces the risks of battery overcharging and overdischarging, and enhances the system’s adaptability to complex load disturbances across diverse operational scenarios.