Reinforcement Learning-Based Optimal Secret Protection in Discrete Event Systems

This paper proposes a reinforcement learning (RL)-based framework for optimizing secret protection policies in discrete event systems, aiming to minimize protection costs while satisfying multi-level security requirements of secret states. In a nondeterministic finite automaton model, several secret states are defined and require protection under varying confidentiality demands. To address this, a reward function is designed to couple protection effectiveness with cost efficiency, guiding the agent to autonomously discover the lowest-cost protection sequence through a Markov decision process formulation. Two representative RL algorithms are employed in the experiments: Q-learning, a value-based method, and Vanilla policy gradient (REINFORCE), a policy-based method. Experimental results demonstrate that the proposed framework can effectively adapt to diverse security requirements while achieving efficient and cost-effective secret protection. Compared with classical supervisory control theory approaches, the RL framework exhibits more intelligent and efficient exploration capability, making it feasible to derive protection strategies for large-scale automata with numerous states and transitions. Moreover, the proposed method eliminates the need to construct security automata, thereby simplifying the computation procedure and providing a lightweight methodological foundation for secure system design.