文献类型：期刊文献

英文题名：Safe Offline-to-Online Reinforcement Learning via Execution-Time Risk-Aware Selection for Industrial Process Control

作者：Luo, Na[1,2]; Chen, Jiyang[1,2]

机构：[1] Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, Shanghai, 200237, China; [2] School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

年份：2026

外文期刊名：SSRN

收录：EI(收录号：20260205197)

语种：英文

外文关键词：Accident prevention - Behavioral research - Constrained optimization - E-learning - Fault tolerance - Industrial management - Intelligent control - Lagrange multipliers - Markov processes - Process control - Reinforcement learning - Risk management - Risk perception

摘要：Industrial process control often needs online adaptation, but direct online exploration is costly and can trigger safety violations. We study a safe offline-toonline reinforcement learning (RL) method for low-fault-tolerance processes that adds execution-time risk control to our earlier backbone. Online finetuning is cast as an episodic constrained Markov decision process (CMDP) with an episode-level chance constraint and absorbing violation states. At execution time, Risk-Aware Selection (RAS) samples candidate actions from the actor and executes the one with the largest Lagrangian utility, computed from the reward critic and a conservative estimate of violation probability, while a single Lagrange multiplier is updated by a primal-dual rule. Experiments on penicillin fermentation and simulated moving-bed (SMB) separation show a better reward–safety trade-off during online fine-tuning without changing the underlying training backbone. The gain is larger in penicillin. In SMB, the same-backbone comparison also yields higher return and more stable rollouts. These results suggest that execution-time risk-aware selection can reduce online violation risk in industrial offline-to-online RL. ? 2026, The Authors. All rights reserved.