文献类型：期刊文献

英文题名：A Novel Hybrid-Action-Based Deep Reinforcement Learning for Industrial Energy Management

作者：Lu, Renzhi[1,2,3];Jiang, Zhenyu[4];Yang, Tao[5];Chen, Ying[6];Wang, Dong[7,8];Peng, Xin[9]

机构：[1]Huazhong Univ Sci & Technol, Engn Res Ctr Autonomous Intelligent Unmanned Syst, Sch Artificial Intelligence & Automat, Key Lab Image Proc & Intelligent Control, Wuhan 430074, Peoples R China;[2]Huazhong Univ Sci & Technol, Key Lab Syst Control & Informat Proc, Minist Educ, Shanghai 200240, Peoples R China;[3]Huazhong Univ Sci & Technol, Hubei Key Lab Adv Control & Intelligent Automat Co, Wuhan 430074, Peoples R China;[4]Huazhong Univ Sci & Technol, Sch Artificial Intelligence & Automat, Wuhan 430074, Peoples R China;[5]Northeastern Univ, State Key Lab Synthet Automat Proc Ind, Shenyang 110819, Peoples R China;[6]Tsinghua Univ, Elect Engn, Beijing 100084, Peoples R China;[7]Dalian Univ Technol, Key Lab Intelligent Control & Optimizat Ind Equipm, Minist Educ, Dalian 116024, Peoples R China;[8]Dalian Univ Technol, Sch Control Sci & Engn, Dalian 116024, Peoples R China;[9]East China Univ Sci & Technol, Key Lab Smart Mfg Energy Chem Proc, Minist Educ, Shanghai 200237, Peoples R China

年份：2024

卷号：20

期号：10

起止页码：12461

外文期刊名：IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

收录：;EI(收录号：20244317227028);WOS:【SCI-EXPANDED(收录号:WOS:001273004600001)】；

基金：This work was supported in part by the National Natural Science Foundation of China under Grant 62373158 and Grant 62133003, in part by the Natural Science Foundation of Hubei Province under Grant 2022CFB041, in part by Wuhan Science and Technology Innovation Special Project under Grant 2022010801020099, in part by the Key Laboratory of System Control and Information Processing under Grant Scip202211, in part by the Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems under Grant ACIA2022001, and in part by 111 Project under Grant B17040. paper no. TII-24-1249.

语种：英文

外文关键词：Energy management; Costs; Power generation; Renewable energy sources; Optimization; Load modeling; Uncertainty; Deep reinforcement learning (DRL); energy management; hybrid actions; industrial energy system

摘要：As environmental pollution becomes increasingly serious and industrial energy consumption continuously rises, an intelligent and efficient industrial energy management policy is urgently needed to reduce costs and maximize the benefits of industrial energy systems. However, modern industrial energy systems are characterized by hybrid industrial equipment actions, diverse objectives, and highly intermittent and stochastically distributed renewable energy sources. Therefore, efficient operation and control are difficult. This article presents a novel, model-free energy management policy using a hybrid action deep reinforcement learning algorithm for energy scheduling of industrial equipments operating in various modes. Specifically, the interaction process between the industrial energy management center and each equipment is modeled as a Markov decision process that minimizes the daily operating cost of the energy system and maximizes the revenue of the production equipment. Then, a double parameterized deep Q-networks that does not require an explicit environmental model is developed to learn the hybrid action signals using actor and critic networks, in which the double Q value mechanism avoids value overestimation and improves the algorithm efficiency. In addition, the policy gradient of the proposed algorithm is derived and its convergence proof is discussed. Finally, numerical studies are conducted using real-world data to evaluate algorithm performance and verify its effectiveness.