文献类型：期刊文献

英文题名：Model-free distributed optimal control for general discrete-time linear systems using reinforcement learning

作者：Feng, Xinjun[1];Zhao, Zhiyun[1];Yang, Wen[1]

机构：[1]East China Univ Sci & Technol, Key Lab Smart Mfg Energy Chem Proc, Minist Educ, Shanghai 200237, Peoples R China

年份：2024

卷号：34

期号：9

起止页码：5570

外文期刊名：INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL

收录：;EI(收录号：20241115746213);WOS:【SCI-EXPANDED(收录号:WOS:001183630900001)】；

基金：This work was supported by the National Natural Science Foundation of China under Grants 61703162 and 62122026.

语种：英文

外文关键词：discrete systems; LQR; model-free; optimal control; Q-learning

摘要：This article proposes a novel data-driven framework of distributed optimal consensus for discrete-time linear multi-agent systems under general digraphs. A fully distributed control protocol is proposed by using linear quadratic regulator approach, which is proved to be a sufficient and necessary condition for optimal control of multi-agent systems through dynamic programming and minimum principle. Moreover, the control protocol can be constructed by using local information with the aid of the solution of the algebraic Riccati equation (ARE). Based on the Q-learning method, a reinforcement learning framework is presented to find the solution of the ARE in a data-driven way, in which we only need to collect information from an arbitrary follower to learn the feedback gain matrix. Thus, the multi-agent system can achieve distributed optimal consensus when system dynamics and global information are completely unavailable. For output feedback cases, accurate state information estimation is established such that optimal consensus control is realized. Moreover, the data-driven optimal consensus method designed in this article is applicable to general digraph that contains a directed spanning tree. Finally, numerical simulations verify the validity of the proposed optimal control protocols and data-driven framework.