文献类型：期刊文献

英文题名：On the linear convergence of policy gradient under Hadamard parameterization

作者：Liu, Jiacai[1];Chen, Jinchi[2];Wei, Ke[1]

机构：[1]Fudan Univ, Sch Data Sci, Shanghai 200433, Peoples R China;[2]East China Univ Sci & Technol, Sch Math, Shanghai 200433, Peoples R China

年份：2025

卷号：14

期号：1

外文期刊名：INFORMATION AND INFERENCE-A JOURNAL OF THE IMA

收录：;WOS:【SCI-EXPANDED(收录号:WOS:001436474300001)】；

语种：英文

外文关键词：policy gradient; Hadamard parameterization; linear convergence; sub-optimal probability

摘要：The convergence of deterministic policy gradient under the Hadamard parameterization is studied in the tabular setting and the linear convergence of the algorithm is established. To this end, we first show that the error decreases at an $O(\frac{1}{k})$ rate for all the iterations. Based on this result, we further show that the algorithm has a faster local linear convergence rate after $k_{0}$ iterations, where $k_{0}$ is a constant that only depends on the MDP problem and the initialization. To show the local linear convergence of the algorithm, we have indeed established the contraction of the sub-optimal probability $b_{s}<^>{k}$ (i.e. the probability of the output policy $\pi <^>{k}$ on non-optimal actions) when $k\ge k_{0}$.