文献类型：期刊文献

英文题名：A working memory model based on recurrent neural networks using reinforcement learning

作者：Wang, Mengyuan[1];Wang, Yihong[1];Xu, Xuying[1];Pan, Xiaochuan[1]

机构：[1]East China Univ Sci & Technol, Inst Cognit Neurodynam, Ctr Intelligent Comp, Sch Math, 130 Meilong Rd, Shanghai 200237, Peoples R China

年份：2024

卷号：18

期号：5

起止页码：3031

外文期刊名：COGNITIVE NEURODYNAMICS

收录：;WOS:【SCI-EXPANDED(收录号:WOS:001246003600001)】；

基金：This study was supported by the National Natural Science Foundation of China (Nos: 11972159, 12172132, 12272136, 12072113, 11802095).

语种：英文

外文关键词：Spatial working memory; Prefrontal cortex; Recurrent neural network; Reinforcement learning; Population coding

摘要：Numerous electrophysiological experiments have reported that the prefrontal cortex (PFC) is involved in the process of working memory. PFC neurons continue firing to maintain stimulus information in the delay period without external stimuli in working memory tasks. Further findings indicate that while the activity of single neurons exhibits strong temporal and spatial dynamics (heterogeneity), the activity of population neurons can encode spatiotemporal information of stimuli stably and reliably. From the perspective of neural networks, the computational mechanism underlying this phenomenon is not well demonstrated. The main purpose of this paper is to adopt a new strategy to explore the neural computation mechanism of working memory. We used reinforcement learning to train a recurrent neural network model to learn a spatial working memory task. The model is composed of a decision network and a baseline network. The decision network is responsible for updating strategies to make action choices, while the baseline network evaluates action choices to predict rewards. Simulated results demonstrate that the model can perform the spatial working memory task. The activity of the recurrent units has characteristics such as temporal dynamics and preferred direction selectivity, but their population activity encodes the stimulus information stably during the delay period in a low-dimensional subspace. These activity characteristics displayed by the model units are similar to those of PFC neurons observed in the same experiments. Meanwhile, as the network model continued learning the task, the temporal stability and spatial separability of the stimulus information encoded by the activity of model units in the low-dimensional subspace gradually strengthened, and the accuracy of the network's action choices also increased. In summary, this network model provides a new simulation method for spatial working memory tasks and a new perspective for understanding the characteristics of neuron activity in the PFC.